Jul 02 21:35:58.905539 ip-10-0-139-75 systemd[1]: Starting Kubernetes Kubelet... Jul 02 21:35:59.362164 ip-10-0-139-75 kubenswrapper[2569]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 21:35:59.362164 ip-10-0-139-75 kubenswrapper[2569]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 02 21:35:59.362164 ip-10-0-139-75 kubenswrapper[2569]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 21:35:59.362164 ip-10-0-139-75 kubenswrapper[2569]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 02 21:35:59.362164 ip-10-0-139-75 kubenswrapper[2569]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 21:35:59.362164 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.273255 2569 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 02 21:35:59.362164 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278052 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 21:35:59.362164 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278065 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 21:35:59.362164 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278069 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 21:35:59.362164 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278073 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 21:35:59.362164 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278076 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 21:35:59.362164 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278079 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 21:35:59.362164 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278083 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 21:35:59.362164 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278086 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 21:35:59.356996 ip-10-0-139-75 systemd[1]: Started Kubernetes Kubelet. Jul 02 21:35:59.363090 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278089 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 21:35:59.363090 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278092 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 21:35:59.363090 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278095 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 21:35:59.363090 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278097 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 21:35:59.363090 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278101 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 21:35:59.363090 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278104 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 21:35:59.363090 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278107 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 21:35:59.363090 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278110 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 21:35:59.363090 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278113 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 21:35:59.363090 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278115 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 21:35:59.363090 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278118 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 21:35:59.363090 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278121 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 21:35:59.363090 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278124 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 21:35:59.363090 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278127 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 21:35:59.363090 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278129 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 21:35:59.363090 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278132 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 21:35:59.363090 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278135 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 21:35:59.363090 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278138 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 21:35:59.363090 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278141 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 21:35:59.363090 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278145 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 21:35:59.363744 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278149 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 21:35:59.363744 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278152 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 21:35:59.363744 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278155 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 21:35:59.363744 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278157 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 21:35:59.363744 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278160 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 21:35:59.363744 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278162 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 21:35:59.363744 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278165 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 21:35:59.363744 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278168 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 21:35:59.363744 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278170 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 21:35:59.363744 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278179 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 21:35:59.363744 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278182 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 21:35:59.363744 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278185 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 21:35:59.363744 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278187 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 21:35:59.363744 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278190 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 21:35:59.363744 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278193 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 21:35:59.363744 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278195 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 21:35:59.363744 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278198 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 21:35:59.363744 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278201 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 21:35:59.363744 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278204 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 21:35:59.364424 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278207 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 21:35:59.364424 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278210 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 21:35:59.364424 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278213 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 21:35:59.364424 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278215 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 21:35:59.364424 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278218 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 21:35:59.364424 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278221 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 21:35:59.364424 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278223 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 21:35:59.364424 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278226 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 21:35:59.364424 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278229 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 21:35:59.364424 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278232 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 21:35:59.364424 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278235 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 21:35:59.364424 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278237 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 21:35:59.364424 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278240 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 21:35:59.364424 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278244 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 21:35:59.364424 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278247 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 21:35:59.364424 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278252 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 21:35:59.364424 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278255 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 21:35:59.364424 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278259 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 21:35:59.364424 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278262 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 21:35:59.365020 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278277 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 21:35:59.365020 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278280 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 21:35:59.365020 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278283 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 21:35:59.365020 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278286 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 21:35:59.365020 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278289 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 21:35:59.365020 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278291 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 21:35:59.365020 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278294 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 21:35:59.365020 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278296 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 02 21:35:59.365020 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278299 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 21:35:59.365020 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278302 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 21:35:59.365020 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278304 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 21:35:59.365020 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278307 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 21:35:59.365020 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278309 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 21:35:59.365020 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278312 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 21:35:59.365020 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278315 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 21:35:59.365020 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278318 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 21:35:59.365020 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278320 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 21:35:59.365020 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278323 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 21:35:59.365020 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278325 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 21:35:59.365020 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278328 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 21:35:59.365592 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278333 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 21:35:59.365592 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278336 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 21:35:59.365592 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278339 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 21:35:59.365592 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278342 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 21:35:59.365592 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278346 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 21:35:59.365592 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278349 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 21:35:59.365592 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278352 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 21:35:59.365592 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278355 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 21:35:59.365592 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278358 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 21:35:59.365592 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278360 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 21:35:59.365592 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278363 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 21:35:59.365592 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278366 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 21:35:59.365592 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.278368 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 21:35:59.365592 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279546 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 21:35:59.365592 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279554 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 21:35:59.365592 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279558 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 21:35:59.365592 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279561 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 21:35:59.365592 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279564 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 21:35:59.365592 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279569 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 21:35:59.366105 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279572 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 21:35:59.366105 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279576 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 21:35:59.366105 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279579 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 21:35:59.366105 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279581 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 21:35:59.366105 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279584 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 21:35:59.366105 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279588 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 21:35:59.366105 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279590 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 21:35:59.366105 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279594 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 21:35:59.366105 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279597 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 21:35:59.366105 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279599 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 21:35:59.366105 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279602 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 21:35:59.366105 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279605 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 21:35:59.366105 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279608 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 21:35:59.366105 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279611 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 21:35:59.366105 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279614 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 21:35:59.366105 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279617 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 21:35:59.366105 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279620 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 21:35:59.366105 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279623 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 21:35:59.366105 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279625 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 21:35:59.366105 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279628 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 21:35:59.366105 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279631 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 21:35:59.366768 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279634 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 21:35:59.366768 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279636 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 21:35:59.366768 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279639 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 21:35:59.366768 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279642 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 21:35:59.366768 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279645 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 21:35:59.366768 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279647 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 21:35:59.366768 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279649 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 21:35:59.366768 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279652 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 21:35:59.366768 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279655 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 21:35:59.366768 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279658 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 21:35:59.366768 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279661 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 21:35:59.366768 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279663 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 21:35:59.366768 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279668 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 21:35:59.366768 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279671 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 21:35:59.366768 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279675 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 21:35:59.366768 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279678 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 21:35:59.366768 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279680 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 21:35:59.366768 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279683 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 21:35:59.366768 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279685 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 21:35:59.367377 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279688 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 21:35:59.367377 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279691 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 21:35:59.367377 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279694 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 21:35:59.367377 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279696 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 21:35:59.367377 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279699 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 21:35:59.367377 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279701 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 21:35:59.367377 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279704 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 21:35:59.367377 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279706 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 21:35:59.367377 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279709 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 21:35:59.367377 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279711 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 21:35:59.367377 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279713 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 21:35:59.367377 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279716 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 21:35:59.367377 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279719 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 21:35:59.367377 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279721 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 21:35:59.367377 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279724 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 21:35:59.367377 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279726 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 21:35:59.367377 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279730 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 21:35:59.367377 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279733 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 21:35:59.367377 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279737 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 21:35:59.367377 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279741 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 21:35:59.368214 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279744 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 21:35:59.368214 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279746 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 21:35:59.368214 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279748 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 21:35:59.368214 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279751 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 21:35:59.368214 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279754 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 21:35:59.368214 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279757 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 21:35:59.368214 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279760 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 21:35:59.368214 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279763 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 21:35:59.368214 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279765 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 21:35:59.368214 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279768 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 21:35:59.368214 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279770 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 21:35:59.368214 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279773 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 21:35:59.368214 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279775 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 21:35:59.368214 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279779 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 21:35:59.368214 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279782 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 21:35:59.368214 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279785 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 21:35:59.368214 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279788 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 21:35:59.368214 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279791 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 21:35:59.368214 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279794 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 21:35:59.369135 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279796 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 21:35:59.369135 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279800 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 21:35:59.369135 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279802 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 21:35:59.369135 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279805 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 21:35:59.369135 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279807 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 21:35:59.369135 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279809 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 02 21:35:59.369135 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279812 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 21:35:59.369135 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279814 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 21:35:59.369135 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279817 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 21:35:59.369135 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279819 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 21:35:59.369135 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279822 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 21:35:59.369135 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279827 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 21:35:59.369135 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279829 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 21:35:59.369135 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.279832 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 21:35:59.369135 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280210 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 21:35:59.369135 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280216 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 21:35:59.369135 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280220 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 21:35:59.369135 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280223 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 21:35:59.369135 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280227 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 21:35:59.369135 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280230 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 21:35:59.369958 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280233 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 21:35:59.369958 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280236 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 21:35:59.369958 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280239 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 21:35:59.369958 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280242 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 21:35:59.369958 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280245 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 21:35:59.369958 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280247 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 21:35:59.369958 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280250 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 21:35:59.369958 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280253 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 21:35:59.369958 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280255 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 21:35:59.369958 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280258 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 21:35:59.369958 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280260 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 21:35:59.369958 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280263 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 21:35:59.369958 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280282 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 21:35:59.369958 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280285 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 21:35:59.369958 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280288 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 21:35:59.369958 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280291 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 21:35:59.369958 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280295 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 21:35:59.369958 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280299 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 21:35:59.369958 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280302 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 21:35:59.369958 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280304 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 21:35:59.371009 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280307 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 21:35:59.371009 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280310 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 21:35:59.371009 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280313 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 21:35:59.371009 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280318 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 21:35:59.371009 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280320 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 21:35:59.371009 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280323 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 21:35:59.371009 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280326 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 21:35:59.371009 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280328 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 21:35:59.371009 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280331 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 21:35:59.371009 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280333 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 21:35:59.371009 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280337 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 21:35:59.371009 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280340 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 21:35:59.371009 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280343 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 21:35:59.371009 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280345 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 02 21:35:59.371009 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280348 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 21:35:59.371009 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280350 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 21:35:59.371009 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280353 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 21:35:59.371009 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280356 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 21:35:59.371009 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280359 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 21:35:59.371009 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280361 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 21:35:59.371978 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280364 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 21:35:59.371978 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280366 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 21:35:59.371978 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280369 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 21:35:59.371978 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280371 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 21:35:59.371978 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280374 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 21:35:59.371978 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280377 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 21:35:59.371978 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280379 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 21:35:59.371978 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280382 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 21:35:59.371978 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280384 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 21:35:59.371978 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280387 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 21:35:59.371978 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280389 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 21:35:59.371978 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280392 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 21:35:59.371978 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280394 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 21:35:59.371978 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280397 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 21:35:59.371978 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280399 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 21:35:59.371978 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280404 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 21:35:59.371978 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280406 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 21:35:59.371978 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280409 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 21:35:59.371978 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280413 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 21:35:59.373162 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280416 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 21:35:59.373162 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280419 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 21:35:59.373162 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280422 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 21:35:59.373162 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280428 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 21:35:59.373162 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280431 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 21:35:59.373162 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280433 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 21:35:59.373162 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280436 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 21:35:59.373162 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280439 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 21:35:59.373162 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280441 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 21:35:59.373162 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280444 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 21:35:59.373162 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280447 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 21:35:59.373162 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280449 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 21:35:59.373162 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280454 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 21:35:59.373162 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280457 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 21:35:59.373162 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280460 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 21:35:59.373162 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280462 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 21:35:59.373162 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280465 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 21:35:59.373162 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280468 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 21:35:59.373162 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280470 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 21:35:59.374066 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280473 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 21:35:59.374066 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280475 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 21:35:59.374066 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280478 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 21:35:59.374066 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280481 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 21:35:59.374066 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280483 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 21:35:59.374066 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280486 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 21:35:59.374066 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280488 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 21:35:59.374066 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280491 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 21:35:59.374066 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280494 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 21:35:59.374066 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280498 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 21:35:59.374066 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280501 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 21:35:59.374066 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280504 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 21:35:59.374066 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280506 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 21:35:59.374066 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280509 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 21:35:59.374066 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.280512 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 21:35:59.374066 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280585 2569 flags.go:64] FLAG: --address="0.0.0.0" Jul 02 21:35:59.374066 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280596 2569 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 02 21:35:59.374066 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280603 2569 flags.go:64] FLAG: --anonymous-auth="true" Jul 02 21:35:59.374066 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280607 2569 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 02 21:35:59.374066 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280611 2569 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 02 21:35:59.374066 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280614 2569 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 02 21:35:59.374918 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280618 2569 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 02 21:35:59.374918 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280621 2569 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 02 21:35:59.374918 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280625 2569 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 02 21:35:59.374918 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280628 2569 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 02 21:35:59.374918 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280631 2569 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 02 21:35:59.374918 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280634 2569 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 02 21:35:59.374918 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280637 2569 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 02 21:35:59.374918 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280640 2569 flags.go:64] FLAG: --cgroup-root="" Jul 02 21:35:59.374918 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280643 2569 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 02 21:35:59.374918 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280646 2569 flags.go:64] FLAG: --client-ca-file="" Jul 02 21:35:59.374918 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280649 2569 flags.go:64] FLAG: --cloud-provider="external" Jul 02 21:35:59.374918 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280652 2569 flags.go:64] FLAG: --cluster-dns="[]" Jul 02 21:35:59.374918 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280656 2569 flags.go:64] FLAG: --cluster-domain="" Jul 02 21:35:59.374918 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280659 2569 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 02 21:35:59.374918 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280670 2569 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 02 21:35:59.374918 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280673 2569 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 02 21:35:59.374918 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280676 2569 flags.go:64] FLAG: --container-log-max-files="5" Jul 02 21:35:59.374918 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280682 2569 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 02 21:35:59.374918 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280685 2569 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 02 21:35:59.374918 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280688 2569 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 02 21:35:59.374918 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280693 2569 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 02 21:35:59.374918 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280696 2569 flags.go:64] FLAG: --contention-profiling="false" Jul 02 21:35:59.374918 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280700 2569 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 02 21:35:59.374918 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280703 2569 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 02 21:35:59.377895 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280706 2569 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 02 21:35:59.377895 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280709 2569 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 02 21:35:59.377895 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280713 2569 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 02 21:35:59.377895 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280717 2569 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 02 21:35:59.377895 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280720 2569 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 02 21:35:59.377895 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280723 2569 flags.go:64] FLAG: --enable-load-reader="false" Jul 02 21:35:59.377895 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280726 2569 flags.go:64] FLAG: --enable-server="true" Jul 02 21:35:59.377895 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280729 2569 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 02 21:35:59.377895 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280734 2569 flags.go:64] FLAG: --event-burst="100" Jul 02 21:35:59.377895 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280737 2569 flags.go:64] FLAG: --event-qps="50" Jul 02 21:35:59.377895 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280739 2569 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 02 21:35:59.377895 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280742 2569 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 02 21:35:59.377895 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280745 2569 flags.go:64] FLAG: --eviction-hard="" Jul 02 21:35:59.377895 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280748 2569 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 02 21:35:59.377895 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280751 2569 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 02 21:35:59.377895 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280754 2569 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 02 21:35:59.377895 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280757 2569 flags.go:64] FLAG: --eviction-soft="" Jul 02 21:35:59.377895 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280760 2569 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 02 21:35:59.377895 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280763 2569 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 02 21:35:59.377895 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280766 2569 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 02 21:35:59.377895 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280769 2569 flags.go:64] FLAG: --experimental-mounter-path="" Jul 02 21:35:59.377895 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280771 2569 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 02 21:35:59.377895 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280774 2569 flags.go:64] FLAG: --fail-swap-on="true" Jul 02 21:35:59.377895 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280777 2569 flags.go:64] FLAG: --feature-gates="" Jul 02 21:35:59.377895 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280780 2569 flags.go:64] FLAG: --file-check-frequency="20s" Jul 02 21:35:59.378819 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280784 2569 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 02 21:35:59.378819 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280787 2569 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 02 21:35:59.378819 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280789 2569 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 02 21:35:59.378819 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280794 2569 flags.go:64] FLAG: --healthz-port="10248" Jul 02 21:35:59.378819 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280797 2569 flags.go:64] FLAG: --help="false" Jul 02 21:35:59.378819 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280800 2569 flags.go:64] FLAG: --hostname-override="ip-10-0-139-75.ec2.internal" Jul 02 21:35:59.378819 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280804 2569 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 02 21:35:59.378819 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280807 2569 flags.go:64] FLAG: --http-check-frequency="20s" Jul 02 21:35:59.378819 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280810 2569 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 02 21:35:59.378819 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280813 2569 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 02 21:35:59.378819 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280818 2569 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 02 21:35:59.378819 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280821 2569 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 02 21:35:59.378819 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280823 2569 flags.go:64] FLAG: --image-service-endpoint="" Jul 02 21:35:59.378819 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280826 2569 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 02 21:35:59.378819 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280829 2569 flags.go:64] FLAG: --kube-api-burst="100" Jul 02 21:35:59.378819 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280832 2569 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 02 21:35:59.378819 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280835 2569 flags.go:64] FLAG: --kube-api-qps="50" Jul 02 21:35:59.378819 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280838 2569 flags.go:64] FLAG: --kube-reserved="" Jul 02 21:35:59.378819 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280841 2569 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 02 21:35:59.378819 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280843 2569 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 02 21:35:59.378819 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280847 2569 flags.go:64] FLAG: --kubelet-cgroups="" Jul 02 21:35:59.378819 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280849 2569 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 02 21:35:59.378819 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280852 2569 flags.go:64] FLAG: --lock-file="" Jul 02 21:35:59.378819 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280855 2569 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 02 21:35:59.519683 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280858 2569 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 02 21:35:59.519683 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280860 2569 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 02 21:35:59.519683 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280865 2569 flags.go:64] FLAG: --log-json-split-stream="false" Jul 02 21:35:59.519683 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280868 2569 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 02 21:35:59.519683 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280871 2569 flags.go:64] FLAG: --log-text-split-stream="false" Jul 02 21:35:59.519683 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280874 2569 flags.go:64] FLAG: --logging-format="text" Jul 02 21:35:59.519683 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280877 2569 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 02 21:35:59.519683 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280880 2569 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 02 21:35:59.519683 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280883 2569 flags.go:64] FLAG: --manifest-url="" Jul 02 21:35:59.519683 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280886 2569 flags.go:64] FLAG: --manifest-url-header="" Jul 02 21:35:59.519683 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280890 2569 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 02 21:35:59.519683 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280895 2569 flags.go:64] FLAG: --max-open-files="1000000" Jul 02 21:35:59.519683 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280899 2569 flags.go:64] FLAG: --max-pods="110" Jul 02 21:35:59.519683 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280902 2569 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 02 21:35:59.519683 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280905 2569 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 02 21:35:59.519683 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280908 2569 flags.go:64] FLAG: --memory-manager-policy="None" Jul 02 21:35:59.519683 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280911 2569 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 02 21:35:59.519683 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280914 2569 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 02 21:35:59.519683 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280918 2569 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 02 21:35:59.519683 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280921 2569 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 02 21:35:59.519683 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280927 2569 flags.go:64] FLAG: --node-status-max-images="50" Jul 02 21:35:59.519683 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280930 2569 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 02 21:35:59.519683 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280933 2569 flags.go:64] FLAG: --oom-score-adj="-999" Jul 02 21:35:59.519683 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280936 2569 flags.go:64] FLAG: --pod-cidr="" Jul 02 21:35:59.543365 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280939 2569 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 02 21:35:59.543365 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280944 2569 flags.go:64] FLAG: --pod-manifest-path="" Jul 02 21:35:59.543365 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280947 2569 flags.go:64] FLAG: --pod-max-pids="-1" Jul 02 21:35:59.543365 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280950 2569 flags.go:64] FLAG: --pods-per-core="0" Jul 02 21:35:59.543365 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280953 2569 flags.go:64] FLAG: --port="10250" Jul 02 21:35:59.543365 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280956 2569 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 02 21:35:59.543365 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280959 2569 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-018c00720ce26d131" Jul 02 21:35:59.543365 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280962 2569 flags.go:64] FLAG: --qos-reserved="" Jul 02 21:35:59.543365 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280965 2569 flags.go:64] FLAG: --read-only-port="10255" Jul 02 21:35:59.543365 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280968 2569 flags.go:64] FLAG: --register-node="true" Jul 02 21:35:59.543365 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280971 2569 flags.go:64] FLAG: --register-with-taints="" Jul 02 21:35:59.543365 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280975 2569 flags.go:64] FLAG: --registry-burst="10" Jul 02 21:35:59.543365 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280978 2569 flags.go:64] FLAG: --registry-qps="5" Jul 02 21:35:59.543365 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280980 2569 flags.go:64] FLAG: --reserved-cpus="" Jul 02 21:35:59.543365 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280983 2569 flags.go:64] FLAG: --reserved-memory="" Jul 02 21:35:59.543365 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280987 2569 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 02 21:35:59.543365 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280990 2569 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 02 21:35:59.543365 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280993 2569 flags.go:64] FLAG: --rotate-certificates="false" Jul 02 21:35:59.543365 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.280995 2569 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 02 21:35:59.543365 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.281000 2569 flags.go:64] FLAG: --runonce="false" Jul 02 21:35:59.543365 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.281002 2569 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 02 21:35:59.543365 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.281005 2569 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 02 21:35:59.543365 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.281008 2569 flags.go:64] FLAG: --seccomp-default="false" Jul 02 21:35:59.543365 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.281011 2569 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 02 21:35:59.543365 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.281015 2569 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 02 21:35:59.544135 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.281018 2569 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 02 21:35:59.544135 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.281021 2569 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 02 21:35:59.544135 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.281025 2569 flags.go:64] FLAG: --storage-driver-password="root" Jul 02 21:35:59.544135 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.281028 2569 flags.go:64] FLAG: --storage-driver-secure="false" Jul 02 21:35:59.544135 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.281031 2569 flags.go:64] FLAG: --storage-driver-table="stats" Jul 02 21:35:59.544135 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.281034 2569 flags.go:64] FLAG: --storage-driver-user="root" Jul 02 21:35:59.544135 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.281037 2569 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 02 21:35:59.544135 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.281040 2569 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 02 21:35:59.544135 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.281043 2569 flags.go:64] FLAG: --system-cgroups="" Jul 02 21:35:59.544135 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.281045 2569 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 02 21:35:59.544135 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.281050 2569 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 02 21:35:59.544135 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.281053 2569 flags.go:64] FLAG: --tls-cert-file="" Jul 02 21:35:59.544135 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.281056 2569 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 02 21:35:59.544135 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.281059 2569 flags.go:64] FLAG: --tls-min-version="" Jul 02 21:35:59.544135 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.281062 2569 flags.go:64] FLAG: --tls-private-key-file="" Jul 02 21:35:59.544135 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.281065 2569 flags.go:64] FLAG: --topology-manager-policy="none" Jul 02 21:35:59.544135 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.281068 2569 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 02 21:35:59.544135 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.281070 2569 flags.go:64] FLAG: --topology-manager-scope="container" Jul 02 21:35:59.544135 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.281073 2569 flags.go:64] FLAG: --v="2" Jul 02 21:35:59.544135 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.281077 2569 flags.go:64] FLAG: --version="false" Jul 02 21:35:59.544135 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.281080 2569 flags.go:64] FLAG: --vmodule="" Jul 02 21:35:59.544135 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.281084 2569 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 02 21:35:59.544135 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.281087 2569 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 02 21:35:59.544135 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281193 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 21:35:59.544135 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281196 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 21:35:59.544927 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281198 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 21:35:59.544927 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281203 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 21:35:59.544927 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281205 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 21:35:59.544927 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281208 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 21:35:59.544927 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281210 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 21:35:59.544927 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281213 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 21:35:59.544927 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281216 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 21:35:59.544927 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281219 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 21:35:59.544927 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281221 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 21:35:59.544927 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281225 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 21:35:59.544927 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281228 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 21:35:59.544927 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281231 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 21:35:59.544927 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281235 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 21:35:59.544927 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281238 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 21:35:59.544927 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281241 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 21:35:59.544927 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281244 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 21:35:59.544927 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281247 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 21:35:59.544927 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281250 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 21:35:59.544927 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281253 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 21:35:59.545737 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281256 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 21:35:59.545737 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281258 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 21:35:59.545737 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281261 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 21:35:59.545737 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281264 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 21:35:59.545737 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281281 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 21:35:59.545737 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281283 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 21:35:59.545737 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281286 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 21:35:59.545737 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281289 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 21:35:59.545737 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281292 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 21:35:59.545737 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281295 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 21:35:59.545737 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281297 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 21:35:59.545737 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281300 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 21:35:59.545737 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281302 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 21:35:59.545737 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281305 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 21:35:59.545737 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281310 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 21:35:59.545737 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281313 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 02 21:35:59.545737 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281316 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 21:35:59.545737 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281319 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 21:35:59.545737 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281322 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 21:35:59.546625 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281324 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 21:35:59.546625 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281327 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 21:35:59.546625 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281330 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 21:35:59.546625 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281334 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 21:35:59.546625 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281336 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 21:35:59.546625 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281339 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 21:35:59.546625 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281342 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 21:35:59.546625 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281345 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 21:35:59.546625 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281347 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 21:35:59.546625 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281350 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 21:35:59.546625 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281352 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 21:35:59.546625 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281355 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 21:35:59.546625 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281357 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 21:35:59.546625 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281360 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 21:35:59.546625 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281363 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 21:35:59.546625 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281365 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 21:35:59.546625 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281368 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 21:35:59.546625 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281370 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 21:35:59.546625 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281373 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 21:35:59.546625 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281376 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 21:35:59.547480 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281378 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 21:35:59.547480 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281381 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 21:35:59.547480 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281383 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 21:35:59.547480 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281386 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 21:35:59.547480 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281388 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 21:35:59.547480 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281391 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 21:35:59.547480 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281394 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 21:35:59.547480 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281397 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 21:35:59.547480 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281400 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 21:35:59.547480 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281403 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 21:35:59.547480 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281405 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 21:35:59.547480 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281408 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 21:35:59.547480 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281410 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 21:35:59.547480 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281413 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 21:35:59.547480 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281415 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 21:35:59.547480 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281423 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 21:35:59.547480 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281425 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 21:35:59.547480 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281428 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 21:35:59.547480 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281431 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 21:35:59.547480 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281433 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 21:35:59.548708 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281436 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 21:35:59.548708 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281439 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 21:35:59.548708 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281441 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 21:35:59.548708 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281444 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 21:35:59.548708 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281447 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 21:35:59.548708 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281449 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 21:35:59.548708 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281452 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 21:35:59.548708 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281454 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 21:35:59.548708 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281457 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 21:35:59.548708 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281459 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 21:35:59.548708 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281462 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 21:35:59.548708 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281464 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 21:35:59.548708 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281467 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 21:35:59.548708 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281469 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 21:35:59.548708 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281472 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 21:35:59.548708 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281475 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 21:35:59.548708 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281477 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 21:35:59.548708 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281480 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 21:35:59.548708 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.281484 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 21:35:59.549530 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.282045 2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 21:35:59.549530 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.292398 2569 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 02 21:35:59.549530 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.292413 2569 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 02 21:35:59.549530 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.292432 2569 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 02 21:35:59.549530 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.292438 2569 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 02 21:35:59.549530 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292505 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 21:35:59.549530 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292510 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 21:35:59.549530 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292513 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 21:35:59.549530 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292517 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 02 21:35:59.549530 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292520 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 21:35:59.549530 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292523 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 21:35:59.549530 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292526 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 21:35:59.549530 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292529 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 21:35:59.549530 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292532 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 21:35:59.549530 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292535 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 21:35:59.550339 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292538 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 21:35:59.550339 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292540 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 21:35:59.550339 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292543 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 21:35:59.550339 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292546 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 21:35:59.550339 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292549 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 21:35:59.550339 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292551 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 21:35:59.550339 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292554 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 21:35:59.550339 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292557 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 21:35:59.550339 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292560 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 21:35:59.550339 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292563 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 21:35:59.550339 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292566 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 21:35:59.550339 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292569 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 21:35:59.550339 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292572 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 21:35:59.550339 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292574 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 21:35:59.550339 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292577 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 21:35:59.550339 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292580 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 21:35:59.550339 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292582 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 21:35:59.550339 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292585 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 21:35:59.550339 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292588 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 21:35:59.550339 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292591 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 21:35:59.551158 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292594 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 21:35:59.551158 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292596 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 21:35:59.551158 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292599 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 21:35:59.551158 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292602 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 21:35:59.551158 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292606 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 21:35:59.551158 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292610 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 21:35:59.551158 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292613 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 21:35:59.551158 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292616 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 21:35:59.551158 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292619 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 21:35:59.551158 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292622 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 21:35:59.551158 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292625 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 21:35:59.551158 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292627 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 21:35:59.551158 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292630 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 21:35:59.551158 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292633 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 21:35:59.551158 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292636 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 21:35:59.551158 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292638 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 21:35:59.551158 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292643 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 21:35:59.551158 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292646 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 21:35:59.551158 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292649 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 21:35:59.551949 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292652 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 21:35:59.551949 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292655 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 21:35:59.551949 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292658 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 21:35:59.551949 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292661 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 21:35:59.551949 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292664 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 21:35:59.551949 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292667 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 21:35:59.551949 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292670 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 21:35:59.551949 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292673 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 21:35:59.551949 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292676 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 21:35:59.551949 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292679 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 21:35:59.551949 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292682 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 21:35:59.551949 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292685 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 21:35:59.551949 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292688 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 21:35:59.551949 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292691 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 21:35:59.551949 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292694 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 21:35:59.551949 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292696 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 21:35:59.551949 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292699 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 21:35:59.551949 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292702 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 21:35:59.551949 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292705 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 21:35:59.551949 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292707 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 21:35:59.552782 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292710 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 21:35:59.552782 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292712 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 21:35:59.552782 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292716 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 21:35:59.552782 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292720 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 21:35:59.552782 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292723 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 21:35:59.552782 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292726 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 21:35:59.552782 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292728 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 21:35:59.552782 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292731 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 21:35:59.552782 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292733 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 21:35:59.552782 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292736 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 21:35:59.552782 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292739 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 21:35:59.552782 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292741 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 21:35:59.552782 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292744 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 21:35:59.552782 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292746 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 21:35:59.552782 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292749 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 21:35:59.552782 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292751 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 21:35:59.552782 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292757 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 21:35:59.552782 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292760 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 21:35:59.552782 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292762 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 21:35:59.553603 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292765 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 21:35:59.553603 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292768 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 21:35:59.553603 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292770 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 21:35:59.553603 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292773 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 21:35:59.553603 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292777 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 21:35:59.553603 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292780 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 21:35:59.553603 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292782 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 21:35:59.553603 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292785 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 21:35:59.553603 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292788 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 21:35:59.553603 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292790 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 21:35:59.553603 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292793 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 21:35:59.553603 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.292798 2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 21:35:59.553603 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292912 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 21:35:59.553603 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292917 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 21:35:59.553603 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292920 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 21:35:59.554824 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292924 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 21:35:59.554824 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292926 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 21:35:59.554824 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292929 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 21:35:59.554824 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292932 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 21:35:59.554824 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292934 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 21:35:59.554824 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292937 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 21:35:59.554824 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292940 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 21:35:59.554824 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292942 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 21:35:59.554824 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292945 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 21:35:59.554824 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292949 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 21:35:59.554824 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292953 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 21:35:59.554824 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292956 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 21:35:59.554824 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292959 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 21:35:59.554824 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292961 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 21:35:59.554824 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292964 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 21:35:59.554824 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292968 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 21:35:59.554824 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292971 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 21:35:59.554824 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292973 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 21:35:59.554824 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292976 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 21:35:59.554824 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292979 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 21:35:59.555622 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292981 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 21:35:59.555622 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292985 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 21:35:59.555622 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292988 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 21:35:59.555622 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292990 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 21:35:59.555622 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292993 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 21:35:59.555622 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292996 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 21:35:59.555622 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.292998 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 21:35:59.555622 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293001 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 21:35:59.555622 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293004 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 21:35:59.555622 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293007 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 21:35:59.555622 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293010 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 21:35:59.555622 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293012 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 21:35:59.555622 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293015 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 21:35:59.555622 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293017 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 21:35:59.555622 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293020 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 21:35:59.555622 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293022 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 21:35:59.555622 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293025 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 21:35:59.555622 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293027 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 21:35:59.555622 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293030 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 21:35:59.556449 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293034 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 21:35:59.556449 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293038 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 21:35:59.556449 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293041 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 21:35:59.556449 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293044 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 21:35:59.556449 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293047 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 21:35:59.556449 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293049 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 21:35:59.556449 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293052 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 02 21:35:59.556449 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293054 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 21:35:59.556449 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293058 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 21:35:59.556449 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293061 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 21:35:59.556449 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293063 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 21:35:59.556449 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293066 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 21:35:59.556449 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293069 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 21:35:59.556449 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293071 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 21:35:59.556449 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293075 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 21:35:59.556449 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293078 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 21:35:59.556449 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293081 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 21:35:59.556449 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293083 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 21:35:59.556449 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293086 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 21:35:59.557230 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293088 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 21:35:59.557230 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293091 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 21:35:59.557230 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293094 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 21:35:59.557230 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293096 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 21:35:59.557230 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293099 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 21:35:59.557230 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293101 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 21:35:59.557230 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293104 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 21:35:59.557230 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293106 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 21:35:59.557230 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293109 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 21:35:59.557230 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293111 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 21:35:59.557230 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293114 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 21:35:59.557230 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293116 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 21:35:59.557230 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293132 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 21:35:59.557230 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293136 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 21:35:59.557230 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293138 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 21:35:59.557230 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293142 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 21:35:59.557230 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293145 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 21:35:59.557230 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293148 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 21:35:59.557230 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293150 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 21:35:59.558050 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293153 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 21:35:59.558050 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293156 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 21:35:59.558050 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293159 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 21:35:59.558050 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293162 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 21:35:59.558050 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293164 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 21:35:59.558050 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293167 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 21:35:59.558050 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293170 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 21:35:59.558050 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293173 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 21:35:59.558050 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293177 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 21:35:59.558050 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293180 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 21:35:59.558050 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293182 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 21:35:59.558050 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293185 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 21:35:59.558050 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293188 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 21:35:59.558050 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293191 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 21:35:59.558050 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293194 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 21:35:59.558050 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293198 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 21:35:59.558050 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293201 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 21:35:59.558050 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293204 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 21:35:59.558050 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:35:59.293208 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 21:35:59.558858 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.293212 2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 21:35:59.558858 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.293351 2569 server.go:962] "Client rotation is on, will bootstrap in background" Jul 02 21:35:59.558858 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.296806 2569 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 02 21:35:59.558858 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.298178 2569 server.go:1019] "Starting client certificate rotation" Jul 02 21:35:59.558858 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.298290 2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 02 21:35:59.558858 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.298320 2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 02 21:35:59.558858 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.300547 2569 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 02 21:35:59.558858 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.305202 2569 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 02 21:35:59.558858 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.317943 2569 log.go:25] "Validated CRI v1 runtime API" Jul 02 21:35:59.558858 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.319934 2569 log.go:25] "Validated CRI v1 image API" Jul 02 21:35:59.558858 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.320655 2569 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 02 21:35:59.558858 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.325596 2569 fs.go:135] Filesystem UUIDs: map[513ce2dc-a7a5-4236-b4f6-052e367664f5:/dev/nvme0n1p4 625b2e3e-3feb-4cbb-8b96-9f4a04bb00d4:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jul 02 21:35:59.564617 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.325609 2569 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 02 21:35:59.564617 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.328688 2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 02 21:36:01.318802 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.330374 2569 manager.go:216] Machine: {Timestamp:2026-07-02 21:35:59.329631962 +0000 UTC m=+0.312120522 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102793 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec270934e20fa1a16a3867b73747e24a SystemUUID:ec270934-e20f-a1a1-6a38-67b73747e24a BootID:4e7b6180-439d-4f49-b463-62c1370fa03f Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:97:05:71:7d:31 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:97:05:71:7d:31 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ca:62:30:95:79:70 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 02 21:36:01.318802 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.330477 2569 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 02 21:36:01.318802 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.330545 2569 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 02 21:36:01.318802 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.332970 2569 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 02 21:36:01.318802 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.332992 2569 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-75.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 02 21:36:01.318802 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.333712 2569 topology_manager.go:138] "Creating topology manager with none policy" Jul 02 21:36:01.318802 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.333721 2569 container_manager_linux.go:309] "Creating device plugin manager" Jul 02 21:36:01.318802 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.333729 2569 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 02 21:36:01.318802 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.334381 2569 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 02 21:36:01.318802 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.336198 2569 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 02 21:36:01.318802 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.340609 2569 state_mem.go:36] "Initialized new in-memory state store" Jul 02 21:36:01.318802 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.340719 2569 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 02 21:36:01.318802 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.340744 2569 kubelet.go:486] "Attempting to sync node with API server" Jul 02 21:36:01.318802 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.340754 2569 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 02 21:36:01.318802 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.340770 2569 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 02 21:36:01.318802 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.340779 2569 kubelet.go:398] "Adding apiserver pod source" Jul 02 21:36:01.318802 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.340794 2569 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 02 21:36:01.318802 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.341070 2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 02 21:36:01.320094 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.341085 2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 02 21:36:01.320094 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.348778 2569 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 02 21:36:01.320094 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:35:59.349657 2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-75.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 02 21:36:01.320094 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:35:59.349829 2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 02 21:36:01.320094 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.351006 2569 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 02 21:36:01.320094 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.351033 2569 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 02 21:36:01.320094 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.352394 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 02 21:36:01.320094 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.352408 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 02 21:36:01.320094 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.352414 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 02 21:36:01.320094 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.352419 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 02 21:36:01.320094 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.352429 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 02 21:36:01.320094 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.352440 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 02 21:36:01.320094 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.352449 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 02 21:36:01.320094 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.352454 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 02 21:36:01.320094 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.352461 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 02 21:36:01.320094 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.352466 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 02 21:36:01.320094 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.352474 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 02 21:36:01.320094 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.352479 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 02 21:36:01.320094 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.353344 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 02 21:36:01.320094 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.353354 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 02 21:36:01.320094 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.355288 2569 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-75.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 02 21:36:01.320094 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.356219 2569 server.go:1268] "Started kubelet" Jul 02 21:36:01.320094 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.356362 2569 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 02 21:36:01.320094 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.356453 2569 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 02 21:36:01.320094 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.356913 2569 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 02 21:36:01.320094 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.358287 2569 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 02 21:36:01.320094 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.358687 2569 server.go:310] "Adding debug handlers to kubelet server" Jul 02 21:36:01.320094 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:35:59.364329 2569 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 02 21:36:01.320094 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.367152 2569 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 02 21:36:01.320094 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.367163 2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 02 21:36:01.320094 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:35:59.367224 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-75.ec2.internal\" not found" Jul 02 21:36:01.320094 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.367819 2569 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 02 21:36:01.320094 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.367822 2569 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 02 21:36:01.320094 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.367867 2569 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 02 21:36:01.320094 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.368036 2569 reconstruct.go:97] "Volume reconstruction finished" Jul 02 21:36:01.320094 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.368047 2569 reconciler.go:29] "Reconciler: start to sync state" Jul 02 21:36:01.320094 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.368184 2569 factory.go:55] Registering systemd factory Jul 02 21:36:01.320094 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.368198 2569 factory.go:223] Registration of the systemd container factory successfully Jul 02 21:36:01.320094 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.368467 2569 factory.go:153] Registering CRI-O factory Jul 02 21:36:01.320094 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.368487 2569 factory.go:223] Registration of the crio container factory successfully Jul 02 21:36:01.320094 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.368543 2569 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 02 21:36:01.320094 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.368570 2569 factory.go:103] Registering Raw factory Jul 02 21:36:01.320094 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.368584 2569 manager.go:1196] Started watching for new ooms in manager Jul 02 21:36:01.320094 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.369031 2569 manager.go:349] Starting recovery of all containers Jul 02 21:36:01.320094 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.376921 2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xlpmw" Jul 02 21:36:01.320094 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.383408 2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xlpmw" Jul 02 21:36:01.321825 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:35:59.384718 2569 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-139-75.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 02 21:36:01.321825 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:35:59.384934 2569 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 02 21:36:01.321825 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:35:59.385045 2569 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-75.ec2.internal.18be96a9fef1bc85 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-75.ec2.internal,UID:ip-10-0-139-75.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-75.ec2.internal,},FirstTimestamp:2026-07-02 21:35:59.356189829 +0000 UTC m=+0.338678389,LastTimestamp:2026-07-02 21:35:59.356189829 +0000 UTC m=+0.338678389,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-75.ec2.internal,}" Jul 02 21:36:01.321825 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.391646 2569 manager.go:354] Recovery completed Jul 02 21:36:01.321825 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.395891 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 21:36:01.321825 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.397872 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-75.ec2.internal" event="NodeHasSufficientMemory" Jul 02 21:36:01.321825 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.397913 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-75.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 21:36:01.321825 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.397925 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-75.ec2.internal" event="NodeHasSufficientPID" Jul 02 21:36:01.321825 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.398423 2569 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 02 21:36:01.321825 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.398434 2569 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 02 21:36:01.321825 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.398450 2569 state_mem.go:36] "Initialized new in-memory state store" Jul 02 21:36:01.321825 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.398499 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 21:36:01.321825 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.399094 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-75.ec2.internal" event="NodeHasSufficientMemory" Jul 02 21:36:01.321825 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.399115 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-75.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 21:36:01.321825 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.399125 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-75.ec2.internal" event="NodeHasSufficientPID" Jul 02 21:36:01.321825 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:35:59.399859 2569 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-75.ec2.internal.18be96aa016e28e3 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-75.ec2.internal,UID:ip-10-0-139-75.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-139-75.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-139-75.ec2.internal,},FirstTimestamp:2026-07-02 21:35:59.397898467 +0000 UTC m=+0.380387034,LastTimestamp:2026-07-02 21:35:59.397898467 +0000 UTC m=+0.380387034,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-75.ec2.internal,}" Jul 02 21:36:01.321825 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.400576 2569 policy_none.go:49] "None policy: Start" Jul 02 21:36:01.321825 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.400589 2569 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 02 21:36:01.321825 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.400599 2569 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 02 21:36:01.321825 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.402539 2569 policy_none.go:47] "Start" Jul 02 21:36:01.321825 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.444774 2569 manager.go:337] "Starting Device Plugin manager" Jul 02 21:36:01.321825 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:35:59.444903 2569 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 02 21:36:01.321825 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.444912 2569 server.go:87] "Starting device plugin registration server" Jul 02 21:36:01.321825 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.445233 2569 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 02 21:36:01.321825 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.445244 2569 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 02 21:36:01.321825 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.445390 2569 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 02 21:36:01.321825 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.445473 2569 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 02 21:36:01.321825 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.445482 2569 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 02 21:36:01.321825 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:35:59.445807 2569 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 02 21:36:01.321825 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:35:59.445851 2569 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-75.ec2.internal\" not found" Jul 02 21:36:01.321825 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.545783 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 21:36:01.321825 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.546866 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-75.ec2.internal" event="NodeHasSufficientMemory" Jul 02 21:36:01.323147 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.546896 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-75.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 21:36:01.323147 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.546911 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-75.ec2.internal" event="NodeHasSufficientPID" Jul 02 21:36:01.323147 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.546946 2569 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-75.ec2.internal" Jul 02 21:36:01.323147 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:35:59.556113 2569 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-75.ec2.internal" Jul 02 21:36:01.323147 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:35:59.556131 2569 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-75.ec2.internal\": node \"ip-10-0-139-75.ec2.internal\" not found" Jul 02 21:36:01.323147 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:35:59.601874 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-75.ec2.internal\" not found" Jul 02 21:36:01.450075 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:01.345724 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-75.ec2.internal\" not found" Jul 02 21:36:01.450075 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:01.446172 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-75.ec2.internal\" not found" Jul 02 21:36:01.564402 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:01.546730 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-75.ec2.internal\" not found" Jul 02 21:36:01.700117 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:01.647170 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-75.ec2.internal\" not found" Jul 02 21:36:01.814227 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:01.747673 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-75.ec2.internal\" not found" Jul 02 21:36:01.931587 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:01.848102 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-75.ec2.internal\" not found" Jul 02 21:36:01.942543 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:01.941591 2569 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 02 21:36:02.047378 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:01.948875 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-75.ec2.internal\" not found" Jul 02 21:36:02.047378 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:01.959961 2569 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 21:36:02.047378 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:02.031555 2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-01 21:31:01 +0000 UTC" deadline="2028-02-04 02:15:12.01498184 +0000 UTC" Jul 02 21:36:02.047378 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:02.031583 2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13948h39m9.983401563s" Jul 02 21:36:02.051449 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:02.049683 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-75.ec2.internal\" not found" Jul 02 21:36:02.164922 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:02.150261 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-75.ec2.internal\" not found" Jul 02 21:36:02.283413 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:02.209126 2569 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 21:36:02.283413 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:02.218449 2569 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 21:36:02.948160 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:02.948118 2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 02 21:36:02.949399 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:02.949382 2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 02 21:36:02.949463 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:02.949432 2569 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 02 21:36:02.949463 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:02.949458 2569 kubelet.go:2448] "Starting kubelet main sync loop" Jul 02 21:36:02.949560 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:02.949493 2569 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 02 21:36:02.951869 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:02.951841 2569 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 21:36:02.985007 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:02.984968 2569 apiserver.go:52] "Watching apiserver" Jul 02 21:36:02.994903 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:02.994884 2569 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 02 21:36:03.011352 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.011325 2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 02 21:36:03.022974 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.022923 2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 02 21:36:03.046468 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.046446 2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-p92sx" Jul 02 21:36:03.051291 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.050528 2569 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-75.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-75.ec2.internal"] Jul 02 21:36:03.053481 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.053464 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-75.ec2.internal" Jul 02 21:36:03.054581 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.054563 2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-p92sx" Jul 02 21:36:03.054969 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.054948 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-bf9dd","openshift-multus/network-metrics-daemon-cn45l","openshift-network-diagnostics/network-check-target-jgn9h","kube-system/konnectivity-agent-mw4n5","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khnpz","openshift-cluster-node-tuning-operator/tuned-zvgzv","openshift-image-registry/node-ca-4rgsb","openshift-multus/multus-additional-cni-plugins-qvnrl","openshift-multus/multus-nhmwq","openshift-network-operator/iptables-alerter-zzlkc","openshift-ovn-kubernetes/ovnkube-node-twszg","kube-system/global-pull-secret-syncer-j9flr"] Jul 02 21:36:03.055080 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.055064 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-75.ec2.internal" Jul 02 21:36:03.057374 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.057358 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bf9dd" Jul 02 21:36:03.059535 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.059364 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cn45l" Jul 02 21:36:03.059535 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:03.059455 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cn45l" podUID="30f00b78-e6a8-48f7-b7b5-801cf436f80b" Jul 02 21:36:03.059737 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.059718 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-vt796\"" Jul 02 21:36:03.059817 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.059722 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 02 21:36:03.059817 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.059728 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 02 21:36:03.061465 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.061448 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jgn9h" Jul 02 21:36:03.061552 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:03.061509 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jgn9h" podUID="42124588-1a15-4e43-a5cb-18c8142bc434" Jul 02 21:36:03.063492 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.063476 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mw4n5" Jul 02 21:36:03.065597 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.065582 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khnpz" Jul 02 21:36:03.065597 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.065589 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 02 21:36:03.065879 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.065864 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-wnsz7\"" Jul 02 21:36:03.066014 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.066000 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 02 21:36:03.067567 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.067548 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zvgzv" Jul 02 21:36:03.067643 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.067624 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 02 21:36:03.067914 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.067899 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 02 21:36:03.068001 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.067983 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 02 21:36:03.068179 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.068167 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-mjpkf\"" Jul 02 21:36:03.069465 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.069452 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-cbgxj\"" Jul 02 21:36:03.069556 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.069543 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4rgsb" Jul 02 21:36:03.069688 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.069674 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 02 21:36:03.069757 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.069717 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 02 21:36:03.071310 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.071296 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 02 21:36:03.071513 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.071501 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 02 21:36:03.071698 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.071666 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 02 21:36:03.071758 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.071722 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-jh89m\"" Jul 02 21:36:03.071870 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.071858 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qvnrl" Jul 02 21:36:03.074028 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.074009 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nhmwq" Jul 02 21:36:03.074189 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.074176 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-lv45z\"" Jul 02 21:36:03.074460 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.074447 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 02 21:36:03.074620 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.074602 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 02 21:36:03.074694 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.074642 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 02 21:36:03.074694 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.074657 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 02 21:36:03.074772 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.074696 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 02 21:36:03.075949 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.075936 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-w5cf9\"" Jul 02 21:36:03.076081 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.076067 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-zzlkc" Jul 02 21:36:03.076308 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.076291 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 02 21:36:03.077869 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.077854 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 02 21:36:03.078077 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.078065 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 02 21:36:03.078261 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.078247 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-h2skp\"" Jul 02 21:36:03.078307 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.078250 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 02 21:36:03.078419 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.078407 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.080551 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.080535 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j9flr" Jul 02 21:36:03.080609 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:03.080585 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j9flr" podUID="d95b88bb-30e2-421a-84a7-96de7cd5d1e1" Jul 02 21:36:03.081330 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.081315 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 02 21:36:03.081407 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.081322 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 02 21:36:03.081509 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.081492 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 02 21:36:03.081546 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.081524 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-wr2tc\"" Jul 02 21:36:03.082381 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.082366 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 02 21:36:03.082633 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.082617 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 02 21:36:03.082760 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.082727 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 02 21:36:03.085963 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.085946 2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-75.ec2.internal" Jul 02 21:36:03.097233 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.097216 2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 02 21:36:03.097312 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.097298 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-75.ec2.internal"] Jul 02 21:36:03.101357 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.101346 2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-75.ec2.internal" Jul 02 21:36:03.109398 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.109381 2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 02 21:36:03.110147 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.110131 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-139-75.ec2.internal"] Jul 02 21:36:03.112482 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.112469 2569 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 02 21:36:03.121054 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.121037 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8m5w6\" (UniqueName: \"kubernetes.io/projected/8571317f-9f7b-4da3-ab3f-03539248e062-kube-api-access-8m5w6\") pod \"tuned-zvgzv\" (UID: \"8571317f-9f7b-4da3-ab3f-03539248e062\") " pod="openshift-cluster-node-tuning-operator/tuned-zvgzv" Jul 02 21:36:03.121132 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.121063 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/283e60d2-fc01-4709-afeb-73699b0d8134-serviceca\") pod \"node-ca-4rgsb\" (UID: \"283e60d2-fc01-4709-afeb-73699b0d8134\") " pod="openshift-image-registry/node-ca-4rgsb" Jul 02 21:36:03.121132 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.121082 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/60b88d05-63e3-4948-96ba-df6c75258a60-host-cni-netd\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.121132 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.121096 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/76102563-0127-409f-9124-0c64395c8b37-cni-binary-copy\") pod \"multus-nhmwq\" (UID: \"76102563-0127-409f-9124-0c64395c8b37\") " pod="openshift-multus/multus-nhmwq" Jul 02 21:36:03.121132 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.121109 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/76102563-0127-409f-9124-0c64395c8b37-hostroot\") pod \"multus-nhmwq\" (UID: \"76102563-0127-409f-9124-0c64395c8b37\") " pod="openshift-multus/multus-nhmwq" Jul 02 21:36:03.121132 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.121132 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/60b88d05-63e3-4948-96ba-df6c75258a60-host-slash\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.121282 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.121164 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/60b88d05-63e3-4948-96ba-df6c75258a60-node-log\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.121282 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.121183 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/60b88d05-63e3-4948-96ba-df6c75258a60-ovnkube-config\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.121282 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.121199 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dc5ec2c9-c5d2-4a0f-94ee-971c99c93d72-konnectivity-ca\") pod \"konnectivity-agent-mw4n5\" (UID: \"dc5ec2c9-c5d2-4a0f-94ee-971c99c93d72\") " pod="kube-system/konnectivity-agent-mw4n5" Jul 02 21:36:03.121282 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.121235 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/04a92c30-9a03-4c2e-b18e-2bad021d2492-socket-dir\") pod \"aws-ebs-csi-driver-node-khnpz\" (UID: \"04a92c30-9a03-4c2e-b18e-2bad021d2492\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khnpz" Jul 02 21:36:03.121282 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.121261 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8571317f-9f7b-4da3-ab3f-03539248e062-etc-sysctl-d\") pod \"tuned-zvgzv\" (UID: \"8571317f-9f7b-4da3-ab3f-03539248e062\") " pod="openshift-cluster-node-tuning-operator/tuned-zvgzv" Jul 02 21:36:03.121407 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.121287 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qvnrl\" (UID: \"06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b\") " pod="openshift-multus/multus-additional-cni-plugins-qvnrl" Jul 02 21:36:03.121407 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.121309 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/76102563-0127-409f-9124-0c64395c8b37-system-cni-dir\") pod \"multus-nhmwq\" (UID: \"76102563-0127-409f-9124-0c64395c8b37\") " pod="openshift-multus/multus-nhmwq" Jul 02 21:36:03.121407 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.121326 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/42c3a1b0818d750cd45841676bb5fa40-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-75.ec2.internal\" (UID: \"42c3a1b0818d750cd45841676bb5fa40\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-75.ec2.internal" Jul 02 21:36:03.121407 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.121360 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/42c3a1b0818d750cd45841676bb5fa40-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-75.ec2.internal\" (UID: \"42c3a1b0818d750cd45841676bb5fa40\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-75.ec2.internal" Jul 02 21:36:03.121407 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.121388 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8571317f-9f7b-4da3-ab3f-03539248e062-etc-tuned\") pod \"tuned-zvgzv\" (UID: \"8571317f-9f7b-4da3-ab3f-03539248e062\") " pod="openshift-cluster-node-tuning-operator/tuned-zvgzv" Jul 02 21:36:03.121541 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.121410 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/04aefd74-fd1d-4021-bc56-e60f8e6ccaf9-iptables-alerter-script\") pod \"iptables-alerter-zzlkc\" (UID: \"04aefd74-fd1d-4021-bc56-e60f8e6ccaf9\") " pod="openshift-network-operator/iptables-alerter-zzlkc" Jul 02 21:36:03.121541 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.121434 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/60b88d05-63e3-4948-96ba-df6c75258a60-host-run-ovn-kubernetes\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.121541 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.121457 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d95b88bb-30e2-421a-84a7-96de7cd5d1e1-kubelet-config\") pod \"global-pull-secret-syncer-j9flr\" (UID: \"d95b88bb-30e2-421a-84a7-96de7cd5d1e1\") " pod="kube-system/global-pull-secret-syncer-j9flr" Jul 02 21:36:03.121541 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.121479 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d95b88bb-30e2-421a-84a7-96de7cd5d1e1-dbus\") pod \"global-pull-secret-syncer-j9flr\" (UID: \"d95b88bb-30e2-421a-84a7-96de7cd5d1e1\") " pod="kube-system/global-pull-secret-syncer-j9flr" Jul 02 21:36:03.121541 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.121495 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/93302d7a5c815ce5fd5abf7c49c032a4-config\") pod \"kube-apiserver-proxy-ip-10-0-139-75.ec2.internal\" (UID: \"93302d7a5c815ce5fd5abf7c49c032a4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-75.ec2.internal" Jul 02 21:36:03.121541 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.121509 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/04a92c30-9a03-4c2e-b18e-2bad021d2492-kubelet-dir\") pod \"aws-ebs-csi-driver-node-khnpz\" (UID: \"04a92c30-9a03-4c2e-b18e-2bad021d2492\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khnpz" Jul 02 21:36:03.121541 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.121529 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8571317f-9f7b-4da3-ab3f-03539248e062-etc-sysctl-conf\") pod \"tuned-zvgzv\" (UID: \"8571317f-9f7b-4da3-ab3f-03539248e062\") " pod="openshift-cluster-node-tuning-operator/tuned-zvgzv" Jul 02 21:36:03.121730 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.121548 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b-cni-binary-copy\") pod \"multus-additional-cni-plugins-qvnrl\" (UID: \"06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b\") " pod="openshift-multus/multus-additional-cni-plugins-qvnrl" Jul 02 21:36:03.121730 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.121564 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qvnrl\" (UID: \"06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b\") " pod="openshift-multus/multus-additional-cni-plugins-qvnrl" Jul 02 21:36:03.121730 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.121581 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgzx9\" (UniqueName: \"kubernetes.io/projected/06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b-kube-api-access-hgzx9\") pod \"multus-additional-cni-plugins-qvnrl\" (UID: \"06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b\") " pod="openshift-multus/multus-additional-cni-plugins-qvnrl" Jul 02 21:36:03.121730 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.121597 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8571317f-9f7b-4da3-ab3f-03539248e062-tmp\") pod \"tuned-zvgzv\" (UID: \"8571317f-9f7b-4da3-ab3f-03539248e062\") " pod="openshift-cluster-node-tuning-operator/tuned-zvgzv" Jul 02 21:36:03.121730 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.121617 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/76102563-0127-409f-9124-0c64395c8b37-multus-conf-dir\") pod \"multus-nhmwq\" (UID: \"76102563-0127-409f-9124-0c64395c8b37\") " pod="openshift-multus/multus-nhmwq" Jul 02 21:36:03.121730 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.121632 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfmtq\" (UniqueName: \"kubernetes.io/projected/76102563-0127-409f-9124-0c64395c8b37-kube-api-access-bfmtq\") pod \"multus-nhmwq\" (UID: \"76102563-0127-409f-9124-0c64395c8b37\") " pod="openshift-multus/multus-nhmwq" Jul 02 21:36:03.121730 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.121646 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/60b88d05-63e3-4948-96ba-df6c75258a60-env-overrides\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.121730 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.121659 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/04a92c30-9a03-4c2e-b18e-2bad021d2492-device-dir\") pod \"aws-ebs-csi-driver-node-khnpz\" (UID: \"04a92c30-9a03-4c2e-b18e-2bad021d2492\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khnpz" Jul 02 21:36:03.121730 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.121684 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8571317f-9f7b-4da3-ab3f-03539248e062-var-lib-kubelet\") pod \"tuned-zvgzv\" (UID: \"8571317f-9f7b-4da3-ab3f-03539248e062\") " pod="openshift-cluster-node-tuning-operator/tuned-zvgzv" Jul 02 21:36:03.121730 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.121703 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/76102563-0127-409f-9124-0c64395c8b37-cnibin\") pod \"multus-nhmwq\" (UID: \"76102563-0127-409f-9124-0c64395c8b37\") " pod="openshift-multus/multus-nhmwq" Jul 02 21:36:03.122058 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.121738 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/76102563-0127-409f-9124-0c64395c8b37-multus-socket-dir-parent\") pod \"multus-nhmwq\" (UID: \"76102563-0127-409f-9124-0c64395c8b37\") " pod="openshift-multus/multus-nhmwq" Jul 02 21:36:03.122058 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.121765 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s22hp\" (UniqueName: \"kubernetes.io/projected/04aefd74-fd1d-4021-bc56-e60f8e6ccaf9-kube-api-access-s22hp\") pod \"iptables-alerter-zzlkc\" (UID: \"04aefd74-fd1d-4021-bc56-e60f8e6ccaf9\") " pod="openshift-network-operator/iptables-alerter-zzlkc" Jul 02 21:36:03.122058 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.121799 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/60b88d05-63e3-4948-96ba-df6c75258a60-systemd-units\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.122058 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.121821 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/60b88d05-63e3-4948-96ba-df6c75258a60-var-lib-openvswitch\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.122058 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.121837 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/60b88d05-63e3-4948-96ba-df6c75258a60-log-socket\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.122058 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.121851 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ed8e72b-e89d-42d5-94f7-3c00d5bb1ca9-tmp-dir\") pod \"node-resolver-bf9dd\" (UID: \"1ed8e72b-e89d-42d5-94f7-3c00d5bb1ca9\") " pod="openshift-dns/node-resolver-bf9dd" Jul 02 21:36:03.122058 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.121867 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30f00b78-e6a8-48f7-b7b5-801cf436f80b-metrics-certs\") pod \"network-metrics-daemon-cn45l\" (UID: \"30f00b78-e6a8-48f7-b7b5-801cf436f80b\") " pod="openshift-multus/network-metrics-daemon-cn45l" Jul 02 21:36:03.122058 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.121890 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bk8rv\" (UniqueName: \"kubernetes.io/projected/42124588-1a15-4e43-a5cb-18c8142bc434-kube-api-access-bk8rv\") pod \"network-check-target-jgn9h\" (UID: \"42124588-1a15-4e43-a5cb-18c8142bc434\") " pod="openshift-network-diagnostics/network-check-target-jgn9h" Jul 02 21:36:03.122058 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.121908 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/76102563-0127-409f-9124-0c64395c8b37-os-release\") pod \"multus-nhmwq\" (UID: \"76102563-0127-409f-9124-0c64395c8b37\") " pod="openshift-multus/multus-nhmwq" Jul 02 21:36:03.122058 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.121922 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/76102563-0127-409f-9124-0c64395c8b37-multus-daemon-config\") pod \"multus-nhmwq\" (UID: \"76102563-0127-409f-9124-0c64395c8b37\") " pod="openshift-multus/multus-nhmwq" Jul 02 21:36:03.122058 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.121944 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8571317f-9f7b-4da3-ab3f-03539248e062-etc-modprobe-d\") pod \"tuned-zvgzv\" (UID: \"8571317f-9f7b-4da3-ab3f-03539248e062\") " pod="openshift-cluster-node-tuning-operator/tuned-zvgzv" Jul 02 21:36:03.122058 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.121969 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8571317f-9f7b-4da3-ab3f-03539248e062-etc-sysconfig\") pod \"tuned-zvgzv\" (UID: \"8571317f-9f7b-4da3-ab3f-03539248e062\") " pod="openshift-cluster-node-tuning-operator/tuned-zvgzv" Jul 02 21:36:03.122058 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.121986 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/76102563-0127-409f-9124-0c64395c8b37-host-var-lib-kubelet\") pod \"multus-nhmwq\" (UID: \"76102563-0127-409f-9124-0c64395c8b37\") " pod="openshift-multus/multus-nhmwq" Jul 02 21:36:03.122058 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.122001 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76102563-0127-409f-9124-0c64395c8b37-etc-kubernetes\") pod \"multus-nhmwq\" (UID: \"76102563-0127-409f-9124-0c64395c8b37\") " pod="openshift-multus/multus-nhmwq" Jul 02 21:36:03.122058 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.122015 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/60b88d05-63e3-4948-96ba-df6c75258a60-ovn-node-metrics-cert\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.122058 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.122039 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/76102563-0127-409f-9124-0c64395c8b37-host-run-k8s-cni-cncf-io\") pod \"multus-nhmwq\" (UID: \"76102563-0127-409f-9124-0c64395c8b37\") " pod="openshift-multus/multus-nhmwq" Jul 02 21:36:03.122639 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.122055 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1ed8e72b-e89d-42d5-94f7-3c00d5bb1ca9-hosts-file\") pod \"node-resolver-bf9dd\" (UID: \"1ed8e72b-e89d-42d5-94f7-3c00d5bb1ca9\") " pod="openshift-dns/node-resolver-bf9dd" Jul 02 21:36:03.122639 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.122069 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8571317f-9f7b-4da3-ab3f-03539248e062-sys\") pod \"tuned-zvgzv\" (UID: \"8571317f-9f7b-4da3-ab3f-03539248e062\") " pod="openshift-cluster-node-tuning-operator/tuned-zvgzv" Jul 02 21:36:03.122639 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.122086 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qvnrl\" (UID: \"06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b\") " pod="openshift-multus/multus-additional-cni-plugins-qvnrl" Jul 02 21:36:03.122639 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.122110 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/60b88d05-63e3-4948-96ba-df6c75258a60-host-run-netns\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.122639 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.122126 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/04a92c30-9a03-4c2e-b18e-2bad021d2492-etc-selinux\") pod \"aws-ebs-csi-driver-node-khnpz\" (UID: \"04a92c30-9a03-4c2e-b18e-2bad021d2492\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khnpz" Jul 02 21:36:03.122639 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.122147 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8571317f-9f7b-4da3-ab3f-03539248e062-etc-kubernetes\") pod \"tuned-zvgzv\" (UID: \"8571317f-9f7b-4da3-ab3f-03539248e062\") " pod="openshift-cluster-node-tuning-operator/tuned-zvgzv" Jul 02 21:36:03.122639 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.122162 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8571317f-9f7b-4da3-ab3f-03539248e062-lib-modules\") pod \"tuned-zvgzv\" (UID: \"8571317f-9f7b-4da3-ab3f-03539248e062\") " pod="openshift-cluster-node-tuning-operator/tuned-zvgzv" Jul 02 21:36:03.122639 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.122201 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/283e60d2-fc01-4709-afeb-73699b0d8134-host\") pod \"node-ca-4rgsb\" (UID: \"283e60d2-fc01-4709-afeb-73699b0d8134\") " pod="openshift-image-registry/node-ca-4rgsb" Jul 02 21:36:03.122639 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.122244 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxhxt\" (UniqueName: \"kubernetes.io/projected/283e60d2-fc01-4709-afeb-73699b0d8134-kube-api-access-gxhxt\") pod \"node-ca-4rgsb\" (UID: \"283e60d2-fc01-4709-afeb-73699b0d8134\") " pod="openshift-image-registry/node-ca-4rgsb" Jul 02 21:36:03.122639 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.122275 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b-os-release\") pod \"multus-additional-cni-plugins-qvnrl\" (UID: \"06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b\") " pod="openshift-multus/multus-additional-cni-plugins-qvnrl" Jul 02 21:36:03.122639 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.122299 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/60b88d05-63e3-4948-96ba-df6c75258a60-host-kubelet\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.122639 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.122324 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/60b88d05-63e3-4948-96ba-df6c75258a60-run-systemd\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.122639 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.122363 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/04a92c30-9a03-4c2e-b18e-2bad021d2492-sys-fs\") pod \"aws-ebs-csi-driver-node-khnpz\" (UID: \"04a92c30-9a03-4c2e-b18e-2bad021d2492\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khnpz" Jul 02 21:36:03.122639 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.122385 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8571317f-9f7b-4da3-ab3f-03539248e062-run\") pod \"tuned-zvgzv\" (UID: \"8571317f-9f7b-4da3-ab3f-03539248e062\") " pod="openshift-cluster-node-tuning-operator/tuned-zvgzv" Jul 02 21:36:03.122639 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.122407 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fksw\" (UniqueName: \"kubernetes.io/projected/30f00b78-e6a8-48f7-b7b5-801cf436f80b-kube-api-access-6fksw\") pod \"network-metrics-daemon-cn45l\" (UID: \"30f00b78-e6a8-48f7-b7b5-801cf436f80b\") " pod="openshift-multus/network-metrics-daemon-cn45l" Jul 02 21:36:03.122639 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.122424 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/76102563-0127-409f-9124-0c64395c8b37-host-run-netns\") pod \"multus-nhmwq\" (UID: \"76102563-0127-409f-9124-0c64395c8b37\") " pod="openshift-multus/multus-nhmwq" Jul 02 21:36:03.123110 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.122439 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/76102563-0127-409f-9124-0c64395c8b37-host-run-multus-certs\") pod \"multus-nhmwq\" (UID: \"76102563-0127-409f-9124-0c64395c8b37\") " pod="openshift-multus/multus-nhmwq" Jul 02 21:36:03.123110 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.122475 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/04aefd74-fd1d-4021-bc56-e60f8e6ccaf9-host-slash\") pod \"iptables-alerter-zzlkc\" (UID: \"04aefd74-fd1d-4021-bc56-e60f8e6ccaf9\") " pod="openshift-network-operator/iptables-alerter-zzlkc" Jul 02 21:36:03.123110 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.122502 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/60b88d05-63e3-4948-96ba-df6c75258a60-etc-openvswitch\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.123110 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.122536 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/60b88d05-63e3-4948-96ba-df6c75258a60-run-openvswitch\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.123110 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.122561 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k27j9\" (UniqueName: \"kubernetes.io/projected/1ed8e72b-e89d-42d5-94f7-3c00d5bb1ca9-kube-api-access-k27j9\") pod \"node-resolver-bf9dd\" (UID: \"1ed8e72b-e89d-42d5-94f7-3c00d5bb1ca9\") " pod="openshift-dns/node-resolver-bf9dd" Jul 02 21:36:03.123110 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.122582 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dc5ec2c9-c5d2-4a0f-94ee-971c99c93d72-agent-certs\") pod \"konnectivity-agent-mw4n5\" (UID: \"dc5ec2c9-c5d2-4a0f-94ee-971c99c93d72\") " pod="kube-system/konnectivity-agent-mw4n5" Jul 02 21:36:03.123110 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.122603 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/76102563-0127-409f-9124-0c64395c8b37-multus-cni-dir\") pod \"multus-nhmwq\" (UID: \"76102563-0127-409f-9124-0c64395c8b37\") " pod="openshift-multus/multus-nhmwq" Jul 02 21:36:03.123110 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.122621 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/76102563-0127-409f-9124-0c64395c8b37-host-var-lib-cni-multus\") pod \"multus-nhmwq\" (UID: \"76102563-0127-409f-9124-0c64395c8b37\") " pod="openshift-multus/multus-nhmwq" Jul 02 21:36:03.123110 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.122637 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/60b88d05-63e3-4948-96ba-df6c75258a60-run-ovn\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.123110 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.122653 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/60b88d05-63e3-4948-96ba-df6c75258a60-host-cni-bin\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.123110 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.122673 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/60b88d05-63e3-4948-96ba-df6c75258a60-ovnkube-script-lib\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.123110 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.122688 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l745t\" (UniqueName: \"kubernetes.io/projected/60b88d05-63e3-4948-96ba-df6c75258a60-kube-api-access-l745t\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.123110 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.122706 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d95b88bb-30e2-421a-84a7-96de7cd5d1e1-original-pull-secret\") pod \"global-pull-secret-syncer-j9flr\" (UID: \"d95b88bb-30e2-421a-84a7-96de7cd5d1e1\") " pod="kube-system/global-pull-secret-syncer-j9flr" Jul 02 21:36:03.123110 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.122730 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b-system-cni-dir\") pod \"multus-additional-cni-plugins-qvnrl\" (UID: \"06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b\") " pod="openshift-multus/multus-additional-cni-plugins-qvnrl" Jul 02 21:36:03.123110 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.122750 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b-cnibin\") pod \"multus-additional-cni-plugins-qvnrl\" (UID: \"06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b\") " pod="openshift-multus/multus-additional-cni-plugins-qvnrl" Jul 02 21:36:03.123110 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.122766 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/76102563-0127-409f-9124-0c64395c8b37-host-var-lib-cni-bin\") pod \"multus-nhmwq\" (UID: \"76102563-0127-409f-9124-0c64395c8b37\") " pod="openshift-multus/multus-nhmwq" Jul 02 21:36:03.123509 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.122798 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8571317f-9f7b-4da3-ab3f-03539248e062-etc-systemd\") pod \"tuned-zvgzv\" (UID: \"8571317f-9f7b-4da3-ab3f-03539248e062\") " pod="openshift-cluster-node-tuning-operator/tuned-zvgzv" Jul 02 21:36:03.123509 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.122819 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/60b88d05-63e3-4948-96ba-df6c75258a60-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.123509 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.122834 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/04a92c30-9a03-4c2e-b18e-2bad021d2492-registration-dir\") pod \"aws-ebs-csi-driver-node-khnpz\" (UID: \"04a92c30-9a03-4c2e-b18e-2bad021d2492\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khnpz" Jul 02 21:36:03.123509 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.122850 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gq92q\" (UniqueName: \"kubernetes.io/projected/04a92c30-9a03-4c2e-b18e-2bad021d2492-kube-api-access-gq92q\") pod \"aws-ebs-csi-driver-node-khnpz\" (UID: \"04a92c30-9a03-4c2e-b18e-2bad021d2492\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khnpz" Jul 02 21:36:03.123509 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.122866 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8571317f-9f7b-4da3-ab3f-03539248e062-host\") pod \"tuned-zvgzv\" (UID: \"8571317f-9f7b-4da3-ab3f-03539248e062\") " pod="openshift-cluster-node-tuning-operator/tuned-zvgzv" Jul 02 21:36:03.223691 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.223666 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/04aefd74-fd1d-4021-bc56-e60f8e6ccaf9-host-slash\") pod \"iptables-alerter-zzlkc\" (UID: \"04aefd74-fd1d-4021-bc56-e60f8e6ccaf9\") " pod="openshift-network-operator/iptables-alerter-zzlkc" Jul 02 21:36:03.223858 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.223694 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/60b88d05-63e3-4948-96ba-df6c75258a60-etc-openvswitch\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.223858 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.223718 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/60b88d05-63e3-4948-96ba-df6c75258a60-run-openvswitch\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.223858 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.223733 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k27j9\" (UniqueName: \"kubernetes.io/projected/1ed8e72b-e89d-42d5-94f7-3c00d5bb1ca9-kube-api-access-k27j9\") pod \"node-resolver-bf9dd\" (UID: \"1ed8e72b-e89d-42d5-94f7-3c00d5bb1ca9\") " pod="openshift-dns/node-resolver-bf9dd" Jul 02 21:36:03.223858 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.223748 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dc5ec2c9-c5d2-4a0f-94ee-971c99c93d72-agent-certs\") pod \"konnectivity-agent-mw4n5\" (UID: \"dc5ec2c9-c5d2-4a0f-94ee-971c99c93d72\") " pod="kube-system/konnectivity-agent-mw4n5" Jul 02 21:36:03.223858 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.223765 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/76102563-0127-409f-9124-0c64395c8b37-multus-cni-dir\") pod \"multus-nhmwq\" (UID: \"76102563-0127-409f-9124-0c64395c8b37\") " pod="openshift-multus/multus-nhmwq" Jul 02 21:36:03.223858 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.223775 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/04aefd74-fd1d-4021-bc56-e60f8e6ccaf9-host-slash\") pod \"iptables-alerter-zzlkc\" (UID: \"04aefd74-fd1d-4021-bc56-e60f8e6ccaf9\") " pod="openshift-network-operator/iptables-alerter-zzlkc" Jul 02 21:36:03.223858 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.223779 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/60b88d05-63e3-4948-96ba-df6c75258a60-etc-openvswitch\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.223858 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.223803 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/76102563-0127-409f-9124-0c64395c8b37-host-var-lib-cni-multus\") pod \"multus-nhmwq\" (UID: \"76102563-0127-409f-9124-0c64395c8b37\") " pod="openshift-multus/multus-nhmwq" Jul 02 21:36:03.223858 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.223819 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/60b88d05-63e3-4948-96ba-df6c75258a60-run-openvswitch\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.223858 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.223839 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/60b88d05-63e3-4948-96ba-df6c75258a60-run-ovn\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.223858 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.223839 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/76102563-0127-409f-9124-0c64395c8b37-host-var-lib-cni-multus\") pod \"multus-nhmwq\" (UID: \"76102563-0127-409f-9124-0c64395c8b37\") " pod="openshift-multus/multus-nhmwq" Jul 02 21:36:03.224414 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.223873 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/60b88d05-63e3-4948-96ba-df6c75258a60-run-ovn\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.224414 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.223903 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/60b88d05-63e3-4948-96ba-df6c75258a60-host-cni-bin\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.224414 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.223935 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/76102563-0127-409f-9124-0c64395c8b37-multus-cni-dir\") pod \"multus-nhmwq\" (UID: \"76102563-0127-409f-9124-0c64395c8b37\") " pod="openshift-multus/multus-nhmwq" Jul 02 21:36:03.224414 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.223941 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/60b88d05-63e3-4948-96ba-df6c75258a60-ovnkube-script-lib\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.224414 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.223944 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/60b88d05-63e3-4948-96ba-df6c75258a60-host-cni-bin\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.224414 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.223979 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l745t\" (UniqueName: \"kubernetes.io/projected/60b88d05-63e3-4948-96ba-df6c75258a60-kube-api-access-l745t\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.224414 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.224032 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d95b88bb-30e2-421a-84a7-96de7cd5d1e1-original-pull-secret\") pod \"global-pull-secret-syncer-j9flr\" (UID: \"d95b88bb-30e2-421a-84a7-96de7cd5d1e1\") " pod="kube-system/global-pull-secret-syncer-j9flr" Jul 02 21:36:03.224414 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.224061 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b-system-cni-dir\") pod \"multus-additional-cni-plugins-qvnrl\" (UID: \"06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b\") " pod="openshift-multus/multus-additional-cni-plugins-qvnrl" Jul 02 21:36:03.224414 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.224083 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b-cnibin\") pod \"multus-additional-cni-plugins-qvnrl\" (UID: \"06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b\") " pod="openshift-multus/multus-additional-cni-plugins-qvnrl" Jul 02 21:36:03.224414 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.224106 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/76102563-0127-409f-9124-0c64395c8b37-host-var-lib-cni-bin\") pod \"multus-nhmwq\" (UID: \"76102563-0127-409f-9124-0c64395c8b37\") " pod="openshift-multus/multus-nhmwq" Jul 02 21:36:03.224414 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.224126 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8571317f-9f7b-4da3-ab3f-03539248e062-etc-systemd\") pod \"tuned-zvgzv\" (UID: \"8571317f-9f7b-4da3-ab3f-03539248e062\") " pod="openshift-cluster-node-tuning-operator/tuned-zvgzv" Jul 02 21:36:03.224414 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.224148 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b-system-cni-dir\") pod \"multus-additional-cni-plugins-qvnrl\" (UID: \"06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b\") " pod="openshift-multus/multus-additional-cni-plugins-qvnrl" Jul 02 21:36:03.224414 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:03.224178 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 21:36:03.224414 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.224162 2569 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 02 21:36:03.224414 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.224201 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8571317f-9f7b-4da3-ab3f-03539248e062-etc-systemd\") pod \"tuned-zvgzv\" (UID: \"8571317f-9f7b-4da3-ab3f-03539248e062\") " pod="openshift-cluster-node-tuning-operator/tuned-zvgzv" Jul 02 21:36:03.224414 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.224150 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/60b88d05-63e3-4948-96ba-df6c75258a60-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.224414 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.224222 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b-cnibin\") pod \"multus-additional-cni-plugins-qvnrl\" (UID: \"06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b\") " pod="openshift-multus/multus-additional-cni-plugins-qvnrl" Jul 02 21:36:03.224414 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.224244 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/60b88d05-63e3-4948-96ba-df6c75258a60-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.225316 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.224189 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/76102563-0127-409f-9124-0c64395c8b37-host-var-lib-cni-bin\") pod \"multus-nhmwq\" (UID: \"76102563-0127-409f-9124-0c64395c8b37\") " pod="openshift-multus/multus-nhmwq" Jul 02 21:36:03.225316 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:03.224269 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d95b88bb-30e2-421a-84a7-96de7cd5d1e1-original-pull-secret podName:d95b88bb-30e2-421a-84a7-96de7cd5d1e1 nodeName:}" failed. No retries permitted until 2026-07-02 21:36:03.724228559 +0000 UTC m=+3.063198988 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d95b88bb-30e2-421a-84a7-96de7cd5d1e1-original-pull-secret") pod "global-pull-secret-syncer-j9flr" (UID: "d95b88bb-30e2-421a-84a7-96de7cd5d1e1") : object "kube-system"/"original-pull-secret" not registered Jul 02 21:36:03.225316 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.224292 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/04a92c30-9a03-4c2e-b18e-2bad021d2492-registration-dir\") pod \"aws-ebs-csi-driver-node-khnpz\" (UID: \"04a92c30-9a03-4c2e-b18e-2bad021d2492\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khnpz" Jul 02 21:36:03.225316 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.224321 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gq92q\" (UniqueName: \"kubernetes.io/projected/04a92c30-9a03-4c2e-b18e-2bad021d2492-kube-api-access-gq92q\") pod \"aws-ebs-csi-driver-node-khnpz\" (UID: \"04a92c30-9a03-4c2e-b18e-2bad021d2492\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khnpz" Jul 02 21:36:03.225316 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.224342 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8571317f-9f7b-4da3-ab3f-03539248e062-host\") pod \"tuned-zvgzv\" (UID: \"8571317f-9f7b-4da3-ab3f-03539248e062\") " pod="openshift-cluster-node-tuning-operator/tuned-zvgzv" Jul 02 21:36:03.225316 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.224347 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/04a92c30-9a03-4c2e-b18e-2bad021d2492-registration-dir\") pod \"aws-ebs-csi-driver-node-khnpz\" (UID: \"04a92c30-9a03-4c2e-b18e-2bad021d2492\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khnpz" Jul 02 21:36:03.225316 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.224369 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8m5w6\" (UniqueName: \"kubernetes.io/projected/8571317f-9f7b-4da3-ab3f-03539248e062-kube-api-access-8m5w6\") pod \"tuned-zvgzv\" (UID: \"8571317f-9f7b-4da3-ab3f-03539248e062\") " pod="openshift-cluster-node-tuning-operator/tuned-zvgzv" Jul 02 21:36:03.225316 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.224417 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/283e60d2-fc01-4709-afeb-73699b0d8134-serviceca\") pod \"node-ca-4rgsb\" (UID: \"283e60d2-fc01-4709-afeb-73699b0d8134\") " pod="openshift-image-registry/node-ca-4rgsb" Jul 02 21:36:03.225316 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.224438 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/60b88d05-63e3-4948-96ba-df6c75258a60-host-cni-netd\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.225316 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.224457 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/76102563-0127-409f-9124-0c64395c8b37-cni-binary-copy\") pod \"multus-nhmwq\" (UID: \"76102563-0127-409f-9124-0c64395c8b37\") " pod="openshift-multus/multus-nhmwq" Jul 02 21:36:03.225316 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.224476 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/76102563-0127-409f-9124-0c64395c8b37-hostroot\") pod \"multus-nhmwq\" (UID: \"76102563-0127-409f-9124-0c64395c8b37\") " pod="openshift-multus/multus-nhmwq" Jul 02 21:36:03.225316 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.224494 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/60b88d05-63e3-4948-96ba-df6c75258a60-host-slash\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.225316 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.224515 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/60b88d05-63e3-4948-96ba-df6c75258a60-node-log\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.225316 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.224534 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/60b88d05-63e3-4948-96ba-df6c75258a60-ovnkube-config\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.225316 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.224553 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dc5ec2c9-c5d2-4a0f-94ee-971c99c93d72-konnectivity-ca\") pod \"konnectivity-agent-mw4n5\" (UID: \"dc5ec2c9-c5d2-4a0f-94ee-971c99c93d72\") " pod="kube-system/konnectivity-agent-mw4n5" Jul 02 21:36:03.225316 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.224571 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/04a92c30-9a03-4c2e-b18e-2bad021d2492-socket-dir\") pod \"aws-ebs-csi-driver-node-khnpz\" (UID: \"04a92c30-9a03-4c2e-b18e-2bad021d2492\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khnpz" Jul 02 21:36:03.226133 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.224592 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8571317f-9f7b-4da3-ab3f-03539248e062-etc-sysctl-d\") pod \"tuned-zvgzv\" (UID: \"8571317f-9f7b-4da3-ab3f-03539248e062\") " pod="openshift-cluster-node-tuning-operator/tuned-zvgzv" Jul 02 21:36:03.226133 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.224610 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qvnrl\" (UID: \"06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b\") " pod="openshift-multus/multus-additional-cni-plugins-qvnrl" Jul 02 21:36:03.226133 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.224616 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/60b88d05-63e3-4948-96ba-df6c75258a60-ovnkube-script-lib\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.226133 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.224630 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/76102563-0127-409f-9124-0c64395c8b37-system-cni-dir\") pod \"multus-nhmwq\" (UID: \"76102563-0127-409f-9124-0c64395c8b37\") " pod="openshift-multus/multus-nhmwq" Jul 02 21:36:03.226133 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.224650 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/42c3a1b0818d750cd45841676bb5fa40-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-75.ec2.internal\" (UID: \"42c3a1b0818d750cd45841676bb5fa40\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-75.ec2.internal" Jul 02 21:36:03.226133 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.224654 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8571317f-9f7b-4da3-ab3f-03539248e062-host\") pod \"tuned-zvgzv\" (UID: \"8571317f-9f7b-4da3-ab3f-03539248e062\") " pod="openshift-cluster-node-tuning-operator/tuned-zvgzv" Jul 02 21:36:03.226133 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.224674 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/42c3a1b0818d750cd45841676bb5fa40-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-75.ec2.internal\" (UID: \"42c3a1b0818d750cd45841676bb5fa40\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-75.ec2.internal" Jul 02 21:36:03.226133 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.224696 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8571317f-9f7b-4da3-ab3f-03539248e062-etc-tuned\") pod \"tuned-zvgzv\" (UID: \"8571317f-9f7b-4da3-ab3f-03539248e062\") " pod="openshift-cluster-node-tuning-operator/tuned-zvgzv" Jul 02 21:36:03.226133 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.224716 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/04aefd74-fd1d-4021-bc56-e60f8e6ccaf9-iptables-alerter-script\") pod \"iptables-alerter-zzlkc\" (UID: \"04aefd74-fd1d-4021-bc56-e60f8e6ccaf9\") " pod="openshift-network-operator/iptables-alerter-zzlkc" Jul 02 21:36:03.226133 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.224717 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/76102563-0127-409f-9124-0c64395c8b37-hostroot\") pod \"multus-nhmwq\" (UID: \"76102563-0127-409f-9124-0c64395c8b37\") " pod="openshift-multus/multus-nhmwq" Jul 02 21:36:03.226133 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.224737 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/60b88d05-63e3-4948-96ba-df6c75258a60-host-run-ovn-kubernetes\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.226133 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.224759 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d95b88bb-30e2-421a-84a7-96de7cd5d1e1-kubelet-config\") pod \"global-pull-secret-syncer-j9flr\" (UID: \"d95b88bb-30e2-421a-84a7-96de7cd5d1e1\") " pod="kube-system/global-pull-secret-syncer-j9flr" Jul 02 21:36:03.226133 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.224779 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d95b88bb-30e2-421a-84a7-96de7cd5d1e1-dbus\") pod \"global-pull-secret-syncer-j9flr\" (UID: \"d95b88bb-30e2-421a-84a7-96de7cd5d1e1\") " pod="kube-system/global-pull-secret-syncer-j9flr" Jul 02 21:36:03.226133 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.224781 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/60b88d05-63e3-4948-96ba-df6c75258a60-host-slash\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.226133 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.224816 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/93302d7a5c815ce5fd5abf7c49c032a4-config\") pod \"kube-apiserver-proxy-ip-10-0-139-75.ec2.internal\" (UID: \"93302d7a5c815ce5fd5abf7c49c032a4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-75.ec2.internal" Jul 02 21:36:03.226133 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.224850 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/60b88d05-63e3-4948-96ba-df6c75258a60-node-log\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.226133 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.224863 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/93302d7a5c815ce5fd5abf7c49c032a4-config\") pod \"kube-apiserver-proxy-ip-10-0-139-75.ec2.internal\" (UID: \"93302d7a5c815ce5fd5abf7c49c032a4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-75.ec2.internal" Jul 02 21:36:03.227021 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.224871 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/283e60d2-fc01-4709-afeb-73699b0d8134-serviceca\") pod \"node-ca-4rgsb\" (UID: \"283e60d2-fc01-4709-afeb-73699b0d8134\") " pod="openshift-image-registry/node-ca-4rgsb" Jul 02 21:36:03.227021 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.224894 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/04a92c30-9a03-4c2e-b18e-2bad021d2492-kubelet-dir\") pod \"aws-ebs-csi-driver-node-khnpz\" (UID: \"04a92c30-9a03-4c2e-b18e-2bad021d2492\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khnpz" Jul 02 21:36:03.227021 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.224921 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8571317f-9f7b-4da3-ab3f-03539248e062-etc-sysctl-conf\") pod \"tuned-zvgzv\" (UID: \"8571317f-9f7b-4da3-ab3f-03539248e062\") " pod="openshift-cluster-node-tuning-operator/tuned-zvgzv" Jul 02 21:36:03.227021 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.224961 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b-cni-binary-copy\") pod \"multus-additional-cni-plugins-qvnrl\" (UID: \"06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b\") " pod="openshift-multus/multus-additional-cni-plugins-qvnrl" Jul 02 21:36:03.227021 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.224973 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/04a92c30-9a03-4c2e-b18e-2bad021d2492-socket-dir\") pod \"aws-ebs-csi-driver-node-khnpz\" (UID: \"04a92c30-9a03-4c2e-b18e-2bad021d2492\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khnpz" Jul 02 21:36:03.227021 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.224991 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qvnrl\" (UID: \"06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b\") " pod="openshift-multus/multus-additional-cni-plugins-qvnrl" Jul 02 21:36:03.227021 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.225019 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qvnrl\" (UID: \"06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b\") " pod="openshift-multus/multus-additional-cni-plugins-qvnrl" Jul 02 21:36:03.227021 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.225018 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgzx9\" (UniqueName: \"kubernetes.io/projected/06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b-kube-api-access-hgzx9\") pod \"multus-additional-cni-plugins-qvnrl\" (UID: \"06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b\") " pod="openshift-multus/multus-additional-cni-plugins-qvnrl" Jul 02 21:36:03.227021 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.225058 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8571317f-9f7b-4da3-ab3f-03539248e062-tmp\") pod \"tuned-zvgzv\" (UID: \"8571317f-9f7b-4da3-ab3f-03539248e062\") " pod="openshift-cluster-node-tuning-operator/tuned-zvgzv" Jul 02 21:36:03.227021 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.225066 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/76102563-0127-409f-9124-0c64395c8b37-system-cni-dir\") pod \"multus-nhmwq\" (UID: \"76102563-0127-409f-9124-0c64395c8b37\") " pod="openshift-multus/multus-nhmwq" Jul 02 21:36:03.227021 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.225080 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/76102563-0127-409f-9124-0c64395c8b37-multus-conf-dir\") pod \"multus-nhmwq\" (UID: \"76102563-0127-409f-9124-0c64395c8b37\") " pod="openshift-multus/multus-nhmwq" Jul 02 21:36:03.227021 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.225090 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8571317f-9f7b-4da3-ab3f-03539248e062-etc-sysctl-d\") pod \"tuned-zvgzv\" (UID: \"8571317f-9f7b-4da3-ab3f-03539248e062\") " pod="openshift-cluster-node-tuning-operator/tuned-zvgzv" Jul 02 21:36:03.227021 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.225101 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/42c3a1b0818d750cd45841676bb5fa40-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-75.ec2.internal\" (UID: \"42c3a1b0818d750cd45841676bb5fa40\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-75.ec2.internal" Jul 02 21:36:03.227021 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.225120 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfmtq\" (UniqueName: \"kubernetes.io/projected/76102563-0127-409f-9124-0c64395c8b37-kube-api-access-bfmtq\") pod \"multus-nhmwq\" (UID: \"76102563-0127-409f-9124-0c64395c8b37\") " pod="openshift-multus/multus-nhmwq" Jul 02 21:36:03.227021 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.225135 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/42c3a1b0818d750cd45841676bb5fa40-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-75.ec2.internal\" (UID: \"42c3a1b0818d750cd45841676bb5fa40\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-75.ec2.internal" Jul 02 21:36:03.227021 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.225145 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/60b88d05-63e3-4948-96ba-df6c75258a60-env-overrides\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.227021 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.225172 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/60b88d05-63e3-4948-96ba-df6c75258a60-ovnkube-config\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.227904 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.225203 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/04a92c30-9a03-4c2e-b18e-2bad021d2492-device-dir\") pod \"aws-ebs-csi-driver-node-khnpz\" (UID: \"04a92c30-9a03-4c2e-b18e-2bad021d2492\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khnpz" Jul 02 21:36:03.227904 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.225218 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/76102563-0127-409f-9124-0c64395c8b37-cni-binary-copy\") pod \"multus-nhmwq\" (UID: \"76102563-0127-409f-9124-0c64395c8b37\") " pod="openshift-multus/multus-nhmwq" Jul 02 21:36:03.227904 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.225228 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8571317f-9f7b-4da3-ab3f-03539248e062-var-lib-kubelet\") pod \"tuned-zvgzv\" (UID: \"8571317f-9f7b-4da3-ab3f-03539248e062\") " pod="openshift-cluster-node-tuning-operator/tuned-zvgzv" Jul 02 21:36:03.227904 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.225273 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/76102563-0127-409f-9124-0c64395c8b37-cnibin\") pod \"multus-nhmwq\" (UID: \"76102563-0127-409f-9124-0c64395c8b37\") " pod="openshift-multus/multus-nhmwq" Jul 02 21:36:03.227904 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.225299 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/76102563-0127-409f-9124-0c64395c8b37-multus-socket-dir-parent\") pod \"multus-nhmwq\" (UID: \"76102563-0127-409f-9124-0c64395c8b37\") " pod="openshift-multus/multus-nhmwq" Jul 02 21:36:03.227904 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.225324 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s22hp\" (UniqueName: \"kubernetes.io/projected/04aefd74-fd1d-4021-bc56-e60f8e6ccaf9-kube-api-access-s22hp\") pod \"iptables-alerter-zzlkc\" (UID: \"04aefd74-fd1d-4021-bc56-e60f8e6ccaf9\") " pod="openshift-network-operator/iptables-alerter-zzlkc" Jul 02 21:36:03.227904 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.225364 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/60b88d05-63e3-4948-96ba-df6c75258a60-systemd-units\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.227904 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.225390 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/60b88d05-63e3-4948-96ba-df6c75258a60-var-lib-openvswitch\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.227904 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.225236 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/76102563-0127-409f-9124-0c64395c8b37-multus-conf-dir\") pod \"multus-nhmwq\" (UID: \"76102563-0127-409f-9124-0c64395c8b37\") " pod="openshift-multus/multus-nhmwq" Jul 02 21:36:03.227904 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.225433 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/60b88d05-63e3-4948-96ba-df6c75258a60-log-socket\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.227904 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.224715 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/60b88d05-63e3-4948-96ba-df6c75258a60-host-cni-netd\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.227904 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.225458 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ed8e72b-e89d-42d5-94f7-3c00d5bb1ca9-tmp-dir\") pod \"node-resolver-bf9dd\" (UID: \"1ed8e72b-e89d-42d5-94f7-3c00d5bb1ca9\") " pod="openshift-dns/node-resolver-bf9dd" Jul 02 21:36:03.227904 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.225483 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30f00b78-e6a8-48f7-b7b5-801cf436f80b-metrics-certs\") pod \"network-metrics-daemon-cn45l\" (UID: \"30f00b78-e6a8-48f7-b7b5-801cf436f80b\") " pod="openshift-multus/network-metrics-daemon-cn45l" Jul 02 21:36:03.227904 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.225499 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/04a92c30-9a03-4c2e-b18e-2bad021d2492-device-dir\") pod \"aws-ebs-csi-driver-node-khnpz\" (UID: \"04a92c30-9a03-4c2e-b18e-2bad021d2492\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khnpz" Jul 02 21:36:03.227904 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.225528 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bk8rv\" (UniqueName: \"kubernetes.io/projected/42124588-1a15-4e43-a5cb-18c8142bc434-kube-api-access-bk8rv\") pod \"network-check-target-jgn9h\" (UID: \"42124588-1a15-4e43-a5cb-18c8142bc434\") " pod="openshift-network-diagnostics/network-check-target-jgn9h" Jul 02 21:36:03.227904 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.225544 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d95b88bb-30e2-421a-84a7-96de7cd5d1e1-kubelet-config\") pod \"global-pull-secret-syncer-j9flr\" (UID: \"d95b88bb-30e2-421a-84a7-96de7cd5d1e1\") " pod="kube-system/global-pull-secret-syncer-j9flr" Jul 02 21:36:03.227904 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.225553 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/76102563-0127-409f-9124-0c64395c8b37-os-release\") pod \"multus-nhmwq\" (UID: \"76102563-0127-409f-9124-0c64395c8b37\") " pod="openshift-multus/multus-nhmwq" Jul 02 21:36:03.228763 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.225593 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/76102563-0127-409f-9124-0c64395c8b37-multus-daemon-config\") pod \"multus-nhmwq\" (UID: \"76102563-0127-409f-9124-0c64395c8b37\") " pod="openshift-multus/multus-nhmwq" Jul 02 21:36:03.228763 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.225631 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dc5ec2c9-c5d2-4a0f-94ee-971c99c93d72-konnectivity-ca\") pod \"konnectivity-agent-mw4n5\" (UID: \"dc5ec2c9-c5d2-4a0f-94ee-971c99c93d72\") " pod="kube-system/konnectivity-agent-mw4n5" Jul 02 21:36:03.228763 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.225631 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qvnrl\" (UID: \"06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b\") " pod="openshift-multus/multus-additional-cni-plugins-qvnrl" Jul 02 21:36:03.228763 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.225669 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/60b88d05-63e3-4948-96ba-df6c75258a60-host-run-ovn-kubernetes\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.228763 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.225724 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d95b88bb-30e2-421a-84a7-96de7cd5d1e1-dbus\") pod \"global-pull-secret-syncer-j9flr\" (UID: \"d95b88bb-30e2-421a-84a7-96de7cd5d1e1\") " pod="kube-system/global-pull-secret-syncer-j9flr" Jul 02 21:36:03.228763 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.225750 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8571317f-9f7b-4da3-ab3f-03539248e062-etc-sysctl-conf\") pod \"tuned-zvgzv\" (UID: \"8571317f-9f7b-4da3-ab3f-03539248e062\") " pod="openshift-cluster-node-tuning-operator/tuned-zvgzv" Jul 02 21:36:03.228763 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.225777 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/60b88d05-63e3-4948-96ba-df6c75258a60-log-socket\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.228763 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.225824 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/04a92c30-9a03-4c2e-b18e-2bad021d2492-kubelet-dir\") pod \"aws-ebs-csi-driver-node-khnpz\" (UID: \"04a92c30-9a03-4c2e-b18e-2bad021d2492\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khnpz" Jul 02 21:36:03.228763 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.225836 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/76102563-0127-409f-9124-0c64395c8b37-cnibin\") pod \"multus-nhmwq\" (UID: \"76102563-0127-409f-9124-0c64395c8b37\") " pod="openshift-multus/multus-nhmwq" Jul 02 21:36:03.228763 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.225852 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/60b88d05-63e3-4948-96ba-df6c75258a60-env-overrides\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.228763 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.225885 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/76102563-0127-409f-9124-0c64395c8b37-multus-socket-dir-parent\") pod \"multus-nhmwq\" (UID: \"76102563-0127-409f-9124-0c64395c8b37\") " pod="openshift-multus/multus-nhmwq" Jul 02 21:36:03.228763 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.225942 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/76102563-0127-409f-9124-0c64395c8b37-os-release\") pod \"multus-nhmwq\" (UID: \"76102563-0127-409f-9124-0c64395c8b37\") " pod="openshift-multus/multus-nhmwq" Jul 02 21:36:03.228763 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.226037 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8571317f-9f7b-4da3-ab3f-03539248e062-etc-modprobe-d\") pod \"tuned-zvgzv\" (UID: \"8571317f-9f7b-4da3-ab3f-03539248e062\") " pod="openshift-cluster-node-tuning-operator/tuned-zvgzv" Jul 02 21:36:03.228763 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.226094 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/76102563-0127-409f-9124-0c64395c8b37-multus-daemon-config\") pod \"multus-nhmwq\" (UID: \"76102563-0127-409f-9124-0c64395c8b37\") " pod="openshift-multus/multus-nhmwq" Jul 02 21:36:03.228763 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:03.226117 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 21:36:03.228763 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.226121 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ed8e72b-e89d-42d5-94f7-3c00d5bb1ca9-tmp-dir\") pod \"node-resolver-bf9dd\" (UID: \"1ed8e72b-e89d-42d5-94f7-3c00d5bb1ca9\") " pod="openshift-dns/node-resolver-bf9dd" Jul 02 21:36:03.228763 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.226127 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8571317f-9f7b-4da3-ab3f-03539248e062-etc-sysconfig\") pod \"tuned-zvgzv\" (UID: \"8571317f-9f7b-4da3-ab3f-03539248e062\") " pod="openshift-cluster-node-tuning-operator/tuned-zvgzv" Jul 02 21:36:03.228763 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.226121 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8571317f-9f7b-4da3-ab3f-03539248e062-var-lib-kubelet\") pod \"tuned-zvgzv\" (UID: \"8571317f-9f7b-4da3-ab3f-03539248e062\") " pod="openshift-cluster-node-tuning-operator/tuned-zvgzv" Jul 02 21:36:03.229300 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.226152 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/60b88d05-63e3-4948-96ba-df6c75258a60-var-lib-openvswitch\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.229300 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.226161 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/76102563-0127-409f-9124-0c64395c8b37-host-var-lib-kubelet\") pod \"multus-nhmwq\" (UID: \"76102563-0127-409f-9124-0c64395c8b37\") " pod="openshift-multus/multus-nhmwq" Jul 02 21:36:03.229300 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.226188 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76102563-0127-409f-9124-0c64395c8b37-etc-kubernetes\") pod \"multus-nhmwq\" (UID: \"76102563-0127-409f-9124-0c64395c8b37\") " pod="openshift-multus/multus-nhmwq" Jul 02 21:36:03.229300 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.226195 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8571317f-9f7b-4da3-ab3f-03539248e062-etc-sysconfig\") pod \"tuned-zvgzv\" (UID: \"8571317f-9f7b-4da3-ab3f-03539248e062\") " pod="openshift-cluster-node-tuning-operator/tuned-zvgzv" Jul 02 21:36:03.229300 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.226198 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/76102563-0127-409f-9124-0c64395c8b37-host-var-lib-kubelet\") pod \"multus-nhmwq\" (UID: \"76102563-0127-409f-9124-0c64395c8b37\") " pod="openshift-multus/multus-nhmwq" Jul 02 21:36:03.229300 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.226212 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/60b88d05-63e3-4948-96ba-df6c75258a60-ovn-node-metrics-cert\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.229300 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.226188 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/60b88d05-63e3-4948-96ba-df6c75258a60-systemd-units\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.229300 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:03.226232 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30f00b78-e6a8-48f7-b7b5-801cf436f80b-metrics-certs podName:30f00b78-e6a8-48f7-b7b5-801cf436f80b nodeName:}" failed. No retries permitted until 2026-07-02 21:36:03.726216319 +0000 UTC m=+3.065186737 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/30f00b78-e6a8-48f7-b7b5-801cf436f80b-metrics-certs") pod "network-metrics-daemon-cn45l" (UID: "30f00b78-e6a8-48f7-b7b5-801cf436f80b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 21:36:03.229300 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.226235 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76102563-0127-409f-9124-0c64395c8b37-etc-kubernetes\") pod \"multus-nhmwq\" (UID: \"76102563-0127-409f-9124-0c64395c8b37\") " pod="openshift-multus/multus-nhmwq" Jul 02 21:36:03.229300 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.226261 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/76102563-0127-409f-9124-0c64395c8b37-host-run-k8s-cni-cncf-io\") pod \"multus-nhmwq\" (UID: \"76102563-0127-409f-9124-0c64395c8b37\") " pod="openshift-multus/multus-nhmwq" Jul 02 21:36:03.229300 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.226279 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8571317f-9f7b-4da3-ab3f-03539248e062-etc-modprobe-d\") pod \"tuned-zvgzv\" (UID: \"8571317f-9f7b-4da3-ab3f-03539248e062\") " pod="openshift-cluster-node-tuning-operator/tuned-zvgzv" Jul 02 21:36:03.229300 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.226285 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1ed8e72b-e89d-42d5-94f7-3c00d5bb1ca9-hosts-file\") pod \"node-resolver-bf9dd\" (UID: \"1ed8e72b-e89d-42d5-94f7-3c00d5bb1ca9\") " pod="openshift-dns/node-resolver-bf9dd" Jul 02 21:36:03.229300 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.226308 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b-cni-binary-copy\") pod \"multus-additional-cni-plugins-qvnrl\" (UID: \"06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b\") " pod="openshift-multus/multus-additional-cni-plugins-qvnrl" Jul 02 21:36:03.229300 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.226332 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1ed8e72b-e89d-42d5-94f7-3c00d5bb1ca9-hosts-file\") pod \"node-resolver-bf9dd\" (UID: \"1ed8e72b-e89d-42d5-94f7-3c00d5bb1ca9\") " pod="openshift-dns/node-resolver-bf9dd" Jul 02 21:36:03.229300 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.226335 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/76102563-0127-409f-9124-0c64395c8b37-host-run-k8s-cni-cncf-io\") pod \"multus-nhmwq\" (UID: \"76102563-0127-409f-9124-0c64395c8b37\") " pod="openshift-multus/multus-nhmwq" Jul 02 21:36:03.229300 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.226338 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8571317f-9f7b-4da3-ab3f-03539248e062-sys\") pod \"tuned-zvgzv\" (UID: \"8571317f-9f7b-4da3-ab3f-03539248e062\") " pod="openshift-cluster-node-tuning-operator/tuned-zvgzv" Jul 02 21:36:03.229300 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.226368 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8571317f-9f7b-4da3-ab3f-03539248e062-sys\") pod \"tuned-zvgzv\" (UID: \"8571317f-9f7b-4da3-ab3f-03539248e062\") " pod="openshift-cluster-node-tuning-operator/tuned-zvgzv" Jul 02 21:36:03.229781 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.226377 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qvnrl\" (UID: \"06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b\") " pod="openshift-multus/multus-additional-cni-plugins-qvnrl" Jul 02 21:36:03.229781 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.226421 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/60b88d05-63e3-4948-96ba-df6c75258a60-host-run-netns\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.229781 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.226451 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/04a92c30-9a03-4c2e-b18e-2bad021d2492-etc-selinux\") pod \"aws-ebs-csi-driver-node-khnpz\" (UID: \"04a92c30-9a03-4c2e-b18e-2bad021d2492\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khnpz" Jul 02 21:36:03.229781 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.226481 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8571317f-9f7b-4da3-ab3f-03539248e062-etc-kubernetes\") pod \"tuned-zvgzv\" (UID: \"8571317f-9f7b-4da3-ab3f-03539248e062\") " pod="openshift-cluster-node-tuning-operator/tuned-zvgzv" Jul 02 21:36:03.229781 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.226505 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8571317f-9f7b-4da3-ab3f-03539248e062-lib-modules\") pod \"tuned-zvgzv\" (UID: \"8571317f-9f7b-4da3-ab3f-03539248e062\") " pod="openshift-cluster-node-tuning-operator/tuned-zvgzv" Jul 02 21:36:03.229781 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.226511 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/60b88d05-63e3-4948-96ba-df6c75258a60-host-run-netns\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.229781 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.226528 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/283e60d2-fc01-4709-afeb-73699b0d8134-host\") pod \"node-ca-4rgsb\" (UID: \"283e60d2-fc01-4709-afeb-73699b0d8134\") " pod="openshift-image-registry/node-ca-4rgsb" Jul 02 21:36:03.229781 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.226542 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/04a92c30-9a03-4c2e-b18e-2bad021d2492-etc-selinux\") pod \"aws-ebs-csi-driver-node-khnpz\" (UID: \"04a92c30-9a03-4c2e-b18e-2bad021d2492\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khnpz" Jul 02 21:36:03.229781 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.226554 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxhxt\" (UniqueName: \"kubernetes.io/projected/283e60d2-fc01-4709-afeb-73699b0d8134-kube-api-access-gxhxt\") pod \"node-ca-4rgsb\" (UID: \"283e60d2-fc01-4709-afeb-73699b0d8134\") " pod="openshift-image-registry/node-ca-4rgsb" Jul 02 21:36:03.229781 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.226579 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b-os-release\") pod \"multus-additional-cni-plugins-qvnrl\" (UID: \"06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b\") " pod="openshift-multus/multus-additional-cni-plugins-qvnrl" Jul 02 21:36:03.229781 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.226601 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8571317f-9f7b-4da3-ab3f-03539248e062-etc-kubernetes\") pod \"tuned-zvgzv\" (UID: \"8571317f-9f7b-4da3-ab3f-03539248e062\") " pod="openshift-cluster-node-tuning-operator/tuned-zvgzv" Jul 02 21:36:03.229781 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.226605 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/60b88d05-63e3-4948-96ba-df6c75258a60-host-kubelet\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.229781 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.226640 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/60b88d05-63e3-4948-96ba-df6c75258a60-run-systemd\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.229781 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.226649 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/60b88d05-63e3-4948-96ba-df6c75258a60-host-kubelet\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.229781 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.226667 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/04a92c30-9a03-4c2e-b18e-2bad021d2492-sys-fs\") pod \"aws-ebs-csi-driver-node-khnpz\" (UID: \"04a92c30-9a03-4c2e-b18e-2bad021d2492\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khnpz" Jul 02 21:36:03.229781 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.226690 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8571317f-9f7b-4da3-ab3f-03539248e062-run\") pod \"tuned-zvgzv\" (UID: \"8571317f-9f7b-4da3-ab3f-03539248e062\") " pod="openshift-cluster-node-tuning-operator/tuned-zvgzv" Jul 02 21:36:03.229781 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.226713 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fksw\" (UniqueName: \"kubernetes.io/projected/30f00b78-e6a8-48f7-b7b5-801cf436f80b-kube-api-access-6fksw\") pod \"network-metrics-daemon-cn45l\" (UID: \"30f00b78-e6a8-48f7-b7b5-801cf436f80b\") " pod="openshift-multus/network-metrics-daemon-cn45l" Jul 02 21:36:03.230385 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.226736 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/76102563-0127-409f-9124-0c64395c8b37-host-run-netns\") pod \"multus-nhmwq\" (UID: \"76102563-0127-409f-9124-0c64395c8b37\") " pod="openshift-multus/multus-nhmwq" Jul 02 21:36:03.230385 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.226750 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/283e60d2-fc01-4709-afeb-73699b0d8134-host\") pod \"node-ca-4rgsb\" (UID: \"283e60d2-fc01-4709-afeb-73699b0d8134\") " pod="openshift-image-registry/node-ca-4rgsb" Jul 02 21:36:03.230385 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.226759 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/76102563-0127-409f-9124-0c64395c8b37-host-run-multus-certs\") pod \"multus-nhmwq\" (UID: \"76102563-0127-409f-9124-0c64395c8b37\") " pod="openshift-multus/multus-nhmwq" Jul 02 21:36:03.230385 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.226812 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8571317f-9f7b-4da3-ab3f-03539248e062-run\") pod \"tuned-zvgzv\" (UID: \"8571317f-9f7b-4da3-ab3f-03539248e062\") " pod="openshift-cluster-node-tuning-operator/tuned-zvgzv" Jul 02 21:36:03.230385 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.226840 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/76102563-0127-409f-9124-0c64395c8b37-host-run-multus-certs\") pod \"multus-nhmwq\" (UID: \"76102563-0127-409f-9124-0c64395c8b37\") " pod="openshift-multus/multus-nhmwq" Jul 02 21:36:03.230385 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.226867 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qvnrl\" (UID: \"06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b\") " pod="openshift-multus/multus-additional-cni-plugins-qvnrl" Jul 02 21:36:03.230385 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.226852 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/60b88d05-63e3-4948-96ba-df6c75258a60-run-systemd\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.230385 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.226916 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/04a92c30-9a03-4c2e-b18e-2bad021d2492-sys-fs\") pod \"aws-ebs-csi-driver-node-khnpz\" (UID: \"04a92c30-9a03-4c2e-b18e-2bad021d2492\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khnpz" Jul 02 21:36:03.230385 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.226714 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b-os-release\") pod \"multus-additional-cni-plugins-qvnrl\" (UID: \"06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b\") " pod="openshift-multus/multus-additional-cni-plugins-qvnrl" Jul 02 21:36:03.230385 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.226977 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/76102563-0127-409f-9124-0c64395c8b37-host-run-netns\") pod \"multus-nhmwq\" (UID: \"76102563-0127-409f-9124-0c64395c8b37\") " pod="openshift-multus/multus-nhmwq" Jul 02 21:36:03.230385 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.226981 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8571317f-9f7b-4da3-ab3f-03539248e062-lib-modules\") pod \"tuned-zvgzv\" (UID: \"8571317f-9f7b-4da3-ab3f-03539248e062\") " pod="openshift-cluster-node-tuning-operator/tuned-zvgzv" Jul 02 21:36:03.230385 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.227121 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/04aefd74-fd1d-4021-bc56-e60f8e6ccaf9-iptables-alerter-script\") pod \"iptables-alerter-zzlkc\" (UID: \"04aefd74-fd1d-4021-bc56-e60f8e6ccaf9\") " pod="openshift-network-operator/iptables-alerter-zzlkc" Jul 02 21:36:03.230385 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.228713 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/60b88d05-63e3-4948-96ba-df6c75258a60-ovn-node-metrics-cert\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.230385 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.228826 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dc5ec2c9-c5d2-4a0f-94ee-971c99c93d72-agent-certs\") pod \"konnectivity-agent-mw4n5\" (UID: \"dc5ec2c9-c5d2-4a0f-94ee-971c99c93d72\") " pod="kube-system/konnectivity-agent-mw4n5" Jul 02 21:36:03.230385 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.228903 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8571317f-9f7b-4da3-ab3f-03539248e062-tmp\") pod \"tuned-zvgzv\" (UID: \"8571317f-9f7b-4da3-ab3f-03539248e062\") " pod="openshift-cluster-node-tuning-operator/tuned-zvgzv" Jul 02 21:36:03.230385 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.228940 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8571317f-9f7b-4da3-ab3f-03539248e062-etc-tuned\") pod \"tuned-zvgzv\" (UID: \"8571317f-9f7b-4da3-ab3f-03539248e062\") " pod="openshift-cluster-node-tuning-operator/tuned-zvgzv" Jul 02 21:36:03.231867 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:03.231845 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 21:36:03.231963 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:03.231872 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 21:36:03.231963 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:03.231886 2569 projected.go:196] Error preparing data for projected volume kube-api-access-bk8rv for pod openshift-network-diagnostics/network-check-target-jgn9h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 21:36:03.231963 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:03.231947 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/42124588-1a15-4e43-a5cb-18c8142bc434-kube-api-access-bk8rv podName:42124588-1a15-4e43-a5cb-18c8142bc434 nodeName:}" failed. No retries permitted until 2026-07-02 21:36:03.731929176 +0000 UTC m=+3.070899603 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-bk8rv" (UniqueName: "kubernetes.io/projected/42124588-1a15-4e43-a5cb-18c8142bc434-kube-api-access-bk8rv") pod "network-check-target-jgn9h" (UID: "42124588-1a15-4e43-a5cb-18c8142bc434") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 21:36:03.232921 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.232479 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k27j9\" (UniqueName: \"kubernetes.io/projected/1ed8e72b-e89d-42d5-94f7-3c00d5bb1ca9-kube-api-access-k27j9\") pod \"node-resolver-bf9dd\" (UID: \"1ed8e72b-e89d-42d5-94f7-3c00d5bb1ca9\") " pod="openshift-dns/node-resolver-bf9dd" Jul 02 21:36:03.232921 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.232769 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gq92q\" (UniqueName: \"kubernetes.io/projected/04a92c30-9a03-4c2e-b18e-2bad021d2492-kube-api-access-gq92q\") pod \"aws-ebs-csi-driver-node-khnpz\" (UID: \"04a92c30-9a03-4c2e-b18e-2bad021d2492\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khnpz" Jul 02 21:36:03.233088 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.233028 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8m5w6\" (UniqueName: \"kubernetes.io/projected/8571317f-9f7b-4da3-ab3f-03539248e062-kube-api-access-8m5w6\") pod \"tuned-zvgzv\" (UID: \"8571317f-9f7b-4da3-ab3f-03539248e062\") " pod="openshift-cluster-node-tuning-operator/tuned-zvgzv" Jul 02 21:36:03.233680 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.233631 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l745t\" (UniqueName: \"kubernetes.io/projected/60b88d05-63e3-4948-96ba-df6c75258a60-kube-api-access-l745t\") pod \"ovnkube-node-twszg\" (UID: \"60b88d05-63e3-4948-96ba-df6c75258a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.234155 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.234132 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s22hp\" (UniqueName: \"kubernetes.io/projected/04aefd74-fd1d-4021-bc56-e60f8e6ccaf9-kube-api-access-s22hp\") pod \"iptables-alerter-zzlkc\" (UID: \"04aefd74-fd1d-4021-bc56-e60f8e6ccaf9\") " pod="openshift-network-operator/iptables-alerter-zzlkc" Jul 02 21:36:03.234290 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.234269 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfmtq\" (UniqueName: \"kubernetes.io/projected/76102563-0127-409f-9124-0c64395c8b37-kube-api-access-bfmtq\") pod \"multus-nhmwq\" (UID: \"76102563-0127-409f-9124-0c64395c8b37\") " pod="openshift-multus/multus-nhmwq" Jul 02 21:36:03.234707 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.234691 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fksw\" (UniqueName: \"kubernetes.io/projected/30f00b78-e6a8-48f7-b7b5-801cf436f80b-kube-api-access-6fksw\") pod \"network-metrics-daemon-cn45l\" (UID: \"30f00b78-e6a8-48f7-b7b5-801cf436f80b\") " pod="openshift-multus/network-metrics-daemon-cn45l" Jul 02 21:36:03.235126 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.235099 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxhxt\" (UniqueName: \"kubernetes.io/projected/283e60d2-fc01-4709-afeb-73699b0d8134-kube-api-access-gxhxt\") pod \"node-ca-4rgsb\" (UID: \"283e60d2-fc01-4709-afeb-73699b0d8134\") " pod="openshift-image-registry/node-ca-4rgsb" Jul 02 21:36:03.235363 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.235345 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgzx9\" (UniqueName: \"kubernetes.io/projected/06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b-kube-api-access-hgzx9\") pod \"multus-additional-cni-plugins-qvnrl\" (UID: \"06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b\") " pod="openshift-multus/multus-additional-cni-plugins-qvnrl" Jul 02 21:36:03.248322 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.248308 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:03.400959 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.400873 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-75.ec2.internal" Jul 02 21:36:03.413495 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.413478 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-75.ec2.internal" Jul 02 21:36:03.418042 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.418021 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bf9dd" Jul 02 21:36:03.434715 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.434697 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mw4n5" Jul 02 21:36:03.452306 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.452288 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khnpz" Jul 02 21:36:03.465796 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.465768 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zvgzv" Jul 02 21:36:03.472638 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.472622 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4rgsb" Jul 02 21:36:03.490254 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.490235 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qvnrl" Jul 02 21:36:03.513702 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.513684 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nhmwq" Jul 02 21:36:03.530331 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.530310 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-zzlkc" Jul 02 21:36:03.731274 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.731188 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30f00b78-e6a8-48f7-b7b5-801cf436f80b-metrics-certs\") pod \"network-metrics-daemon-cn45l\" (UID: \"30f00b78-e6a8-48f7-b7b5-801cf436f80b\") " pod="openshift-multus/network-metrics-daemon-cn45l" Jul 02 21:36:03.731411 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.731277 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d95b88bb-30e2-421a-84a7-96de7cd5d1e1-original-pull-secret\") pod \"global-pull-secret-syncer-j9flr\" (UID: \"d95b88bb-30e2-421a-84a7-96de7cd5d1e1\") " pod="kube-system/global-pull-secret-syncer-j9flr" Jul 02 21:36:03.731411 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:03.731335 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 21:36:03.731411 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:03.731356 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 21:36:03.731411 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:03.731395 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30f00b78-e6a8-48f7-b7b5-801cf436f80b-metrics-certs podName:30f00b78-e6a8-48f7-b7b5-801cf436f80b nodeName:}" failed. No retries permitted until 2026-07-02 21:36:04.731380978 +0000 UTC m=+4.070351390 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/30f00b78-e6a8-48f7-b7b5-801cf436f80b-metrics-certs") pod "network-metrics-daemon-cn45l" (UID: "30f00b78-e6a8-48f7-b7b5-801cf436f80b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 21:36:03.731411 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:03.731411 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d95b88bb-30e2-421a-84a7-96de7cd5d1e1-original-pull-secret podName:d95b88bb-30e2-421a-84a7-96de7cd5d1e1 nodeName:}" failed. No retries permitted until 2026-07-02 21:36:04.731403479 +0000 UTC m=+4.070373892 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d95b88bb-30e2-421a-84a7-96de7cd5d1e1-original-pull-secret") pod "global-pull-secret-syncer-j9flr" (UID: "d95b88bb-30e2-421a-84a7-96de7cd5d1e1") : object "kube-system"/"original-pull-secret" not registered Jul 02 21:36:03.831837 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.831807 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bk8rv\" (UniqueName: \"kubernetes.io/projected/42124588-1a15-4e43-a5cb-18c8142bc434-kube-api-access-bk8rv\") pod \"network-check-target-jgn9h\" (UID: \"42124588-1a15-4e43-a5cb-18c8142bc434\") " pod="openshift-network-diagnostics/network-check-target-jgn9h" Jul 02 21:36:03.832010 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:03.831935 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 21:36:03.832010 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:03.831950 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 21:36:03.832010 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:03.831959 2569 projected.go:196] Error preparing data for projected volume kube-api-access-bk8rv for pod openshift-network-diagnostics/network-check-target-jgn9h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 21:36:03.832010 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:03.832008 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/42124588-1a15-4e43-a5cb-18c8142bc434-kube-api-access-bk8rv podName:42124588-1a15-4e43-a5cb-18c8142bc434 nodeName:}" failed. No retries permitted until 2026-07-02 21:36:04.831994712 +0000 UTC m=+4.170965128 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-bk8rv" (UniqueName: "kubernetes.io/projected/42124588-1a15-4e43-a5cb-18c8142bc434-kube-api-access-bk8rv") pod "network-check-target-jgn9h" (UID: "42124588-1a15-4e43-a5cb-18c8142bc434") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 21:36:03.922914 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:36:03.922869 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc5ec2c9_c5d2_4a0f_94ee_971c99c93d72.slice/crio-76b884aaee8c97b8991e3ef86b43696dce81843ea390a3a9dae3100d883f2cee WatchSource:0}: Error finding container 76b884aaee8c97b8991e3ef86b43696dce81843ea390a3a9dae3100d883f2cee: Status 404 returned error can't find the container with id 76b884aaee8c97b8991e3ef86b43696dce81843ea390a3a9dae3100d883f2cee Jul 02 21:36:03.925272 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.925251 2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 21:36:03.929489 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:36:03.929466 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42c3a1b0818d750cd45841676bb5fa40.slice/crio-b09656de4043545bf91323e3185d6ad1b0cbd9a789af4a1f822c5a9df8a540ec WatchSource:0}: Error finding container b09656de4043545bf91323e3185d6ad1b0cbd9a789af4a1f822c5a9df8a540ec: Status 404 returned error can't find the container with id b09656de4043545bf91323e3185d6ad1b0cbd9a789af4a1f822c5a9df8a540ec Jul 02 21:36:03.930380 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:36:03.930357 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76102563_0127_409f_9124_0c64395c8b37.slice/crio-158f490f49b3018e4867d869d06107cf01cc6a8d3eb113516111135d6ad229fd WatchSource:0}: Error finding container 158f490f49b3018e4867d869d06107cf01cc6a8d3eb113516111135d6ad229fd: Status 404 returned error can't find the container with id 158f490f49b3018e4867d869d06107cf01cc6a8d3eb113516111135d6ad229fd Jul 02 21:36:03.952195 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:36:03.952175 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ed8e72b_e89d_42d5_94f7_3c00d5bb1ca9.slice/crio-2cc3cbfed1c85d7b8d3c7e611246bcab1306e0c0fcd35033d0b2c2f1a167bc7d WatchSource:0}: Error finding container 2cc3cbfed1c85d7b8d3c7e611246bcab1306e0c0fcd35033d0b2c2f1a167bc7d: Status 404 returned error can't find the container with id 2cc3cbfed1c85d7b8d3c7e611246bcab1306e0c0fcd35033d0b2c2f1a167bc7d Jul 02 21:36:03.952666 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.952574 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nhmwq" event={"ID":"76102563-0127-409f-9124-0c64395c8b37","Type":"ContainerStarted","Data":"158f490f49b3018e4867d869d06107cf01cc6a8d3eb113516111135d6ad229fd"} Jul 02 21:36:03.953611 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.953589 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-75.ec2.internal" event={"ID":"42c3a1b0818d750cd45841676bb5fa40","Type":"ContainerStarted","Data":"b09656de4043545bf91323e3185d6ad1b0cbd9a789af4a1f822c5a9df8a540ec"} Jul 02 21:36:03.954661 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.954639 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mw4n5" event={"ID":"dc5ec2c9-c5d2-4a0f-94ee-971c99c93d72","Type":"ContainerStarted","Data":"76b884aaee8c97b8991e3ef86b43696dce81843ea390a3a9dae3100d883f2cee"} Jul 02 21:36:03.955754 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.955734 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-75.ec2.internal" event={"ID":"93302d7a5c815ce5fd5abf7c49c032a4","Type":"ContainerStarted","Data":"79cb1f47515ad096bb1fa6be9993dd6aa5903d2986f767a2609467dbd0ad3256"} Jul 02 21:36:03.956750 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:03.956701 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-twszg" event={"ID":"60b88d05-63e3-4948-96ba-df6c75258a60","Type":"ContainerStarted","Data":"b1474acb33d803046e15e2f6ccf4cfeac3ae0051839eb005fff439002a949d72"} Jul 02 21:36:03.979604 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:36:03.979575 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04a92c30_9a03_4c2e_b18e_2bad021d2492.slice/crio-31d92622a4f120b52266da0b28fab3d0ef7adc132a6405274e2e04517f11d398 WatchSource:0}: Error finding container 31d92622a4f120b52266da0b28fab3d0ef7adc132a6405274e2e04517f11d398: Status 404 returned error can't find the container with id 31d92622a4f120b52266da0b28fab3d0ef7adc132a6405274e2e04517f11d398 Jul 02 21:36:03.995183 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:36:03.995155 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8571317f_9f7b_4da3_ab3f_03539248e062.slice/crio-71dc02dba8297966deadc37c04ab29bf7a0e0313442627d40c4b961cd38da9ee WatchSource:0}: Error finding container 71dc02dba8297966deadc37c04ab29bf7a0e0313442627d40c4b961cd38da9ee: Status 404 returned error can't find the container with id 71dc02dba8297966deadc37c04ab29bf7a0e0313442627d40c4b961cd38da9ee Jul 02 21:36:04.037248 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:36:04.037221 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06f7dfd3_0cdd_4dd0_a5ed_3c4237c4180b.slice/crio-3b9060e227d7fa5d19ed5d3122ba9984911867340a7eb656a79cf5e572ab6869 WatchSource:0}: Error finding container 3b9060e227d7fa5d19ed5d3122ba9984911867340a7eb656a79cf5e572ab6869: Status 404 returned error can't find the container with id 3b9060e227d7fa5d19ed5d3122ba9984911867340a7eb656a79cf5e572ab6869 Jul 02 21:36:04.054928 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:36:04.054901 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod283e60d2_fc01_4709_afeb_73699b0d8134.slice/crio-62bc44ef748ba4fce08ad83042db773ab36ce3993316e5ef47b4819d1521ab33 WatchSource:0}: Error finding container 62bc44ef748ba4fce08ad83042db773ab36ce3993316e5ef47b4819d1521ab33: Status 404 returned error can't find the container with id 62bc44ef748ba4fce08ad83042db773ab36ce3993316e5ef47b4819d1521ab33 Jul 02 21:36:04.057839 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:04.057802 2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-01 21:31:03 +0000 UTC" deadline="2028-01-18 11:36:52.552982097 +0000 UTC" Jul 02 21:36:04.057892 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:04.057840 2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13550h0m48.495144757s" Jul 02 21:36:04.064233 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:36:04.064211 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04aefd74_fd1d_4021_bc56_e60f8e6ccaf9.slice/crio-2027a2ae47e571c59d941906deb55d557149033b7adecba79829abe4bf6a128e WatchSource:0}: Error finding container 2027a2ae47e571c59d941906deb55d557149033b7adecba79829abe4bf6a128e: Status 404 returned error can't find the container with id 2027a2ae47e571c59d941906deb55d557149033b7adecba79829abe4bf6a128e Jul 02 21:36:04.065916 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:04.065885 2569 kuberuntime_manager.go:1459] "Unhandled Error" err="container iptables-alerter start failed in pod iptables-alerter-zzlkc_openshift-network-operator(04aefd74-fd1d-4021-bc56-e60f8e6ccaf9): ErrImagePull: pull QPS exceeded" logger="UnhandledError" Jul 02 21:36:04.065975 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:04.065947 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"iptables-alerter\" with ErrImagePull: \"pull QPS exceeded\"" pod="openshift-network-operator/iptables-alerter-zzlkc" podUID="04aefd74-fd1d-4021-bc56-e60f8e6ccaf9" Jul 02 21:36:04.739453 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:04.739406 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d95b88bb-30e2-421a-84a7-96de7cd5d1e1-original-pull-secret\") pod \"global-pull-secret-syncer-j9flr\" (UID: \"d95b88bb-30e2-421a-84a7-96de7cd5d1e1\") " pod="kube-system/global-pull-secret-syncer-j9flr" Jul 02 21:36:04.739651 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:04.739484 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30f00b78-e6a8-48f7-b7b5-801cf436f80b-metrics-certs\") pod \"network-metrics-daemon-cn45l\" (UID: \"30f00b78-e6a8-48f7-b7b5-801cf436f80b\") " pod="openshift-multus/network-metrics-daemon-cn45l" Jul 02 21:36:04.739651 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:04.739621 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 21:36:04.739773 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:04.739681 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30f00b78-e6a8-48f7-b7b5-801cf436f80b-metrics-certs podName:30f00b78-e6a8-48f7-b7b5-801cf436f80b nodeName:}" failed. No retries permitted until 2026-07-02 21:36:06.739663831 +0000 UTC m=+6.078634251 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/30f00b78-e6a8-48f7-b7b5-801cf436f80b-metrics-certs") pod "network-metrics-daemon-cn45l" (UID: "30f00b78-e6a8-48f7-b7b5-801cf436f80b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 21:36:04.740101 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:04.740081 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 21:36:04.740173 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:04.740139 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d95b88bb-30e2-421a-84a7-96de7cd5d1e1-original-pull-secret podName:d95b88bb-30e2-421a-84a7-96de7cd5d1e1 nodeName:}" failed. No retries permitted until 2026-07-02 21:36:06.7401234 +0000 UTC m=+6.079093818 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d95b88bb-30e2-421a-84a7-96de7cd5d1e1-original-pull-secret") pod "global-pull-secret-syncer-j9flr" (UID: "d95b88bb-30e2-421a-84a7-96de7cd5d1e1") : object "kube-system"/"original-pull-secret" not registered Jul 02 21:36:04.840664 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:04.840623 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bk8rv\" (UniqueName: \"kubernetes.io/projected/42124588-1a15-4e43-a5cb-18c8142bc434-kube-api-access-bk8rv\") pod \"network-check-target-jgn9h\" (UID: \"42124588-1a15-4e43-a5cb-18c8142bc434\") " pod="openshift-network-diagnostics/network-check-target-jgn9h" Jul 02 21:36:04.840874 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:04.840845 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 21:36:04.840874 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:04.840870 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 21:36:04.840999 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:04.840884 2569 projected.go:196] Error preparing data for projected volume kube-api-access-bk8rv for pod openshift-network-diagnostics/network-check-target-jgn9h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 21:36:04.840999 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:04.840945 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/42124588-1a15-4e43-a5cb-18c8142bc434-kube-api-access-bk8rv podName:42124588-1a15-4e43-a5cb-18c8142bc434 nodeName:}" failed. No retries permitted until 2026-07-02 21:36:06.840924914 +0000 UTC m=+6.179895332 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-bk8rv" (UniqueName: "kubernetes.io/projected/42124588-1a15-4e43-a5cb-18c8142bc434-kube-api-access-bk8rv") pod "network-check-target-jgn9h" (UID: "42124588-1a15-4e43-a5cb-18c8142bc434") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 21:36:04.950952 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:04.950918 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jgn9h" Jul 02 21:36:04.951136 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:04.950969 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j9flr" Jul 02 21:36:04.951136 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:04.951057 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jgn9h" podUID="42124588-1a15-4e43-a5cb-18c8142bc434" Jul 02 21:36:04.951136 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:04.950918 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cn45l" Jul 02 21:36:04.951294 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:04.951184 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cn45l" podUID="30f00b78-e6a8-48f7-b7b5-801cf436f80b" Jul 02 21:36:04.951294 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:04.951253 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j9flr" podUID="d95b88bb-30e2-421a-84a7-96de7cd5d1e1" Jul 02 21:36:04.985232 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:04.985111 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zvgzv" event={"ID":"8571317f-9f7b-4da3-ab3f-03539248e062","Type":"ContainerStarted","Data":"71dc02dba8297966deadc37c04ab29bf7a0e0313442627d40c4b961cd38da9ee"} Jul 02 21:36:05.006966 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:05.006849 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khnpz" event={"ID":"04a92c30-9a03-4c2e-b18e-2bad021d2492","Type":"ContainerStarted","Data":"31d92622a4f120b52266da0b28fab3d0ef7adc132a6405274e2e04517f11d398"} Jul 02 21:36:05.030797 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:05.030739 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bf9dd" event={"ID":"1ed8e72b-e89d-42d5-94f7-3c00d5bb1ca9","Type":"ContainerStarted","Data":"2cc3cbfed1c85d7b8d3c7e611246bcab1306e0c0fcd35033d0b2c2f1a167bc7d"} Jul 02 21:36:05.045014 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:05.044981 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-zzlkc" event={"ID":"04aefd74-fd1d-4021-bc56-e60f8e6ccaf9","Type":"ContainerStarted","Data":"2027a2ae47e571c59d941906deb55d557149033b7adecba79829abe4bf6a128e"} Jul 02 21:36:05.055180 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:05.055149 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4rgsb" event={"ID":"283e60d2-fc01-4709-afeb-73699b0d8134","Type":"ContainerStarted","Data":"62bc44ef748ba4fce08ad83042db773ab36ce3993316e5ef47b4819d1521ab33"} Jul 02 21:36:05.058171 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:05.058091 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qvnrl" event={"ID":"06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b","Type":"ContainerStarted","Data":"3b9060e227d7fa5d19ed5d3122ba9984911867340a7eb656a79cf5e572ab6869"} Jul 02 21:36:05.058278 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:05.058167 2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-01 21:31:03 +0000 UTC" deadline="2028-01-03 04:44:08.550366672 +0000 UTC" Jul 02 21:36:05.058278 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:05.058189 2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13183h8m3.492181657s" Jul 02 21:36:05.064175 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:05.064134 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"iptables-alerter\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:28389dd118629aa9d7c2525202f3ad6359afb98b6a3bf7672cb1ecfae2418e23\\\": ErrImagePull: pull QPS exceeded\"" pod="openshift-network-operator/iptables-alerter-zzlkc" podUID="04aefd74-fd1d-4021-bc56-e60f8e6ccaf9" Jul 02 21:36:06.756132 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:06.756097 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30f00b78-e6a8-48f7-b7b5-801cf436f80b-metrics-certs\") pod \"network-metrics-daemon-cn45l\" (UID: \"30f00b78-e6a8-48f7-b7b5-801cf436f80b\") " pod="openshift-multus/network-metrics-daemon-cn45l" Jul 02 21:36:06.756578 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:06.756169 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d95b88bb-30e2-421a-84a7-96de7cd5d1e1-original-pull-secret\") pod \"global-pull-secret-syncer-j9flr\" (UID: \"d95b88bb-30e2-421a-84a7-96de7cd5d1e1\") " pod="kube-system/global-pull-secret-syncer-j9flr" Jul 02 21:36:06.756578 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:06.756288 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 21:36:06.756578 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:06.756345 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d95b88bb-30e2-421a-84a7-96de7cd5d1e1-original-pull-secret podName:d95b88bb-30e2-421a-84a7-96de7cd5d1e1 nodeName:}" failed. No retries permitted until 2026-07-02 21:36:10.75632725 +0000 UTC m=+10.095297662 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d95b88bb-30e2-421a-84a7-96de7cd5d1e1-original-pull-secret") pod "global-pull-secret-syncer-j9flr" (UID: "d95b88bb-30e2-421a-84a7-96de7cd5d1e1") : object "kube-system"/"original-pull-secret" not registered Jul 02 21:36:06.756766 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:06.756748 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 21:36:06.756849 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:06.756821 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30f00b78-e6a8-48f7-b7b5-801cf436f80b-metrics-certs podName:30f00b78-e6a8-48f7-b7b5-801cf436f80b nodeName:}" failed. No retries permitted until 2026-07-02 21:36:10.756802444 +0000 UTC m=+10.095772871 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/30f00b78-e6a8-48f7-b7b5-801cf436f80b-metrics-certs") pod "network-metrics-daemon-cn45l" (UID: "30f00b78-e6a8-48f7-b7b5-801cf436f80b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 21:36:06.857005 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:06.856972 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bk8rv\" (UniqueName: \"kubernetes.io/projected/42124588-1a15-4e43-a5cb-18c8142bc434-kube-api-access-bk8rv\") pod \"network-check-target-jgn9h\" (UID: \"42124588-1a15-4e43-a5cb-18c8142bc434\") " pod="openshift-network-diagnostics/network-check-target-jgn9h" Jul 02 21:36:06.857179 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:06.857166 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 21:36:06.857241 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:06.857188 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 21:36:06.857241 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:06.857200 2569 projected.go:196] Error preparing data for projected volume kube-api-access-bk8rv for pod openshift-network-diagnostics/network-check-target-jgn9h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 21:36:06.857334 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:06.857255 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/42124588-1a15-4e43-a5cb-18c8142bc434-kube-api-access-bk8rv podName:42124588-1a15-4e43-a5cb-18c8142bc434 nodeName:}" failed. No retries permitted until 2026-07-02 21:36:10.857237788 +0000 UTC m=+10.196208216 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-bk8rv" (UniqueName: "kubernetes.io/projected/42124588-1a15-4e43-a5cb-18c8142bc434-kube-api-access-bk8rv") pod "network-check-target-jgn9h" (UID: "42124588-1a15-4e43-a5cb-18c8142bc434") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 21:36:06.950411 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:06.950381 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j9flr" Jul 02 21:36:06.950590 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:06.950498 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j9flr" podUID="d95b88bb-30e2-421a-84a7-96de7cd5d1e1" Jul 02 21:36:06.951294 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:06.950893 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cn45l" Jul 02 21:36:06.951294 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:06.950999 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cn45l" podUID="30f00b78-e6a8-48f7-b7b5-801cf436f80b" Jul 02 21:36:06.951294 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:06.951058 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jgn9h" Jul 02 21:36:06.951294 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:06.951122 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jgn9h" podUID="42124588-1a15-4e43-a5cb-18c8142bc434" Jul 02 21:36:08.954403 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:08.954374 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j9flr" Jul 02 21:36:08.954865 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:08.954507 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j9flr" podUID="d95b88bb-30e2-421a-84a7-96de7cd5d1e1" Jul 02 21:36:08.954865 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:08.954515 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cn45l" Jul 02 21:36:08.954865 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:08.954540 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jgn9h" Jul 02 21:36:08.954865 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:08.954603 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cn45l" podUID="30f00b78-e6a8-48f7-b7b5-801cf436f80b" Jul 02 21:36:08.954865 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:08.954667 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jgn9h" podUID="42124588-1a15-4e43-a5cb-18c8142bc434" Jul 02 21:36:10.790267 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:10.790227 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30f00b78-e6a8-48f7-b7b5-801cf436f80b-metrics-certs\") pod \"network-metrics-daemon-cn45l\" (UID: \"30f00b78-e6a8-48f7-b7b5-801cf436f80b\") " pod="openshift-multus/network-metrics-daemon-cn45l" Jul 02 21:36:10.790713 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:10.790318 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d95b88bb-30e2-421a-84a7-96de7cd5d1e1-original-pull-secret\") pod \"global-pull-secret-syncer-j9flr\" (UID: \"d95b88bb-30e2-421a-84a7-96de7cd5d1e1\") " pod="kube-system/global-pull-secret-syncer-j9flr" Jul 02 21:36:10.790713 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:10.790443 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 21:36:10.790713 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:10.790468 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 21:36:10.790713 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:10.790505 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d95b88bb-30e2-421a-84a7-96de7cd5d1e1-original-pull-secret podName:d95b88bb-30e2-421a-84a7-96de7cd5d1e1 nodeName:}" failed. No retries permitted until 2026-07-02 21:36:18.790487789 +0000 UTC m=+18.129458207 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d95b88bb-30e2-421a-84a7-96de7cd5d1e1-original-pull-secret") pod "global-pull-secret-syncer-j9flr" (UID: "d95b88bb-30e2-421a-84a7-96de7cd5d1e1") : object "kube-system"/"original-pull-secret" not registered Jul 02 21:36:10.790713 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:10.790524 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30f00b78-e6a8-48f7-b7b5-801cf436f80b-metrics-certs podName:30f00b78-e6a8-48f7-b7b5-801cf436f80b nodeName:}" failed. No retries permitted until 2026-07-02 21:36:18.790514162 +0000 UTC m=+18.129484580 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/30f00b78-e6a8-48f7-b7b5-801cf436f80b-metrics-certs") pod "network-metrics-daemon-cn45l" (UID: "30f00b78-e6a8-48f7-b7b5-801cf436f80b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 21:36:10.890898 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:10.890811 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bk8rv\" (UniqueName: \"kubernetes.io/projected/42124588-1a15-4e43-a5cb-18c8142bc434-kube-api-access-bk8rv\") pod \"network-check-target-jgn9h\" (UID: \"42124588-1a15-4e43-a5cb-18c8142bc434\") " pod="openshift-network-diagnostics/network-check-target-jgn9h" Jul 02 21:36:10.891081 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:10.890955 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 21:36:10.891081 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:10.890975 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 21:36:10.891081 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:10.890988 2569 projected.go:196] Error preparing data for projected volume kube-api-access-bk8rv for pod openshift-network-diagnostics/network-check-target-jgn9h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 21:36:10.891081 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:10.891045 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/42124588-1a15-4e43-a5cb-18c8142bc434-kube-api-access-bk8rv podName:42124588-1a15-4e43-a5cb-18c8142bc434 nodeName:}" failed. No retries permitted until 2026-07-02 21:36:18.891029122 +0000 UTC m=+18.229999550 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-bk8rv" (UniqueName: "kubernetes.io/projected/42124588-1a15-4e43-a5cb-18c8142bc434-kube-api-access-bk8rv") pod "network-check-target-jgn9h" (UID: "42124588-1a15-4e43-a5cb-18c8142bc434") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 21:36:10.956410 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:10.956378 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j9flr" Jul 02 21:36:10.956586 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:10.956514 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j9flr" podUID="d95b88bb-30e2-421a-84a7-96de7cd5d1e1" Jul 02 21:36:10.956654 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:10.956602 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cn45l" Jul 02 21:36:10.956884 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:10.956699 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cn45l" podUID="30f00b78-e6a8-48f7-b7b5-801cf436f80b" Jul 02 21:36:10.956884 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:10.956766 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jgn9h" Jul 02 21:36:10.956884 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:10.956853 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jgn9h" podUID="42124588-1a15-4e43-a5cb-18c8142bc434" Jul 02 21:36:12.954194 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:12.953922 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j9flr" Jul 02 21:36:12.954656 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:12.954273 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j9flr" podUID="d95b88bb-30e2-421a-84a7-96de7cd5d1e1" Jul 02 21:36:12.954656 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:12.954006 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jgn9h" Jul 02 21:36:12.954656 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:12.954371 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jgn9h" podUID="42124588-1a15-4e43-a5cb-18c8142bc434" Jul 02 21:36:12.954656 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:12.953975 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cn45l" Jul 02 21:36:12.954656 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:12.954469 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cn45l" podUID="30f00b78-e6a8-48f7-b7b5-801cf436f80b" Jul 02 21:36:14.950412 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:14.950377 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j9flr" Jul 02 21:36:14.950858 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:14.950421 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jgn9h" Jul 02 21:36:14.950858 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:14.950386 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cn45l" Jul 02 21:36:14.950858 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:14.950478 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j9flr" podUID="d95b88bb-30e2-421a-84a7-96de7cd5d1e1" Jul 02 21:36:14.950858 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:14.950593 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cn45l" podUID="30f00b78-e6a8-48f7-b7b5-801cf436f80b" Jul 02 21:36:14.950858 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:14.950679 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jgn9h" podUID="42124588-1a15-4e43-a5cb-18c8142bc434" Jul 02 21:36:16.949861 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:16.949823 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j9flr" Jul 02 21:36:16.950318 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:16.949831 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cn45l" Jul 02 21:36:16.950318 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:16.949834 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jgn9h" Jul 02 21:36:16.950318 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:16.949962 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j9flr" podUID="d95b88bb-30e2-421a-84a7-96de7cd5d1e1" Jul 02 21:36:16.950318 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:16.950046 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cn45l" podUID="30f00b78-e6a8-48f7-b7b5-801cf436f80b" Jul 02 21:36:16.950318 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:16.950137 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jgn9h" podUID="42124588-1a15-4e43-a5cb-18c8142bc434" Jul 02 21:36:18.848703 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:18.848673 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d95b88bb-30e2-421a-84a7-96de7cd5d1e1-original-pull-secret\") pod \"global-pull-secret-syncer-j9flr\" (UID: \"d95b88bb-30e2-421a-84a7-96de7cd5d1e1\") " pod="kube-system/global-pull-secret-syncer-j9flr" Jul 02 21:36:18.849068 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:18.848724 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30f00b78-e6a8-48f7-b7b5-801cf436f80b-metrics-certs\") pod \"network-metrics-daemon-cn45l\" (UID: \"30f00b78-e6a8-48f7-b7b5-801cf436f80b\") " pod="openshift-multus/network-metrics-daemon-cn45l" Jul 02 21:36:18.849068 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:18.848842 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 21:36:18.849068 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:18.848894 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30f00b78-e6a8-48f7-b7b5-801cf436f80b-metrics-certs podName:30f00b78-e6a8-48f7-b7b5-801cf436f80b nodeName:}" failed. No retries permitted until 2026-07-02 21:36:34.848877323 +0000 UTC m=+34.187847756 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/30f00b78-e6a8-48f7-b7b5-801cf436f80b-metrics-certs") pod "network-metrics-daemon-cn45l" (UID: "30f00b78-e6a8-48f7-b7b5-801cf436f80b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 21:36:18.849068 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:18.848909 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 21:36:18.849068 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:18.848959 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d95b88bb-30e2-421a-84a7-96de7cd5d1e1-original-pull-secret podName:d95b88bb-30e2-421a-84a7-96de7cd5d1e1 nodeName:}" failed. No retries permitted until 2026-07-02 21:36:34.848946843 +0000 UTC m=+34.187917255 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d95b88bb-30e2-421a-84a7-96de7cd5d1e1-original-pull-secret") pod "global-pull-secret-syncer-j9flr" (UID: "d95b88bb-30e2-421a-84a7-96de7cd5d1e1") : object "kube-system"/"original-pull-secret" not registered Jul 02 21:36:18.949697 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:18.949665 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bk8rv\" (UniqueName: \"kubernetes.io/projected/42124588-1a15-4e43-a5cb-18c8142bc434-kube-api-access-bk8rv\") pod \"network-check-target-jgn9h\" (UID: \"42124588-1a15-4e43-a5cb-18c8142bc434\") " pod="openshift-network-diagnostics/network-check-target-jgn9h" Jul 02 21:36:18.949850 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:18.949796 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 21:36:18.949850 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:18.949815 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 21:36:18.949850 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:18.949824 2569 projected.go:196] Error preparing data for projected volume kube-api-access-bk8rv for pod openshift-network-diagnostics/network-check-target-jgn9h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 21:36:18.950051 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:18.949861 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/42124588-1a15-4e43-a5cb-18c8142bc434-kube-api-access-bk8rv podName:42124588-1a15-4e43-a5cb-18c8142bc434 nodeName:}" failed. No retries permitted until 2026-07-02 21:36:34.949848314 +0000 UTC m=+34.288818728 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-bk8rv" (UniqueName: "kubernetes.io/projected/42124588-1a15-4e43-a5cb-18c8142bc434-kube-api-access-bk8rv") pod "network-check-target-jgn9h" (UID: "42124588-1a15-4e43-a5cb-18c8142bc434") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 21:36:18.950051 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:18.950026 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cn45l" Jul 02 21:36:18.950172 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:18.950026 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jgn9h" Jul 02 21:36:18.950172 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:18.950154 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cn45l" podUID="30f00b78-e6a8-48f7-b7b5-801cf436f80b" Jul 02 21:36:18.950284 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:18.950209 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jgn9h" podUID="42124588-1a15-4e43-a5cb-18c8142bc434" Jul 02 21:36:18.950284 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:18.950027 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j9flr" Jul 02 21:36:18.950284 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:18.950277 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j9flr" podUID="d95b88bb-30e2-421a-84a7-96de7cd5d1e1" Jul 02 21:36:20.949647 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:20.949614 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jgn9h" Jul 02 21:36:20.950136 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:20.949614 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cn45l" Jul 02 21:36:20.950136 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:20.949724 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jgn9h" podUID="42124588-1a15-4e43-a5cb-18c8142bc434" Jul 02 21:36:20.950136 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:20.949835 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cn45l" podUID="30f00b78-e6a8-48f7-b7b5-801cf436f80b" Jul 02 21:36:20.950136 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:20.949615 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j9flr" Jul 02 21:36:20.950136 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:20.949976 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j9flr" podUID="d95b88bb-30e2-421a-84a7-96de7cd5d1e1" Jul 02 21:36:22.953016 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:22.952987 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jgn9h" Jul 02 21:36:22.953482 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:22.952987 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cn45l" Jul 02 21:36:22.953482 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:22.953108 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jgn9h" podUID="42124588-1a15-4e43-a5cb-18c8142bc434" Jul 02 21:36:22.953482 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:22.952988 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j9flr" Jul 02 21:36:22.953482 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:22.953201 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cn45l" podUID="30f00b78-e6a8-48f7-b7b5-801cf436f80b" Jul 02 21:36:22.953482 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:22.953270 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j9flr" podUID="d95b88bb-30e2-421a-84a7-96de7cd5d1e1" Jul 02 21:36:24.099820 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:24.099602 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nhmwq" event={"ID":"76102563-0127-409f-9124-0c64395c8b37","Type":"ContainerStarted","Data":"4702d23f89e0fd441bb5e5bdc2f6325b5d5a8abe0a92bfc3667b10142a5f0033"} Jul 02 21:36:24.101557 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:24.101532 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zvgzv" event={"ID":"8571317f-9f7b-4da3-ab3f-03539248e062","Type":"ContainerStarted","Data":"e1601fce6567f2ead885042d7e6b8c3e3d0875277c6c6f9874ae12fef37f1278"} Jul 02 21:36:24.103015 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:24.102994 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-75.ec2.internal" event={"ID":"93302d7a5c815ce5fd5abf7c49c032a4","Type":"ContainerStarted","Data":"35b46a07662515f1b0b0fcf9d22288d0b30cef0a1e695f0df9e83ea0a294af61"} Jul 02 21:36:24.105710 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:24.105689 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-twszg" event={"ID":"60b88d05-63e3-4948-96ba-df6c75258a60","Type":"ContainerStarted","Data":"4063c9342eccdd1bd76e24ffbc2f089e667f3cfd2ce3fef10a9d8e5272c29b51"} Jul 02 21:36:24.105822 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:24.105758 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-twszg" event={"ID":"60b88d05-63e3-4948-96ba-df6c75258a60","Type":"ContainerStarted","Data":"f157451b92f277d50bd67117526ae9efdb299fb0c053bcb776f663b5e98dc10f"} Jul 02 21:36:24.105881 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:24.105832 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-twszg" event={"ID":"60b88d05-63e3-4948-96ba-df6c75258a60","Type":"ContainerStarted","Data":"b46836ac491013df0beedb0a36ebaeb72e73f8370c82d22bb4d228ff6b9c35b9"} Jul 02 21:36:24.105940 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:24.105883 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-twszg" event={"ID":"60b88d05-63e3-4948-96ba-df6c75258a60","Type":"ContainerStarted","Data":"e11142d88edb7bb198a9a6553c9af8bd938ceaab9004de066da4f3132bcef511"} Jul 02 21:36:24.105990 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:24.105939 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-twszg" event={"ID":"60b88d05-63e3-4948-96ba-df6c75258a60","Type":"ContainerStarted","Data":"94a134f3efabd8d2e81f631a9b8a2f9bd961241aa91da489d326109f6a359a68"} Jul 02 21:36:24.116274 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:24.116230 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-nhmwq" podStartSLOduration=3.483758229 podStartE2EDuration="23.11620246s" totalImagesPullingTime="19.632444231s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 21:36:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:36:24.115628641 +0000 UTC m=+23.454599079" watchObservedRunningTime="2026-07-02 21:36:24.11620246 +0000 UTC m=+23.455172895" Jul 02 21:36:24.133847 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:24.133798 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-zvgzv" podStartSLOduration=3.6649025760000002 podStartE2EDuration="23.133765527s" totalImagesPullingTime="19.468862951s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 21:36:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:36:24.133290802 +0000 UTC m=+23.472261238" watchObservedRunningTime="2026-07-02 21:36:24.133765527 +0000 UTC m=+23.472735964" Jul 02 21:36:24.950743 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:24.950560 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cn45l" Jul 02 21:36:24.950910 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:24.950560 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j9flr" Jul 02 21:36:24.950910 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:24.950862 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cn45l" podUID="30f00b78-e6a8-48f7-b7b5-801cf436f80b" Jul 02 21:36:24.951019 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:24.950642 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jgn9h" Jul 02 21:36:24.951019 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:24.950937 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j9flr" podUID="d95b88bb-30e2-421a-84a7-96de7cd5d1e1" Jul 02 21:36:24.951111 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:24.951042 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jgn9h" podUID="42124588-1a15-4e43-a5cb-18c8142bc434" Jul 02 21:36:25.108887 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:25.108853 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mw4n5" event={"ID":"dc5ec2c9-c5d2-4a0f-94ee-971c99c93d72","Type":"ContainerStarted","Data":"816a3acd5d7c77971f46332466baf435eb96cedba495d2c8e4338a3363a25d65"} Jul 02 21:36:25.110288 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:25.110266 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khnpz" event={"ID":"04a92c30-9a03-4c2e-b18e-2bad021d2492","Type":"ContainerStarted","Data":"b9663000093dd71877139c03b36788cff17e59fb8eee3ee10b91c2825f65b8e6"} Jul 02 21:36:25.111503 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:25.111483 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bf9dd" event={"ID":"1ed8e72b-e89d-42d5-94f7-3c00d5bb1ca9","Type":"ContainerStarted","Data":"8ba891a3260c600ffc078ec5f36a7d7ee1a6843be3a07a98c6968526893c77d4"} Jul 02 21:36:25.114366 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:25.114348 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-twszg" event={"ID":"60b88d05-63e3-4948-96ba-df6c75258a60","Type":"ContainerStarted","Data":"d245d0eccceac21a9b1e8daccef87e6aac8ff314f043bdfb0190f56c36eaf7f6"} Jul 02 21:36:25.115640 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:25.115623 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-zzlkc" event={"ID":"04aefd74-fd1d-4021-bc56-e60f8e6ccaf9","Type":"ContainerStarted","Data":"c9a16b56cad2c6c28cb3cd8d5e11de380cb30afd93ee5af3cbb3f9519699b39c"} Jul 02 21:36:25.116891 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:25.116871 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4rgsb" event={"ID":"283e60d2-fc01-4709-afeb-73699b0d8134","Type":"ContainerStarted","Data":"200d6a40d94b99f0023a5170708f00dc674b1a9038b30e9fd7d9beaa75f87eab"} Jul 02 21:36:25.118274 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:25.118254 2569 generic.go:356] "Generic (PLEG): container finished" podID="06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b" containerID="9ba9dbdcd20689fa3cd00d361b536fe40f8ad05c4bac438984cb8559b1782d71" exitCode=0 Jul 02 21:36:25.118357 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:25.118332 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qvnrl" event={"ID":"06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b","Type":"ContainerDied","Data":"9ba9dbdcd20689fa3cd00d361b536fe40f8ad05c4bac438984cb8559b1782d71"} Jul 02 21:36:25.119752 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:25.119732 2569 generic.go:356] "Generic (PLEG): container finished" podID="42c3a1b0818d750cd45841676bb5fa40" containerID="11f12beaf4b914cdbace2de17b01521dca74c3f0afafbc5f51174db2e0631a90" exitCode=0 Jul 02 21:36:25.119881 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:25.119857 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-75.ec2.internal" event={"ID":"42c3a1b0818d750cd45841676bb5fa40","Type":"ContainerDied","Data":"11f12beaf4b914cdbace2de17b01521dca74c3f0afafbc5f51174db2e0631a90"} Jul 02 21:36:25.124277 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:25.124243 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-75.ec2.internal" podStartSLOduration=22.124231355 podStartE2EDuration="22.124231355s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 21:36:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:36:24.147756389 +0000 UTC m=+23.486726825" watchObservedRunningTime="2026-07-02 21:36:25.124231355 +0000 UTC m=+24.463201826" Jul 02 21:36:25.141258 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:25.141226 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-bf9dd" podStartSLOduration=4.631111401 podStartE2EDuration="24.141216515s" totalImagesPullingTime="19.510105114s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 21:36:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:36:25.141000473 +0000 UTC m=+24.479970919" watchObservedRunningTime="2026-07-02 21:36:25.141216515 +0000 UTC m=+24.480186950" Jul 02 21:36:25.141429 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:25.141393 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-mw4n5" podStartSLOduration=4.6235861 podStartE2EDuration="24.141387965s" totalImagesPullingTime="19.517801865s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:36:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:36:25.123742444 +0000 UTC m=+24.462712876" watchObservedRunningTime="2026-07-02 21:36:25.141387965 +0000 UTC m=+24.480358399" Jul 02 21:36:25.157372 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:25.157338 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-4rgsb" podStartSLOduration=4.793446821 podStartE2EDuration="24.157332044s" totalImagesPullingTime="19.363885223s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 21:36:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:36:25.156986589 +0000 UTC m=+24.495957025" watchObservedRunningTime="2026-07-02 21:36:25.157332044 +0000 UTC m=+24.496302478" Jul 02 21:36:25.183571 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:25.183549 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-mw4n5" Jul 02 21:36:25.183999 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:25.183984 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-mw4n5" Jul 02 21:36:25.206497 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:25.206451 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-zzlkc" podStartSLOduration=4.732014417 podStartE2EDuration="24.206435648s" totalImagesPullingTime="19.474421231s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 21:36:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=1 observedRunningTime="2026-07-02 21:36:25.20580014 +0000 UTC m=+24.544770570" watchObservedRunningTime="2026-07-02 21:36:25.206435648 +0000 UTC m=+24.545406093" Jul 02 21:36:25.458011 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:25.457977 2569 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 02 21:36:26.104048 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:26.103933 2569 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-02T21:36:25.458008006Z","UUID":"1d73d5c0-6859-4008-9aa9-6c59a6ce52a7","Handler":null,"Name":"","Endpoint":""} Jul 02 21:36:26.105460 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:26.105435 2569 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 02 21:36:26.105601 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:26.105483 2569 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 02 21:36:26.123461 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:26.123429 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-75.ec2.internal" event={"ID":"42c3a1b0818d750cd45841676bb5fa40","Type":"ContainerStarted","Data":"68963f5c200ba81792c0b9ce994ed5ac8c0dd63ef94eb031af5ac9cbb7e3a807"} Jul 02 21:36:26.125533 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:26.125511 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khnpz" event={"ID":"04a92c30-9a03-4c2e-b18e-2bad021d2492","Type":"ContainerStarted","Data":"aa25b4d9e54d2578b09b002d44f02daba332a3b71f41134d44494978b1503c69"} Jul 02 21:36:26.125832 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:26.125810 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-mw4n5" Jul 02 21:36:26.126171 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:26.126155 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-mw4n5" Jul 02 21:36:26.156438 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:26.156344 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-75.ec2.internal" podStartSLOduration=23.156328967 podStartE2EDuration="23.156328967s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 21:36:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:36:26.141370848 +0000 UTC m=+25.480341284" watchObservedRunningTime="2026-07-02 21:36:26.156328967 +0000 UTC m=+25.495299402" Jul 02 21:36:26.950285 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:26.950264 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jgn9h" Jul 02 21:36:26.950391 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:26.950264 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j9flr" Jul 02 21:36:26.950458 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:26.950391 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jgn9h" podUID="42124588-1a15-4e43-a5cb-18c8142bc434" Jul 02 21:36:26.950516 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:26.950482 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j9flr" podUID="d95b88bb-30e2-421a-84a7-96de7cd5d1e1" Jul 02 21:36:26.950516 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:26.950265 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cn45l" Jul 02 21:36:26.950654 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:26.950601 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cn45l" podUID="30f00b78-e6a8-48f7-b7b5-801cf436f80b" Jul 02 21:36:27.130389 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:27.130348 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khnpz" event={"ID":"04a92c30-9a03-4c2e-b18e-2bad021d2492","Type":"ContainerStarted","Data":"86e170e7dd50b0da6ab307016c8fc690dfaadce00176b85273235683280a2b3f"} Jul 02 21:36:27.134159 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:27.134055 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-twszg" event={"ID":"60b88d05-63e3-4948-96ba-df6c75258a60","Type":"ContainerStarted","Data":"30f86b35037e4e8cf91dc40b7ac6f319ce049d76d0ba3c7aca4310fd934e8b28"} Jul 02 21:36:27.147829 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:27.147774 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khnpz" podStartSLOduration=4.195729804 podStartE2EDuration="26.147759585s" totalImagesPullingTime="21.952029781s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 21:36:01 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:36:27.147371481 +0000 UTC m=+26.486341916" watchObservedRunningTime="2026-07-02 21:36:27.147759585 +0000 UTC m=+26.486730019" Jul 02 21:36:28.950379 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:28.950181 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jgn9h" Jul 02 21:36:28.951136 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:28.950181 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cn45l" Jul 02 21:36:28.951136 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:28.950478 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jgn9h" podUID="42124588-1a15-4e43-a5cb-18c8142bc434" Jul 02 21:36:28.951136 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:28.950189 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j9flr" Jul 02 21:36:28.951136 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:28.950536 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cn45l" podUID="30f00b78-e6a8-48f7-b7b5-801cf436f80b" Jul 02 21:36:28.951136 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:28.950600 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j9flr" podUID="d95b88bb-30e2-421a-84a7-96de7cd5d1e1" Jul 02 21:36:29.142764 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:29.142727 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-twszg" event={"ID":"60b88d05-63e3-4948-96ba-df6c75258a60","Type":"ContainerStarted","Data":"e4a07519352829db83cd5f0245a3dbd27a3e6ad32afb3f787f5237be80a4e438"} Jul 02 21:36:29.143032 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:29.143012 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:29.159326 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:29.159302 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:29.169665 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:29.169617 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-twszg" podStartSLOduration=8.589166686 podStartE2EDuration="28.169599927s" totalImagesPullingTime="19.580433241s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 21:36:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:36:29.169102007 +0000 UTC m=+28.508072453" watchObservedRunningTime="2026-07-02 21:36:29.169599927 +0000 UTC m=+28.508570363" Jul 02 21:36:30.145411 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:30.145386 2569 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 02 21:36:30.145913 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:30.145828 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:30.160423 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:30.160399 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:30.515741 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:30.515557 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jgn9h"] Jul 02 21:36:30.515890 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:30.515860 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jgn9h" Jul 02 21:36:30.515959 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:30.515941 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jgn9h" podUID="42124588-1a15-4e43-a5cb-18c8142bc434" Jul 02 21:36:30.524009 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:30.523980 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cn45l"] Jul 02 21:36:30.524108 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:30.524080 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cn45l" Jul 02 21:36:30.524170 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:30.524153 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cn45l" podUID="30f00b78-e6a8-48f7-b7b5-801cf436f80b" Jul 02 21:36:30.528072 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:30.528053 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-j9flr"] Jul 02 21:36:30.528143 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:30.528127 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j9flr" Jul 02 21:36:30.528206 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:30.528191 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j9flr" podUID="d95b88bb-30e2-421a-84a7-96de7cd5d1e1" Jul 02 21:36:31.148402 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:31.148369 2569 generic.go:356] "Generic (PLEG): container finished" podID="06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b" containerID="af821d1377fb543c9072c22d651692f186c0445541764ab533dc41c92cf92d94" exitCode=0 Jul 02 21:36:31.148838 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:31.148455 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qvnrl" event={"ID":"06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b","Type":"ContainerDied","Data":"af821d1377fb543c9072c22d651692f186c0445541764ab533dc41c92cf92d94"} Jul 02 21:36:31.148838 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:31.148510 2569 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 02 21:36:31.950146 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:31.950106 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cn45l" Jul 02 21:36:31.950146 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:31.950143 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j9flr" Jul 02 21:36:31.950397 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:31.950221 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cn45l" podUID="30f00b78-e6a8-48f7-b7b5-801cf436f80b" Jul 02 21:36:31.950397 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:31.950334 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j9flr" podUID="d95b88bb-30e2-421a-84a7-96de7cd5d1e1" Jul 02 21:36:32.150480 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:32.150457 2569 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 02 21:36:32.726606 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:32.726527 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:36:32.951419 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:32.951392 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jgn9h" Jul 02 21:36:32.951579 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:32.951480 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jgn9h" podUID="42124588-1a15-4e43-a5cb-18c8142bc434" Jul 02 21:36:33.155385 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:33.155352 2569 generic.go:356] "Generic (PLEG): container finished" podID="06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b" containerID="65072e5059220f1647425049fe243490071ba1b4ff4337ba43d56474151300a7" exitCode=0 Jul 02 21:36:33.155818 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:33.155433 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qvnrl" event={"ID":"06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b","Type":"ContainerDied","Data":"65072e5059220f1647425049fe243490071ba1b4ff4337ba43d56474151300a7"} Jul 02 21:36:33.165812 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:33.165769 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-twszg" podUID="60b88d05-63e3-4948-96ba-df6c75258a60" containerName="ovnkube-controller" probeResult="failure" output="" Jul 02 21:36:33.949861 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:33.949837 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j9flr" Jul 02 21:36:33.949968 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:33.949840 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cn45l" Jul 02 21:36:33.949968 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:33.949944 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j9flr" podUID="d95b88bb-30e2-421a-84a7-96de7cd5d1e1" Jul 02 21:36:33.950078 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:33.950053 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cn45l" podUID="30f00b78-e6a8-48f7-b7b5-801cf436f80b" Jul 02 21:36:34.160399 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:34.160314 2569 generic.go:356] "Generic (PLEG): container finished" podID="06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b" containerID="49dc813710048b9dd7bdc3156b3c42a68497f80ac512aa6717a6194ebe41394b" exitCode=0 Jul 02 21:36:34.160399 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:34.160350 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qvnrl" event={"ID":"06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b","Type":"ContainerDied","Data":"49dc813710048b9dd7bdc3156b3c42a68497f80ac512aa6717a6194ebe41394b"} Jul 02 21:36:34.880137 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:34.879869 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30f00b78-e6a8-48f7-b7b5-801cf436f80b-metrics-certs\") pod \"network-metrics-daemon-cn45l\" (UID: \"30f00b78-e6a8-48f7-b7b5-801cf436f80b\") " pod="openshift-multus/network-metrics-daemon-cn45l" Jul 02 21:36:34.880287 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:34.880147 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d95b88bb-30e2-421a-84a7-96de7cd5d1e1-original-pull-secret\") pod \"global-pull-secret-syncer-j9flr\" (UID: \"d95b88bb-30e2-421a-84a7-96de7cd5d1e1\") " pod="kube-system/global-pull-secret-syncer-j9flr" Jul 02 21:36:34.880287 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:34.880070 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 21:36:34.880287 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:34.880228 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30f00b78-e6a8-48f7-b7b5-801cf436f80b-metrics-certs podName:30f00b78-e6a8-48f7-b7b5-801cf436f80b nodeName:}" failed. No retries permitted until 2026-07-02 21:37:06.880206874 +0000 UTC m=+66.219177288 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/30f00b78-e6a8-48f7-b7b5-801cf436f80b-metrics-certs") pod "network-metrics-daemon-cn45l" (UID: "30f00b78-e6a8-48f7-b7b5-801cf436f80b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 21:36:34.880287 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:34.880267 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 21:36:34.880444 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:34.880314 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d95b88bb-30e2-421a-84a7-96de7cd5d1e1-original-pull-secret podName:d95b88bb-30e2-421a-84a7-96de7cd5d1e1 nodeName:}" failed. No retries permitted until 2026-07-02 21:37:06.880299957 +0000 UTC m=+66.219270371 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d95b88bb-30e2-421a-84a7-96de7cd5d1e1-original-pull-secret") pod "global-pull-secret-syncer-j9flr" (UID: "d95b88bb-30e2-421a-84a7-96de7cd5d1e1") : object "kube-system"/"original-pull-secret" not registered Jul 02 21:36:34.950249 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:34.950217 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jgn9h" Jul 02 21:36:34.950465 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:34.950441 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jgn9h" podUID="42124588-1a15-4e43-a5cb-18c8142bc434" Jul 02 21:36:34.980774 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:34.980740 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bk8rv\" (UniqueName: \"kubernetes.io/projected/42124588-1a15-4e43-a5cb-18c8142bc434-kube-api-access-bk8rv\") pod \"network-check-target-jgn9h\" (UID: \"42124588-1a15-4e43-a5cb-18c8142bc434\") " pod="openshift-network-diagnostics/network-check-target-jgn9h" Jul 02 21:36:34.980931 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:34.980903 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 21:36:34.980931 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:34.980924 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 21:36:34.980931 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:34.980933 2569 projected.go:196] Error preparing data for projected volume kube-api-access-bk8rv for pod openshift-network-diagnostics/network-check-target-jgn9h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 21:36:34.981053 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:34.980985 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/42124588-1a15-4e43-a5cb-18c8142bc434-kube-api-access-bk8rv podName:42124588-1a15-4e43-a5cb-18c8142bc434 nodeName:}" failed. No retries permitted until 2026-07-02 21:37:06.980967244 +0000 UTC m=+66.319937663 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-bk8rv" (UniqueName: "kubernetes.io/projected/42124588-1a15-4e43-a5cb-18c8142bc434-kube-api-access-bk8rv") pod "network-check-target-jgn9h" (UID: "42124588-1a15-4e43-a5cb-18c8142bc434") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 21:36:35.949956 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:35.949909 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cn45l" Jul 02 21:36:35.949956 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:35.949955 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j9flr" Jul 02 21:36:35.950409 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:35.950069 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cn45l" podUID="30f00b78-e6a8-48f7-b7b5-801cf436f80b" Jul 02 21:36:35.950409 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:35.950205 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j9flr" podUID="d95b88bb-30e2-421a-84a7-96de7cd5d1e1" Jul 02 21:36:36.409589 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.409521 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-75.ec2.internal" event="NodeReady" Jul 02 21:36:36.409706 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.409647 2569 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 02 21:36:36.454686 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.453956 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7c596bcd6b-rzj8c"] Jul 02 21:36:36.458886 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.458863 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-rw2zb"] Jul 02 21:36:36.459019 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.458991 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c596bcd6b-rzj8c" Jul 02 21:36:36.461349 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.461318 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 02 21:36:36.461484 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.461440 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 02 21:36:36.461484 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.461473 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-wzgxn\"" Jul 02 21:36:36.461604 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.461541 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 02 21:36:36.462189 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.462166 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-fqj8h"] Jul 02 21:36:36.462361 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.462339 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rw2zb" Jul 02 21:36:36.464615 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.464597 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-nsr8n\"" Jul 02 21:36:36.464702 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.464646 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 02 21:36:36.464702 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.464661 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 02 21:36:36.465566 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.465547 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fqj8h" Jul 02 21:36:36.465983 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.465962 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 02 21:36:36.466551 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.466530 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 02 21:36:36.467572 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.467540 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 02 21:36:36.467671 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.467615 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 02 21:36:36.467728 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.467702 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bv4sq\"" Jul 02 21:36:36.472837 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.472766 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rw2zb"] Jul 02 21:36:36.473729 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.473712 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fqj8h"] Jul 02 21:36:36.474435 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.474411 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7c596bcd6b-rzj8c"] Jul 02 21:36:36.593354 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.593314 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7337a38e-9618-4a34-b207-44dc712dd12b-metrics-tls\") pod \"dns-default-fqj8h\" (UID: \"7337a38e-9618-4a34-b207-44dc712dd12b\") " pod="openshift-dns/dns-default-fqj8h" Jul 02 21:36:36.593526 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.593364 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2dc606a9-6a0c-49de-be3e-4c090d343d3e-image-registry-private-configuration\") pod \"image-registry-7c596bcd6b-rzj8c\" (UID: \"2dc606a9-6a0c-49de-be3e-4c090d343d3e\") " pod="openshift-image-registry/image-registry-7c596bcd6b-rzj8c" Jul 02 21:36:36.593526 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.593388 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7337a38e-9618-4a34-b207-44dc712dd12b-tmp-dir\") pod \"dns-default-fqj8h\" (UID: \"7337a38e-9618-4a34-b207-44dc712dd12b\") " pod="openshift-dns/dns-default-fqj8h" Jul 02 21:36:36.593526 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.593414 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2dc606a9-6a0c-49de-be3e-4c090d343d3e-ca-trust-extracted\") pod \"image-registry-7c596bcd6b-rzj8c\" (UID: \"2dc606a9-6a0c-49de-be3e-4c090d343d3e\") " pod="openshift-image-registry/image-registry-7c596bcd6b-rzj8c" Jul 02 21:36:36.593526 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.593453 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2dc606a9-6a0c-49de-be3e-4c090d343d3e-registry-certificates\") pod \"image-registry-7c596bcd6b-rzj8c\" (UID: \"2dc606a9-6a0c-49de-be3e-4c090d343d3e\") " pod="openshift-image-registry/image-registry-7c596bcd6b-rzj8c" Jul 02 21:36:36.593718 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.593549 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2dc606a9-6a0c-49de-be3e-4c090d343d3e-trusted-ca\") pod \"image-registry-7c596bcd6b-rzj8c\" (UID: \"2dc606a9-6a0c-49de-be3e-4c090d343d3e\") " pod="openshift-image-registry/image-registry-7c596bcd6b-rzj8c" Jul 02 21:36:36.593718 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.593576 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2dc606a9-6a0c-49de-be3e-4c090d343d3e-bound-sa-token\") pod \"image-registry-7c596bcd6b-rzj8c\" (UID: \"2dc606a9-6a0c-49de-be3e-4c090d343d3e\") " pod="openshift-image-registry/image-registry-7c596bcd6b-rzj8c" Jul 02 21:36:36.593718 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.593636 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9xps\" (UniqueName: \"kubernetes.io/projected/2dc606a9-6a0c-49de-be3e-4c090d343d3e-kube-api-access-x9xps\") pod \"image-registry-7c596bcd6b-rzj8c\" (UID: \"2dc606a9-6a0c-49de-be3e-4c090d343d3e\") " pod="openshift-image-registry/image-registry-7c596bcd6b-rzj8c" Jul 02 21:36:36.593718 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.593673 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2dc606a9-6a0c-49de-be3e-4c090d343d3e-installation-pull-secrets\") pod \"image-registry-7c596bcd6b-rzj8c\" (UID: \"2dc606a9-6a0c-49de-be3e-4c090d343d3e\") " pod="openshift-image-registry/image-registry-7c596bcd6b-rzj8c" Jul 02 21:36:36.593718 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.593712 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3d916985-dd9a-4407-8ed4-e302a9670f50-cert\") pod \"ingress-canary-rw2zb\" (UID: \"3d916985-dd9a-4407-8ed4-e302a9670f50\") " pod="openshift-ingress-canary/ingress-canary-rw2zb" Jul 02 21:36:36.593983 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.593773 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6z5lc\" (UniqueName: \"kubernetes.io/projected/7337a38e-9618-4a34-b207-44dc712dd12b-kube-api-access-6z5lc\") pod \"dns-default-fqj8h\" (UID: \"7337a38e-9618-4a34-b207-44dc712dd12b\") " pod="openshift-dns/dns-default-fqj8h" Jul 02 21:36:36.593983 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.593814 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dq675\" (UniqueName: \"kubernetes.io/projected/3d916985-dd9a-4407-8ed4-e302a9670f50-kube-api-access-dq675\") pod \"ingress-canary-rw2zb\" (UID: \"3d916985-dd9a-4407-8ed4-e302a9670f50\") " pod="openshift-ingress-canary/ingress-canary-rw2zb" Jul 02 21:36:36.593983 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.593839 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7337a38e-9618-4a34-b207-44dc712dd12b-config-volume\") pod \"dns-default-fqj8h\" (UID: \"7337a38e-9618-4a34-b207-44dc712dd12b\") " pod="openshift-dns/dns-default-fqj8h" Jul 02 21:36:36.593983 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.593870 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2dc606a9-6a0c-49de-be3e-4c090d343d3e-registry-tls\") pod \"image-registry-7c596bcd6b-rzj8c\" (UID: \"2dc606a9-6a0c-49de-be3e-4c090d343d3e\") " pod="openshift-image-registry/image-registry-7c596bcd6b-rzj8c" Jul 02 21:36:36.694730 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.694636 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3d916985-dd9a-4407-8ed4-e302a9670f50-cert\") pod \"ingress-canary-rw2zb\" (UID: \"3d916985-dd9a-4407-8ed4-e302a9670f50\") " pod="openshift-ingress-canary/ingress-canary-rw2zb" Jul 02 21:36:36.694730 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.694693 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6z5lc\" (UniqueName: \"kubernetes.io/projected/7337a38e-9618-4a34-b207-44dc712dd12b-kube-api-access-6z5lc\") pod \"dns-default-fqj8h\" (UID: \"7337a38e-9618-4a34-b207-44dc712dd12b\") " pod="openshift-dns/dns-default-fqj8h" Jul 02 21:36:36.694730 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.694722 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dq675\" (UniqueName: \"kubernetes.io/projected/3d916985-dd9a-4407-8ed4-e302a9670f50-kube-api-access-dq675\") pod \"ingress-canary-rw2zb\" (UID: \"3d916985-dd9a-4407-8ed4-e302a9670f50\") " pod="openshift-ingress-canary/ingress-canary-rw2zb" Jul 02 21:36:36.695008 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.694740 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7337a38e-9618-4a34-b207-44dc712dd12b-config-volume\") pod \"dns-default-fqj8h\" (UID: \"7337a38e-9618-4a34-b207-44dc712dd12b\") " pod="openshift-dns/dns-default-fqj8h" Jul 02 21:36:36.695008 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.694765 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2dc606a9-6a0c-49de-be3e-4c090d343d3e-registry-tls\") pod \"image-registry-7c596bcd6b-rzj8c\" (UID: \"2dc606a9-6a0c-49de-be3e-4c090d343d3e\") " pod="openshift-image-registry/image-registry-7c596bcd6b-rzj8c" Jul 02 21:36:36.695008 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.694841 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7337a38e-9618-4a34-b207-44dc712dd12b-metrics-tls\") pod \"dns-default-fqj8h\" (UID: \"7337a38e-9618-4a34-b207-44dc712dd12b\") " pod="openshift-dns/dns-default-fqj8h" Jul 02 21:36:36.695008 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.694872 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2dc606a9-6a0c-49de-be3e-4c090d343d3e-image-registry-private-configuration\") pod \"image-registry-7c596bcd6b-rzj8c\" (UID: \"2dc606a9-6a0c-49de-be3e-4c090d343d3e\") " pod="openshift-image-registry/image-registry-7c596bcd6b-rzj8c" Jul 02 21:36:36.695008 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.694894 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7337a38e-9618-4a34-b207-44dc712dd12b-tmp-dir\") pod \"dns-default-fqj8h\" (UID: \"7337a38e-9618-4a34-b207-44dc712dd12b\") " pod="openshift-dns/dns-default-fqj8h" Jul 02 21:36:36.695008 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.694914 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2dc606a9-6a0c-49de-be3e-4c090d343d3e-ca-trust-extracted\") pod \"image-registry-7c596bcd6b-rzj8c\" (UID: \"2dc606a9-6a0c-49de-be3e-4c090d343d3e\") " pod="openshift-image-registry/image-registry-7c596bcd6b-rzj8c" Jul 02 21:36:36.695008 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.694929 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2dc606a9-6a0c-49de-be3e-4c090d343d3e-registry-certificates\") pod \"image-registry-7c596bcd6b-rzj8c\" (UID: \"2dc606a9-6a0c-49de-be3e-4c090d343d3e\") " pod="openshift-image-registry/image-registry-7c596bcd6b-rzj8c" Jul 02 21:36:36.695008 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.694946 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2dc606a9-6a0c-49de-be3e-4c090d343d3e-trusted-ca\") pod \"image-registry-7c596bcd6b-rzj8c\" (UID: \"2dc606a9-6a0c-49de-be3e-4c090d343d3e\") " pod="openshift-image-registry/image-registry-7c596bcd6b-rzj8c" Jul 02 21:36:36.695008 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.694965 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2dc606a9-6a0c-49de-be3e-4c090d343d3e-bound-sa-token\") pod \"image-registry-7c596bcd6b-rzj8c\" (UID: \"2dc606a9-6a0c-49de-be3e-4c090d343d3e\") " pod="openshift-image-registry/image-registry-7c596bcd6b-rzj8c" Jul 02 21:36:36.695008 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.694987 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9xps\" (UniqueName: \"kubernetes.io/projected/2dc606a9-6a0c-49de-be3e-4c090d343d3e-kube-api-access-x9xps\") pod \"image-registry-7c596bcd6b-rzj8c\" (UID: \"2dc606a9-6a0c-49de-be3e-4c090d343d3e\") " pod="openshift-image-registry/image-registry-7c596bcd6b-rzj8c" Jul 02 21:36:36.695008 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.695005 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2dc606a9-6a0c-49de-be3e-4c090d343d3e-installation-pull-secrets\") pod \"image-registry-7c596bcd6b-rzj8c\" (UID: \"2dc606a9-6a0c-49de-be3e-4c090d343d3e\") " pod="openshift-image-registry/image-registry-7c596bcd6b-rzj8c" Jul 02 21:36:36.695334 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.695215 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7337a38e-9618-4a34-b207-44dc712dd12b-tmp-dir\") pod \"dns-default-fqj8h\" (UID: \"7337a38e-9618-4a34-b207-44dc712dd12b\") " pod="openshift-dns/dns-default-fqj8h" Jul 02 21:36:36.695334 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:36.695310 2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 21:36:36.695334 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:36.695325 2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7c596bcd6b-rzj8c: secret "image-registry-tls" not found Jul 02 21:36:36.695439 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:36.695375 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2dc606a9-6a0c-49de-be3e-4c090d343d3e-registry-tls podName:2dc606a9-6a0c-49de-be3e-4c090d343d3e nodeName:}" failed. No retries permitted until 2026-07-02 21:36:37.195357818 +0000 UTC m=+36.534328240 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2dc606a9-6a0c-49de-be3e-4c090d343d3e-registry-tls") pod "image-registry-7c596bcd6b-rzj8c" (UID: "2dc606a9-6a0c-49de-be3e-4c090d343d3e") : secret "image-registry-tls" not found Jul 02 21:36:36.695439 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.695410 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7337a38e-9618-4a34-b207-44dc712dd12b-config-volume\") pod \"dns-default-fqj8h\" (UID: \"7337a38e-9618-4a34-b207-44dc712dd12b\") " pod="openshift-dns/dns-default-fqj8h" Jul 02 21:36:36.695544 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:36.695515 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 21:36:36.695590 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:36.695567 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7337a38e-9618-4a34-b207-44dc712dd12b-metrics-tls podName:7337a38e-9618-4a34-b207-44dc712dd12b nodeName:}" failed. No retries permitted until 2026-07-02 21:36:37.195551389 +0000 UTC m=+36.534521805 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7337a38e-9618-4a34-b207-44dc712dd12b-metrics-tls") pod "dns-default-fqj8h" (UID: "7337a38e-9618-4a34-b207-44dc712dd12b") : secret "dns-default-metrics-tls" not found Jul 02 21:36:36.695901 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.695867 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2dc606a9-6a0c-49de-be3e-4c090d343d3e-ca-trust-extracted\") pod \"image-registry-7c596bcd6b-rzj8c\" (UID: \"2dc606a9-6a0c-49de-be3e-4c090d343d3e\") " pod="openshift-image-registry/image-registry-7c596bcd6b-rzj8c" Jul 02 21:36:36.696064 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.696043 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2dc606a9-6a0c-49de-be3e-4c090d343d3e-trusted-ca\") pod \"image-registry-7c596bcd6b-rzj8c\" (UID: \"2dc606a9-6a0c-49de-be3e-4c090d343d3e\") " pod="openshift-image-registry/image-registry-7c596bcd6b-rzj8c" Jul 02 21:36:36.696136 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.696079 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2dc606a9-6a0c-49de-be3e-4c090d343d3e-registry-certificates\") pod \"image-registry-7c596bcd6b-rzj8c\" (UID: \"2dc606a9-6a0c-49de-be3e-4c090d343d3e\") " pod="openshift-image-registry/image-registry-7c596bcd6b-rzj8c" Jul 02 21:36:36.699920 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.699895 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2dc606a9-6a0c-49de-be3e-4c090d343d3e-installation-pull-secrets\") pod \"image-registry-7c596bcd6b-rzj8c\" (UID: \"2dc606a9-6a0c-49de-be3e-4c090d343d3e\") " pod="openshift-image-registry/image-registry-7c596bcd6b-rzj8c" Jul 02 21:36:36.699920 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.699908 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3d916985-dd9a-4407-8ed4-e302a9670f50-cert\") pod \"ingress-canary-rw2zb\" (UID: \"3d916985-dd9a-4407-8ed4-e302a9670f50\") " pod="openshift-ingress-canary/ingress-canary-rw2zb" Jul 02 21:36:36.700135 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.699895 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2dc606a9-6a0c-49de-be3e-4c090d343d3e-image-registry-private-configuration\") pod \"image-registry-7c596bcd6b-rzj8c\" (UID: \"2dc606a9-6a0c-49de-be3e-4c090d343d3e\") " pod="openshift-image-registry/image-registry-7c596bcd6b-rzj8c" Jul 02 21:36:36.704505 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.704481 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dq675\" (UniqueName: \"kubernetes.io/projected/3d916985-dd9a-4407-8ed4-e302a9670f50-kube-api-access-dq675\") pod \"ingress-canary-rw2zb\" (UID: \"3d916985-dd9a-4407-8ed4-e302a9670f50\") " pod="openshift-ingress-canary/ingress-canary-rw2zb" Jul 02 21:36:36.704611 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.704583 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9xps\" (UniqueName: \"kubernetes.io/projected/2dc606a9-6a0c-49de-be3e-4c090d343d3e-kube-api-access-x9xps\") pod \"image-registry-7c596bcd6b-rzj8c\" (UID: \"2dc606a9-6a0c-49de-be3e-4c090d343d3e\") " pod="openshift-image-registry/image-registry-7c596bcd6b-rzj8c" Jul 02 21:36:36.704863 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.704841 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2dc606a9-6a0c-49de-be3e-4c090d343d3e-bound-sa-token\") pod \"image-registry-7c596bcd6b-rzj8c\" (UID: \"2dc606a9-6a0c-49de-be3e-4c090d343d3e\") " pod="openshift-image-registry/image-registry-7c596bcd6b-rzj8c" Jul 02 21:36:36.705046 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.705029 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6z5lc\" (UniqueName: \"kubernetes.io/projected/7337a38e-9618-4a34-b207-44dc712dd12b-kube-api-access-6z5lc\") pod \"dns-default-fqj8h\" (UID: \"7337a38e-9618-4a34-b207-44dc712dd12b\") " pod="openshift-dns/dns-default-fqj8h" Jul 02 21:36:36.782057 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.782019 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rw2zb" Jul 02 21:36:36.950141 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.950066 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jgn9h" Jul 02 21:36:36.952469 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.952445 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ckjxv\"" Jul 02 21:36:36.952597 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.952445 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 02 21:36:36.952597 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:36.952503 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 02 21:36:37.199992 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:37.199958 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7337a38e-9618-4a34-b207-44dc712dd12b-metrics-tls\") pod \"dns-default-fqj8h\" (UID: \"7337a38e-9618-4a34-b207-44dc712dd12b\") " pod="openshift-dns/dns-default-fqj8h" Jul 02 21:36:37.200211 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:37.200057 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2dc606a9-6a0c-49de-be3e-4c090d343d3e-registry-tls\") pod \"image-registry-7c596bcd6b-rzj8c\" (UID: \"2dc606a9-6a0c-49de-be3e-4c090d343d3e\") " pod="openshift-image-registry/image-registry-7c596bcd6b-rzj8c" Jul 02 21:36:37.200211 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:37.200138 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 21:36:37.200211 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:37.200163 2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 21:36:37.200211 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:37.200176 2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7c596bcd6b-rzj8c: secret "image-registry-tls" not found Jul 02 21:36:37.200379 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:37.200216 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7337a38e-9618-4a34-b207-44dc712dd12b-metrics-tls podName:7337a38e-9618-4a34-b207-44dc712dd12b nodeName:}" failed. No retries permitted until 2026-07-02 21:36:38.200194981 +0000 UTC m=+37.539165399 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7337a38e-9618-4a34-b207-44dc712dd12b-metrics-tls") pod "dns-default-fqj8h" (UID: "7337a38e-9618-4a34-b207-44dc712dd12b") : secret "dns-default-metrics-tls" not found Jul 02 21:36:37.200379 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:37.200239 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2dc606a9-6a0c-49de-be3e-4c090d343d3e-registry-tls podName:2dc606a9-6a0c-49de-be3e-4c090d343d3e nodeName:}" failed. No retries permitted until 2026-07-02 21:36:38.200228282 +0000 UTC m=+37.539198694 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2dc606a9-6a0c-49de-be3e-4c090d343d3e-registry-tls") pod "image-registry-7c596bcd6b-rzj8c" (UID: "2dc606a9-6a0c-49de-be3e-4c090d343d3e") : secret "image-registry-tls" not found Jul 02 21:36:37.440989 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:37.440953 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-rzzw8"] Jul 02 21:36:37.445208 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:37.445184 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-rzzw8" Jul 02 21:36:37.447925 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:37.447904 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 02 21:36:37.447925 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:37.447917 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 02 21:36:37.447925 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:37.447923 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-gv4ks\"" Jul 02 21:36:37.453987 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:37.453953 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-rzzw8"] Jul 02 21:36:37.603970 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:37.603935 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/af3c0081-d6c4-418c-8cfc-84b0248778f1-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-rzzw8\" (UID: \"af3c0081-d6c4-418c-8cfc-84b0248778f1\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-rzzw8" Jul 02 21:36:37.604139 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:37.604013 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/af3c0081-d6c4-418c-8cfc-84b0248778f1-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-rzzw8\" (UID: \"af3c0081-d6c4-418c-8cfc-84b0248778f1\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-rzzw8" Jul 02 21:36:37.704609 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:37.704530 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/af3c0081-d6c4-418c-8cfc-84b0248778f1-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-rzzw8\" (UID: \"af3c0081-d6c4-418c-8cfc-84b0248778f1\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-rzzw8" Jul 02 21:36:37.704609 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:37.704606 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/af3c0081-d6c4-418c-8cfc-84b0248778f1-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-rzzw8\" (UID: \"af3c0081-d6c4-418c-8cfc-84b0248778f1\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-rzzw8" Jul 02 21:36:37.704837 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:37.704663 2569 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 02 21:36:37.704837 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:37.704737 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af3c0081-d6c4-418c-8cfc-84b0248778f1-networking-console-plugin-cert podName:af3c0081-d6c4-418c-8cfc-84b0248778f1 nodeName:}" failed. No retries permitted until 2026-07-02 21:36:38.204718723 +0000 UTC m=+37.543689156 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/af3c0081-d6c4-418c-8cfc-84b0248778f1-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-rzzw8" (UID: "af3c0081-d6c4-418c-8cfc-84b0248778f1") : secret "networking-console-plugin-cert" not found Jul 02 21:36:37.705368 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:37.705343 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/af3c0081-d6c4-418c-8cfc-84b0248778f1-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-rzzw8\" (UID: \"af3c0081-d6c4-418c-8cfc-84b0248778f1\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-rzzw8" Jul 02 21:36:37.950244 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:37.950214 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j9flr" Jul 02 21:36:37.950595 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:37.950214 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cn45l" Jul 02 21:36:37.952533 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:37.952513 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 02 21:36:37.952665 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:37.952651 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 02 21:36:37.953490 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:37.953476 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-dkhhs\"" Jul 02 21:36:38.120238 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:38.120079 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rw2zb"] Jul 02 21:36:38.166793 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:36:38.166745 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d916985_dd9a_4407_8ed4_e302a9670f50.slice/crio-9ef705f2ba12ee781ff4ef00109d49ace115f0e380ebcd329e61171ce2566f4d WatchSource:0}: Error finding container 9ef705f2ba12ee781ff4ef00109d49ace115f0e380ebcd329e61171ce2566f4d: Status 404 returned error can't find the container with id 9ef705f2ba12ee781ff4ef00109d49ace115f0e380ebcd329e61171ce2566f4d Jul 02 21:36:38.172587 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:38.172562 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rw2zb" event={"ID":"3d916985-dd9a-4407-8ed4-e302a9670f50","Type":"ContainerStarted","Data":"9ef705f2ba12ee781ff4ef00109d49ace115f0e380ebcd329e61171ce2566f4d"} Jul 02 21:36:38.210050 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:38.209395 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7337a38e-9618-4a34-b207-44dc712dd12b-metrics-tls\") pod \"dns-default-fqj8h\" (UID: \"7337a38e-9618-4a34-b207-44dc712dd12b\") " pod="openshift-dns/dns-default-fqj8h" Jul 02 21:36:38.210050 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:38.209437 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/af3c0081-d6c4-418c-8cfc-84b0248778f1-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-rzzw8\" (UID: \"af3c0081-d6c4-418c-8cfc-84b0248778f1\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-rzzw8" Jul 02 21:36:38.210050 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:38.209534 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2dc606a9-6a0c-49de-be3e-4c090d343d3e-registry-tls\") pod \"image-registry-7c596bcd6b-rzj8c\" (UID: \"2dc606a9-6a0c-49de-be3e-4c090d343d3e\") " pod="openshift-image-registry/image-registry-7c596bcd6b-rzj8c" Jul 02 21:36:38.210050 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:38.209685 2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 21:36:38.210050 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:38.209700 2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7c596bcd6b-rzj8c: secret "image-registry-tls" not found Jul 02 21:36:38.210050 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:38.209749 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2dc606a9-6a0c-49de-be3e-4c090d343d3e-registry-tls podName:2dc606a9-6a0c-49de-be3e-4c090d343d3e nodeName:}" failed. No retries permitted until 2026-07-02 21:36:40.209731458 +0000 UTC m=+39.548701893 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2dc606a9-6a0c-49de-be3e-4c090d343d3e-registry-tls") pod "image-registry-7c596bcd6b-rzj8c" (UID: "2dc606a9-6a0c-49de-be3e-4c090d343d3e") : secret "image-registry-tls" not found Jul 02 21:36:38.210393 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:38.210298 2569 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 02 21:36:38.210393 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:38.210362 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af3c0081-d6c4-418c-8cfc-84b0248778f1-networking-console-plugin-cert podName:af3c0081-d6c4-418c-8cfc-84b0248778f1 nodeName:}" failed. No retries permitted until 2026-07-02 21:36:39.210344445 +0000 UTC m=+38.549314863 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/af3c0081-d6c4-418c-8cfc-84b0248778f1-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-rzzw8" (UID: "af3c0081-d6c4-418c-8cfc-84b0248778f1") : secret "networking-console-plugin-cert" not found Jul 02 21:36:38.210393 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:38.210366 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 21:36:38.210502 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:38.210416 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7337a38e-9618-4a34-b207-44dc712dd12b-metrics-tls podName:7337a38e-9618-4a34-b207-44dc712dd12b nodeName:}" failed. No retries permitted until 2026-07-02 21:36:40.210400436 +0000 UTC m=+39.549370868 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7337a38e-9618-4a34-b207-44dc712dd12b-metrics-tls") pod "dns-default-fqj8h" (UID: "7337a38e-9618-4a34-b207-44dc712dd12b") : secret "dns-default-metrics-tls" not found Jul 02 21:36:39.177835 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:39.177800 2569 generic.go:356] "Generic (PLEG): container finished" podID="06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b" containerID="250071c8c497a5daf2e4b56473fb6954e9415bb4c91bb5deb703c6ac0dbb57a0" exitCode=0 Jul 02 21:36:39.178485 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:39.177856 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qvnrl" event={"ID":"06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b","Type":"ContainerDied","Data":"250071c8c497a5daf2e4b56473fb6954e9415bb4c91bb5deb703c6ac0dbb57a0"} Jul 02 21:36:39.217941 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:39.217904 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/af3c0081-d6c4-418c-8cfc-84b0248778f1-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-rzzw8\" (UID: \"af3c0081-d6c4-418c-8cfc-84b0248778f1\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-rzzw8" Jul 02 21:36:39.218075 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:39.218064 2569 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 02 21:36:39.218138 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:39.218125 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af3c0081-d6c4-418c-8cfc-84b0248778f1-networking-console-plugin-cert podName:af3c0081-d6c4-418c-8cfc-84b0248778f1 nodeName:}" failed. No retries permitted until 2026-07-02 21:36:41.218105576 +0000 UTC m=+40.557075992 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/af3c0081-d6c4-418c-8cfc-84b0248778f1-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-rzzw8" (UID: "af3c0081-d6c4-418c-8cfc-84b0248778f1") : secret "networking-console-plugin-cert" not found Jul 02 21:36:40.183418 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:40.183383 2569 generic.go:356] "Generic (PLEG): container finished" podID="06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b" containerID="ff16cb528638eac27243e4a8c36e93206d85b0607ebf5c6575b87d3f1cc5b44f" exitCode=0 Jul 02 21:36:40.184085 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:40.183443 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qvnrl" event={"ID":"06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b","Type":"ContainerDied","Data":"ff16cb528638eac27243e4a8c36e93206d85b0607ebf5c6575b87d3f1cc5b44f"} Jul 02 21:36:40.224438 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:40.224409 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2dc606a9-6a0c-49de-be3e-4c090d343d3e-registry-tls\") pod \"image-registry-7c596bcd6b-rzj8c\" (UID: \"2dc606a9-6a0c-49de-be3e-4c090d343d3e\") " pod="openshift-image-registry/image-registry-7c596bcd6b-rzj8c" Jul 02 21:36:40.224551 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:40.224482 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7337a38e-9618-4a34-b207-44dc712dd12b-metrics-tls\") pod \"dns-default-fqj8h\" (UID: \"7337a38e-9618-4a34-b207-44dc712dd12b\") " pod="openshift-dns/dns-default-fqj8h" Jul 02 21:36:40.224815 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:40.224610 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 21:36:40.224815 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:40.224665 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7337a38e-9618-4a34-b207-44dc712dd12b-metrics-tls podName:7337a38e-9618-4a34-b207-44dc712dd12b nodeName:}" failed. No retries permitted until 2026-07-02 21:36:44.224648407 +0000 UTC m=+43.563618836 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7337a38e-9618-4a34-b207-44dc712dd12b-metrics-tls") pod "dns-default-fqj8h" (UID: "7337a38e-9618-4a34-b207-44dc712dd12b") : secret "dns-default-metrics-tls" not found Jul 02 21:36:40.224815 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:40.224703 2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 21:36:40.224815 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:40.224721 2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7c596bcd6b-rzj8c: secret "image-registry-tls" not found Jul 02 21:36:40.224815 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:40.224767 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2dc606a9-6a0c-49de-be3e-4c090d343d3e-registry-tls podName:2dc606a9-6a0c-49de-be3e-4c090d343d3e nodeName:}" failed. No retries permitted until 2026-07-02 21:36:44.224751906 +0000 UTC m=+43.563722325 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2dc606a9-6a0c-49de-be3e-4c090d343d3e-registry-tls") pod "image-registry-7c596bcd6b-rzj8c" (UID: "2dc606a9-6a0c-49de-be3e-4c090d343d3e") : secret "image-registry-tls" not found Jul 02 21:36:41.188591 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:41.188551 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qvnrl" event={"ID":"06f7dfd3-0cdd-4dd0-a5ed-3c4237c4180b","Type":"ContainerStarted","Data":"d39386841b2d84d6f9d2eaded1f1804b6fdd7fb3785dac6c60c8e8c6aeb91d9a"} Jul 02 21:36:41.216439 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:41.216390 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-qvnrl" podStartSLOduration=9.958043226000001 podStartE2EDuration="40.216375543s" totalImagesPullingTime="30.258332317s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 21:36:01 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:36:41.214956417 +0000 UTC m=+40.553926851" watchObservedRunningTime="2026-07-02 21:36:41.216375543 +0000 UTC m=+40.555345978" Jul 02 21:36:41.231393 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:41.231371 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/af3c0081-d6c4-418c-8cfc-84b0248778f1-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-rzzw8\" (UID: \"af3c0081-d6c4-418c-8cfc-84b0248778f1\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-rzzw8" Jul 02 21:36:41.231528 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:41.231513 2569 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 02 21:36:41.231575 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:41.231566 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af3c0081-d6c4-418c-8cfc-84b0248778f1-networking-console-plugin-cert podName:af3c0081-d6c4-418c-8cfc-84b0248778f1 nodeName:}" failed. No retries permitted until 2026-07-02 21:36:45.231552542 +0000 UTC m=+44.570522955 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/af3c0081-d6c4-418c-8cfc-84b0248778f1-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-rzzw8" (UID: "af3c0081-d6c4-418c-8cfc-84b0248778f1") : secret "networking-console-plugin-cert" not found Jul 02 21:36:42.196120 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:42.196081 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rw2zb" event={"ID":"3d916985-dd9a-4407-8ed4-e302a9670f50","Type":"ContainerStarted","Data":"13239b40771eba6ee658419fc252753c9568a3881a2ff4f6c933be03997d8e47"} Jul 02 21:36:42.210861 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:42.210813 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-rw2zb" podStartSLOduration=3.030220111 podStartE2EDuration="6.210798513s" totalImagesPullingTime="3.180578402s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:36:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:36:42.210069574 +0000 UTC m=+41.549040032" watchObservedRunningTime="2026-07-02 21:36:42.210798513 +0000 UTC m=+41.549768938" Jul 02 21:36:44.255566 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:44.255534 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2dc606a9-6a0c-49de-be3e-4c090d343d3e-registry-tls\") pod \"image-registry-7c596bcd6b-rzj8c\" (UID: \"2dc606a9-6a0c-49de-be3e-4c090d343d3e\") " pod="openshift-image-registry/image-registry-7c596bcd6b-rzj8c" Jul 02 21:36:44.255930 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:44.255589 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7337a38e-9618-4a34-b207-44dc712dd12b-metrics-tls\") pod \"dns-default-fqj8h\" (UID: \"7337a38e-9618-4a34-b207-44dc712dd12b\") " pod="openshift-dns/dns-default-fqj8h" Jul 02 21:36:44.255930 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:44.255692 2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 21:36:44.255930 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:44.255704 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 21:36:44.255930 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:44.255708 2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7c596bcd6b-rzj8c: secret "image-registry-tls" not found Jul 02 21:36:44.255930 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:44.255762 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7337a38e-9618-4a34-b207-44dc712dd12b-metrics-tls podName:7337a38e-9618-4a34-b207-44dc712dd12b nodeName:}" failed. No retries permitted until 2026-07-02 21:36:52.255745277 +0000 UTC m=+51.594715691 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7337a38e-9618-4a34-b207-44dc712dd12b-metrics-tls") pod "dns-default-fqj8h" (UID: "7337a38e-9618-4a34-b207-44dc712dd12b") : secret "dns-default-metrics-tls" not found Jul 02 21:36:44.255930 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:44.255778 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2dc606a9-6a0c-49de-be3e-4c090d343d3e-registry-tls podName:2dc606a9-6a0c-49de-be3e-4c090d343d3e nodeName:}" failed. No retries permitted until 2026-07-02 21:36:52.255770544 +0000 UTC m=+51.594740956 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2dc606a9-6a0c-49de-be3e-4c090d343d3e-registry-tls") pod "image-registry-7c596bcd6b-rzj8c" (UID: "2dc606a9-6a0c-49de-be3e-4c090d343d3e") : secret "image-registry-tls" not found Jul 02 21:36:45.260710 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:45.260675 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/af3c0081-d6c4-418c-8cfc-84b0248778f1-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-rzzw8\" (UID: \"af3c0081-d6c4-418c-8cfc-84b0248778f1\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-rzzw8" Jul 02 21:36:45.261179 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:45.260855 2569 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 02 21:36:45.261179 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:45.260927 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af3c0081-d6c4-418c-8cfc-84b0248778f1-networking-console-plugin-cert podName:af3c0081-d6c4-418c-8cfc-84b0248778f1 nodeName:}" failed. No retries permitted until 2026-07-02 21:36:53.260907336 +0000 UTC m=+52.599877752 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/af3c0081-d6c4-418c-8cfc-84b0248778f1-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-rzzw8" (UID: "af3c0081-d6c4-418c-8cfc-84b0248778f1") : secret "networking-console-plugin-cert" not found Jul 02 21:36:47.547277 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:47.547241 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-fsdjr"] Jul 02 21:36:47.569724 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:47.569700 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fsdjr" Jul 02 21:36:47.574781 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:47.574758 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 02 21:36:47.574924 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:47.574839 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 02 21:36:47.574924 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:47.574814 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 02 21:36:47.574924 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:47.574762 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-pbj6p\"" Jul 02 21:36:47.575283 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:47.575268 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 02 21:36:47.579329 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:47.579307 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fsdjr"] Jul 02 21:36:47.678704 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:47.678678 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqqsv\" (UniqueName: \"kubernetes.io/projected/f9e06bbd-441d-43a5-afdd-48eb709a5ecd-kube-api-access-pqqsv\") pod \"insights-runtime-extractor-fsdjr\" (UID: \"f9e06bbd-441d-43a5-afdd-48eb709a5ecd\") " pod="openshift-insights/insights-runtime-extractor-fsdjr" Jul 02 21:36:47.678909 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:47.678738 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f9e06bbd-441d-43a5-afdd-48eb709a5ecd-crio-socket\") pod \"insights-runtime-extractor-fsdjr\" (UID: \"f9e06bbd-441d-43a5-afdd-48eb709a5ecd\") " pod="openshift-insights/insights-runtime-extractor-fsdjr" Jul 02 21:36:47.678909 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:47.678755 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9e06bbd-441d-43a5-afdd-48eb709a5ecd-data-volume\") pod \"insights-runtime-extractor-fsdjr\" (UID: \"f9e06bbd-441d-43a5-afdd-48eb709a5ecd\") " pod="openshift-insights/insights-runtime-extractor-fsdjr" Jul 02 21:36:47.678909 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:47.678772 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f9e06bbd-441d-43a5-afdd-48eb709a5ecd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fsdjr\" (UID: \"f9e06bbd-441d-43a5-afdd-48eb709a5ecd\") " pod="openshift-insights/insights-runtime-extractor-fsdjr" Jul 02 21:36:47.678909 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:47.678834 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f9e06bbd-441d-43a5-afdd-48eb709a5ecd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fsdjr\" (UID: \"f9e06bbd-441d-43a5-afdd-48eb709a5ecd\") " pod="openshift-insights/insights-runtime-extractor-fsdjr" Jul 02 21:36:47.780030 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:47.779999 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqqsv\" (UniqueName: \"kubernetes.io/projected/f9e06bbd-441d-43a5-afdd-48eb709a5ecd-kube-api-access-pqqsv\") pod \"insights-runtime-extractor-fsdjr\" (UID: \"f9e06bbd-441d-43a5-afdd-48eb709a5ecd\") " pod="openshift-insights/insights-runtime-extractor-fsdjr" Jul 02 21:36:47.780187 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:47.780071 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f9e06bbd-441d-43a5-afdd-48eb709a5ecd-crio-socket\") pod \"insights-runtime-extractor-fsdjr\" (UID: \"f9e06bbd-441d-43a5-afdd-48eb709a5ecd\") " pod="openshift-insights/insights-runtime-extractor-fsdjr" Jul 02 21:36:47.780187 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:47.780100 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9e06bbd-441d-43a5-afdd-48eb709a5ecd-data-volume\") pod \"insights-runtime-extractor-fsdjr\" (UID: \"f9e06bbd-441d-43a5-afdd-48eb709a5ecd\") " pod="openshift-insights/insights-runtime-extractor-fsdjr" Jul 02 21:36:47.780187 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:47.780125 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f9e06bbd-441d-43a5-afdd-48eb709a5ecd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fsdjr\" (UID: \"f9e06bbd-441d-43a5-afdd-48eb709a5ecd\") " pod="openshift-insights/insights-runtime-extractor-fsdjr" Jul 02 21:36:47.780187 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:47.780163 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f9e06bbd-441d-43a5-afdd-48eb709a5ecd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fsdjr\" (UID: \"f9e06bbd-441d-43a5-afdd-48eb709a5ecd\") " pod="openshift-insights/insights-runtime-extractor-fsdjr" Jul 02 21:36:47.780387 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:47.780240 2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 21:36:47.780387 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:47.780278 2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 21:36:47.780387 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:47.780311 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f9e06bbd-441d-43a5-afdd-48eb709a5ecd-kube-rbac-proxy-cm podName:f9e06bbd-441d-43a5-afdd-48eb709a5ecd nodeName:}" failed. No retries permitted until 2026-07-02 21:36:48.280293695 +0000 UTC m=+47.619264109 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f9e06bbd-441d-43a5-afdd-48eb709a5ecd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fsdjr" (UID: "f9e06bbd-441d-43a5-afdd-48eb709a5ecd") : configmap "kube-rbac-proxy" not found Jul 02 21:36:47.780387 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:47.780349 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f9e06bbd-441d-43a5-afdd-48eb709a5ecd-crio-socket\") pod \"insights-runtime-extractor-fsdjr\" (UID: \"f9e06bbd-441d-43a5-afdd-48eb709a5ecd\") " pod="openshift-insights/insights-runtime-extractor-fsdjr" Jul 02 21:36:47.780575 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:47.780419 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9e06bbd-441d-43a5-afdd-48eb709a5ecd-insights-runtime-extractor-tls podName:f9e06bbd-441d-43a5-afdd-48eb709a5ecd nodeName:}" failed. No retries permitted until 2026-07-02 21:36:48.280389794 +0000 UTC m=+47.619360212 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f9e06bbd-441d-43a5-afdd-48eb709a5ecd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fsdjr" (UID: "f9e06bbd-441d-43a5-afdd-48eb709a5ecd") : secret "insights-runtime-extractor-tls" not found Jul 02 21:36:47.780575 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:47.780496 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9e06bbd-441d-43a5-afdd-48eb709a5ecd-data-volume\") pod \"insights-runtime-extractor-fsdjr\" (UID: \"f9e06bbd-441d-43a5-afdd-48eb709a5ecd\") " pod="openshift-insights/insights-runtime-extractor-fsdjr" Jul 02 21:36:47.802839 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:47.802763 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqqsv\" (UniqueName: \"kubernetes.io/projected/f9e06bbd-441d-43a5-afdd-48eb709a5ecd-kube-api-access-pqqsv\") pod \"insights-runtime-extractor-fsdjr\" (UID: \"f9e06bbd-441d-43a5-afdd-48eb709a5ecd\") " pod="openshift-insights/insights-runtime-extractor-fsdjr" Jul 02 21:36:48.284662 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:48.284636 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f9e06bbd-441d-43a5-afdd-48eb709a5ecd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fsdjr\" (UID: \"f9e06bbd-441d-43a5-afdd-48eb709a5ecd\") " pod="openshift-insights/insights-runtime-extractor-fsdjr" Jul 02 21:36:48.284837 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:48.284731 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f9e06bbd-441d-43a5-afdd-48eb709a5ecd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fsdjr\" (UID: \"f9e06bbd-441d-43a5-afdd-48eb709a5ecd\") " pod="openshift-insights/insights-runtime-extractor-fsdjr" Jul 02 21:36:48.284837 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:48.284758 2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 21:36:48.284931 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:48.284839 2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 21:36:48.284931 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:48.284841 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f9e06bbd-441d-43a5-afdd-48eb709a5ecd-kube-rbac-proxy-cm podName:f9e06bbd-441d-43a5-afdd-48eb709a5ecd nodeName:}" failed. No retries permitted until 2026-07-02 21:36:49.28482781 +0000 UTC m=+48.623798227 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f9e06bbd-441d-43a5-afdd-48eb709a5ecd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fsdjr" (UID: "f9e06bbd-441d-43a5-afdd-48eb709a5ecd") : configmap "kube-rbac-proxy" not found Jul 02 21:36:48.284931 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:48.284892 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9e06bbd-441d-43a5-afdd-48eb709a5ecd-insights-runtime-extractor-tls podName:f9e06bbd-441d-43a5-afdd-48eb709a5ecd nodeName:}" failed. No retries permitted until 2026-07-02 21:36:49.284882315 +0000 UTC m=+48.623852728 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f9e06bbd-441d-43a5-afdd-48eb709a5ecd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fsdjr" (UID: "f9e06bbd-441d-43a5-afdd-48eb709a5ecd") : secret "insights-runtime-extractor-tls" not found Jul 02 21:36:49.291551 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:49.291512 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f9e06bbd-441d-43a5-afdd-48eb709a5ecd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fsdjr\" (UID: \"f9e06bbd-441d-43a5-afdd-48eb709a5ecd\") " pod="openshift-insights/insights-runtime-extractor-fsdjr" Jul 02 21:36:49.292027 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:49.291562 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f9e06bbd-441d-43a5-afdd-48eb709a5ecd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fsdjr\" (UID: \"f9e06bbd-441d-43a5-afdd-48eb709a5ecd\") " pod="openshift-insights/insights-runtime-extractor-fsdjr" Jul 02 21:36:49.292027 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:49.291634 2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 21:36:49.292027 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:49.291679 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f9e06bbd-441d-43a5-afdd-48eb709a5ecd-kube-rbac-proxy-cm podName:f9e06bbd-441d-43a5-afdd-48eb709a5ecd nodeName:}" failed. No retries permitted until 2026-07-02 21:36:51.291666234 +0000 UTC m=+50.630636647 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f9e06bbd-441d-43a5-afdd-48eb709a5ecd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fsdjr" (UID: "f9e06bbd-441d-43a5-afdd-48eb709a5ecd") : configmap "kube-rbac-proxy" not found Jul 02 21:36:49.292027 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:49.291681 2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 21:36:49.292027 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:49.291745 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9e06bbd-441d-43a5-afdd-48eb709a5ecd-insights-runtime-extractor-tls podName:f9e06bbd-441d-43a5-afdd-48eb709a5ecd nodeName:}" failed. No retries permitted until 2026-07-02 21:36:51.291728983 +0000 UTC m=+50.630699398 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f9e06bbd-441d-43a5-afdd-48eb709a5ecd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fsdjr" (UID: "f9e06bbd-441d-43a5-afdd-48eb709a5ecd") : secret "insights-runtime-extractor-tls" not found Jul 02 21:36:49.352170 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:49.352147 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bf9dd_1ed8e72b-e89d-42d5-94f7-3c00d5bb1ca9/dns-node-resolver/0.log" Jul 02 21:36:50.055783 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:50.055750 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-f5q5q"] Jul 02 21:36:50.082846 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:50.082817 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-f5q5q"] Jul 02 21:36:50.082996 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:50.082915 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-f5q5q" Jul 02 21:36:50.085258 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:50.085229 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 02 21:36:50.085389 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:50.085260 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 02 21:36:50.085389 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:50.085242 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 02 21:36:50.085389 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:50.085297 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 02 21:36:50.086304 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:50.086288 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-sn9rq\"" Jul 02 21:36:50.199301 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:50.199262 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/eb960f65-bb21-4786-bf82-4ad6cae6e080-signing-key\") pod \"service-ca-758f599c5-f5q5q\" (UID: \"eb960f65-bb21-4786-bf82-4ad6cae6e080\") " pod="openshift-service-ca/service-ca-758f599c5-f5q5q" Jul 02 21:36:50.199487 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:50.199338 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/eb960f65-bb21-4786-bf82-4ad6cae6e080-signing-cabundle\") pod \"service-ca-758f599c5-f5q5q\" (UID: \"eb960f65-bb21-4786-bf82-4ad6cae6e080\") " pod="openshift-service-ca/service-ca-758f599c5-f5q5q" Jul 02 21:36:50.199487 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:50.199358 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9x7m7\" (UniqueName: \"kubernetes.io/projected/eb960f65-bb21-4786-bf82-4ad6cae6e080-kube-api-access-9x7m7\") pod \"service-ca-758f599c5-f5q5q\" (UID: \"eb960f65-bb21-4786-bf82-4ad6cae6e080\") " pod="openshift-service-ca/service-ca-758f599c5-f5q5q" Jul 02 21:36:50.299759 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:50.299725 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/eb960f65-bb21-4786-bf82-4ad6cae6e080-signing-key\") pod \"service-ca-758f599c5-f5q5q\" (UID: \"eb960f65-bb21-4786-bf82-4ad6cae6e080\") " pod="openshift-service-ca/service-ca-758f599c5-f5q5q" Jul 02 21:36:50.300234 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:50.299820 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/eb960f65-bb21-4786-bf82-4ad6cae6e080-signing-cabundle\") pod \"service-ca-758f599c5-f5q5q\" (UID: \"eb960f65-bb21-4786-bf82-4ad6cae6e080\") " pod="openshift-service-ca/service-ca-758f599c5-f5q5q" Jul 02 21:36:50.300234 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:50.299921 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9x7m7\" (UniqueName: \"kubernetes.io/projected/eb960f65-bb21-4786-bf82-4ad6cae6e080-kube-api-access-9x7m7\") pod \"service-ca-758f599c5-f5q5q\" (UID: \"eb960f65-bb21-4786-bf82-4ad6cae6e080\") " pod="openshift-service-ca/service-ca-758f599c5-f5q5q" Jul 02 21:36:50.300700 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:50.300676 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/eb960f65-bb21-4786-bf82-4ad6cae6e080-signing-cabundle\") pod \"service-ca-758f599c5-f5q5q\" (UID: \"eb960f65-bb21-4786-bf82-4ad6cae6e080\") " pod="openshift-service-ca/service-ca-758f599c5-f5q5q" Jul 02 21:36:50.302106 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:50.302088 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/eb960f65-bb21-4786-bf82-4ad6cae6e080-signing-key\") pod \"service-ca-758f599c5-f5q5q\" (UID: \"eb960f65-bb21-4786-bf82-4ad6cae6e080\") " pod="openshift-service-ca/service-ca-758f599c5-f5q5q" Jul 02 21:36:50.309013 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:50.308963 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9x7m7\" (UniqueName: \"kubernetes.io/projected/eb960f65-bb21-4786-bf82-4ad6cae6e080-kube-api-access-9x7m7\") pod \"service-ca-758f599c5-f5q5q\" (UID: \"eb960f65-bb21-4786-bf82-4ad6cae6e080\") " pod="openshift-service-ca/service-ca-758f599c5-f5q5q" Jul 02 21:36:50.348279 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:50.348253 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4rgsb_283e60d2-fc01-4709-afeb-73699b0d8134/node-ca/0.log" Jul 02 21:36:50.391550 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:50.391523 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-f5q5q" Jul 02 21:36:50.545716 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:50.545686 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-f5q5q"] Jul 02 21:36:50.548369 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:36:50.548346 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb960f65_bb21_4786_bf82_4ad6cae6e080.slice/crio-85773f99f9abac435e4125ac382446d02786f2bf37b78990b5f8cfa25a691018 WatchSource:0}: Error finding container 85773f99f9abac435e4125ac382446d02786f2bf37b78990b5f8cfa25a691018: Status 404 returned error can't find the container with id 85773f99f9abac435e4125ac382446d02786f2bf37b78990b5f8cfa25a691018 Jul 02 21:36:51.219893 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:51.217874 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-f5q5q" event={"ID":"eb960f65-bb21-4786-bf82-4ad6cae6e080","Type":"ContainerStarted","Data":"85773f99f9abac435e4125ac382446d02786f2bf37b78990b5f8cfa25a691018"} Jul 02 21:36:51.309827 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:51.309778 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f9e06bbd-441d-43a5-afdd-48eb709a5ecd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fsdjr\" (UID: \"f9e06bbd-441d-43a5-afdd-48eb709a5ecd\") " pod="openshift-insights/insights-runtime-extractor-fsdjr" Jul 02 21:36:51.310276 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:51.309848 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f9e06bbd-441d-43a5-afdd-48eb709a5ecd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fsdjr\" (UID: \"f9e06bbd-441d-43a5-afdd-48eb709a5ecd\") " pod="openshift-insights/insights-runtime-extractor-fsdjr" Jul 02 21:36:51.310276 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:51.309951 2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 21:36:51.310276 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:51.309988 2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 21:36:51.310276 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:51.310009 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f9e06bbd-441d-43a5-afdd-48eb709a5ecd-kube-rbac-proxy-cm podName:f9e06bbd-441d-43a5-afdd-48eb709a5ecd nodeName:}" failed. No retries permitted until 2026-07-02 21:36:55.309995276 +0000 UTC m=+54.648965691 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f9e06bbd-441d-43a5-afdd-48eb709a5ecd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fsdjr" (UID: "f9e06bbd-441d-43a5-afdd-48eb709a5ecd") : configmap "kube-rbac-proxy" not found Jul 02 21:36:51.310276 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:51.310058 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9e06bbd-441d-43a5-afdd-48eb709a5ecd-insights-runtime-extractor-tls podName:f9e06bbd-441d-43a5-afdd-48eb709a5ecd nodeName:}" failed. No retries permitted until 2026-07-02 21:36:55.310036446 +0000 UTC m=+54.649006864 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f9e06bbd-441d-43a5-afdd-48eb709a5ecd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fsdjr" (UID: "f9e06bbd-441d-43a5-afdd-48eb709a5ecd") : secret "insights-runtime-extractor-tls" not found Jul 02 21:36:51.550085 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:51.550056 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rw2zb_3d916985-dd9a-4407-8ed4-e302a9670f50/serve-healthcheck-canary/0.log" Jul 02 21:36:52.318747 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:52.318712 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2dc606a9-6a0c-49de-be3e-4c090d343d3e-registry-tls\") pod \"image-registry-7c596bcd6b-rzj8c\" (UID: \"2dc606a9-6a0c-49de-be3e-4c090d343d3e\") " pod="openshift-image-registry/image-registry-7c596bcd6b-rzj8c" Jul 02 21:36:52.319202 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:52.318766 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7337a38e-9618-4a34-b207-44dc712dd12b-metrics-tls\") pod \"dns-default-fqj8h\" (UID: \"7337a38e-9618-4a34-b207-44dc712dd12b\") " pod="openshift-dns/dns-default-fqj8h" Jul 02 21:36:52.319202 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:52.318889 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 21:36:52.319202 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:52.318952 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7337a38e-9618-4a34-b207-44dc712dd12b-metrics-tls podName:7337a38e-9618-4a34-b207-44dc712dd12b nodeName:}" failed. No retries permitted until 2026-07-02 21:37:08.318938403 +0000 UTC m=+67.657908820 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7337a38e-9618-4a34-b207-44dc712dd12b-metrics-tls") pod "dns-default-fqj8h" (UID: "7337a38e-9618-4a34-b207-44dc712dd12b") : secret "dns-default-metrics-tls" not found Jul 02 21:36:52.319202 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:52.318890 2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 21:36:52.319202 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:52.318998 2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7c596bcd6b-rzj8c: secret "image-registry-tls" not found Jul 02 21:36:52.319202 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:52.319051 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2dc606a9-6a0c-49de-be3e-4c090d343d3e-registry-tls podName:2dc606a9-6a0c-49de-be3e-4c090d343d3e nodeName:}" failed. No retries permitted until 2026-07-02 21:37:08.319034532 +0000 UTC m=+67.658004946 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2dc606a9-6a0c-49de-be3e-4c090d343d3e-registry-tls") pod "image-registry-7c596bcd6b-rzj8c" (UID: "2dc606a9-6a0c-49de-be3e-4c090d343d3e") : secret "image-registry-tls" not found Jul 02 21:36:53.224140 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:53.224063 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-f5q5q" event={"ID":"eb960f65-bb21-4786-bf82-4ad6cae6e080","Type":"ContainerStarted","Data":"4b5a902eaf9d1573376cde34fca92a39790a64c452ceb26efb085d035f2f57d1"} Jul 02 21:36:53.240678 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:53.240636 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-f5q5q" podStartSLOduration=0.843503609 podStartE2EDuration="3.240621545s" totalImagesPullingTime="2.397117936s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:36:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:36:53.239263659 +0000 UTC m=+52.578234099" watchObservedRunningTime="2026-07-02 21:36:53.240621545 +0000 UTC m=+52.579591981" Jul 02 21:36:53.326101 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:53.326069 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/af3c0081-d6c4-418c-8cfc-84b0248778f1-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-rzzw8\" (UID: \"af3c0081-d6c4-418c-8cfc-84b0248778f1\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-rzzw8" Jul 02 21:36:53.326482 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:53.326233 2569 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 02 21:36:53.326482 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:53.326304 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af3c0081-d6c4-418c-8cfc-84b0248778f1-networking-console-plugin-cert podName:af3c0081-d6c4-418c-8cfc-84b0248778f1 nodeName:}" failed. No retries permitted until 2026-07-02 21:37:09.326286368 +0000 UTC m=+68.665256791 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/af3c0081-d6c4-418c-8cfc-84b0248778f1-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-rzzw8" (UID: "af3c0081-d6c4-418c-8cfc-84b0248778f1") : secret "networking-console-plugin-cert" not found Jul 02 21:36:55.345289 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:55.345261 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f9e06bbd-441d-43a5-afdd-48eb709a5ecd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fsdjr\" (UID: \"f9e06bbd-441d-43a5-afdd-48eb709a5ecd\") " pod="openshift-insights/insights-runtime-extractor-fsdjr" Jul 02 21:36:55.345709 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:55.345301 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f9e06bbd-441d-43a5-afdd-48eb709a5ecd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fsdjr\" (UID: \"f9e06bbd-441d-43a5-afdd-48eb709a5ecd\") " pod="openshift-insights/insights-runtime-extractor-fsdjr" Jul 02 21:36:55.345709 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:55.345412 2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 21:36:55.345709 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:36:55.345472 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9e06bbd-441d-43a5-afdd-48eb709a5ecd-insights-runtime-extractor-tls podName:f9e06bbd-441d-43a5-afdd-48eb709a5ecd nodeName:}" failed. No retries permitted until 2026-07-02 21:37:03.345455818 +0000 UTC m=+62.684426231 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f9e06bbd-441d-43a5-afdd-48eb709a5ecd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fsdjr" (UID: "f9e06bbd-441d-43a5-afdd-48eb709a5ecd") : secret "insights-runtime-extractor-tls" not found Jul 02 21:36:55.345835 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:36:55.345771 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f9e06bbd-441d-43a5-afdd-48eb709a5ecd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fsdjr\" (UID: \"f9e06bbd-441d-43a5-afdd-48eb709a5ecd\") " pod="openshift-insights/insights-runtime-extractor-fsdjr" Jul 02 21:37:03.164962 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:03.164931 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-twszg" Jul 02 21:37:03.408153 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:03.408120 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f9e06bbd-441d-43a5-afdd-48eb709a5ecd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fsdjr\" (UID: \"f9e06bbd-441d-43a5-afdd-48eb709a5ecd\") " pod="openshift-insights/insights-runtime-extractor-fsdjr" Jul 02 21:37:03.410253 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:03.410234 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f9e06bbd-441d-43a5-afdd-48eb709a5ecd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fsdjr\" (UID: \"f9e06bbd-441d-43a5-afdd-48eb709a5ecd\") " pod="openshift-insights/insights-runtime-extractor-fsdjr" Jul 02 21:37:03.481405 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:03.481347 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-pbj6p\"" Jul 02 21:37:03.489302 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:03.489285 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fsdjr" Jul 02 21:37:03.605876 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:03.605848 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fsdjr"] Jul 02 21:37:03.608984 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:37:03.608964 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9e06bbd_441d_43a5_afdd_48eb709a5ecd.slice/crio-929556a2720d674f6502069cc07444bf2a93ec93abeec0668baafcea3cbf57f9 WatchSource:0}: Error finding container 929556a2720d674f6502069cc07444bf2a93ec93abeec0668baafcea3cbf57f9: Status 404 returned error can't find the container with id 929556a2720d674f6502069cc07444bf2a93ec93abeec0668baafcea3cbf57f9 Jul 02 21:37:04.253383 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:04.253145 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fsdjr" event={"ID":"f9e06bbd-441d-43a5-afdd-48eb709a5ecd","Type":"ContainerStarted","Data":"929556a2720d674f6502069cc07444bf2a93ec93abeec0668baafcea3cbf57f9"} Jul 02 21:37:06.938248 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:06.938189 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d95b88bb-30e2-421a-84a7-96de7cd5d1e1-original-pull-secret\") pod \"global-pull-secret-syncer-j9flr\" (UID: \"d95b88bb-30e2-421a-84a7-96de7cd5d1e1\") " pod="kube-system/global-pull-secret-syncer-j9flr" Jul 02 21:37:06.938691 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:06.938304 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30f00b78-e6a8-48f7-b7b5-801cf436f80b-metrics-certs\") pod \"network-metrics-daemon-cn45l\" (UID: \"30f00b78-e6a8-48f7-b7b5-801cf436f80b\") " pod="openshift-multus/network-metrics-daemon-cn45l" Jul 02 21:37:06.940981 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:06.940954 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 02 21:37:06.941802 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:06.941761 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 02 21:37:06.951223 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:06.951195 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d95b88bb-30e2-421a-84a7-96de7cd5d1e1-original-pull-secret\") pod \"global-pull-secret-syncer-j9flr\" (UID: \"d95b88bb-30e2-421a-84a7-96de7cd5d1e1\") " pod="kube-system/global-pull-secret-syncer-j9flr" Jul 02 21:37:06.951344 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:06.951274 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30f00b78-e6a8-48f7-b7b5-801cf436f80b-metrics-certs\") pod \"network-metrics-daemon-cn45l\" (UID: \"30f00b78-e6a8-48f7-b7b5-801cf436f80b\") " pod="openshift-multus/network-metrics-daemon-cn45l" Jul 02 21:37:07.038890 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:07.038840 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bk8rv\" (UniqueName: \"kubernetes.io/projected/42124588-1a15-4e43-a5cb-18c8142bc434-kube-api-access-bk8rv\") pod \"network-check-target-jgn9h\" (UID: \"42124588-1a15-4e43-a5cb-18c8142bc434\") " pod="openshift-network-diagnostics/network-check-target-jgn9h" Jul 02 21:37:07.041852 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:07.041827 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 02 21:37:07.052016 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:07.051986 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 02 21:37:07.059814 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:07.059781 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j9flr" Jul 02 21:37:07.063450 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:07.063431 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bk8rv\" (UniqueName: \"kubernetes.io/projected/42124588-1a15-4e43-a5cb-18c8142bc434-kube-api-access-bk8rv\") pod \"network-check-target-jgn9h\" (UID: \"42124588-1a15-4e43-a5cb-18c8142bc434\") " pod="openshift-network-diagnostics/network-check-target-jgn9h" Jul 02 21:37:07.066320 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:07.066302 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-dkhhs\"" Jul 02 21:37:07.075183 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:07.075162 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cn45l" Jul 02 21:37:07.263875 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:07.262862 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ckjxv\"" Jul 02 21:37:07.270609 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:07.270586 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jgn9h" Jul 02 21:37:07.355060 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:07.354967 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cn45l"] Jul 02 21:37:07.358054 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:37:07.358016 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30f00b78_e6a8_48f7_b7b5_801cf436f80b.slice/crio-6b5196828a98b5b7f9368f799e7f545100e7d2377002bde2b68955845ba05e1f WatchSource:0}: Error finding container 6b5196828a98b5b7f9368f799e7f545100e7d2377002bde2b68955845ba05e1f: Status 404 returned error can't find the container with id 6b5196828a98b5b7f9368f799e7f545100e7d2377002bde2b68955845ba05e1f Jul 02 21:37:07.371281 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:07.371251 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-j9flr"] Jul 02 21:37:07.374903 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:37:07.374871 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd95b88bb_30e2_421a_84a7_96de7cd5d1e1.slice/crio-87b6866666c8533aecb894bf86a5f9036a2f60af3b405ca6be70185c45142284 WatchSource:0}: Error finding container 87b6866666c8533aecb894bf86a5f9036a2f60af3b405ca6be70185c45142284: Status 404 returned error can't find the container with id 87b6866666c8533aecb894bf86a5f9036a2f60af3b405ca6be70185c45142284 Jul 02 21:37:07.411544 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:07.411515 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jgn9h"] Jul 02 21:37:07.414603 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:37:07.414580 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42124588_1a15_4e43_a5cb_18c8142bc434.slice/crio-3b570022446112d8723312816b1e4afbbb42027753aa82357c6713191bfb491a WatchSource:0}: Error finding container 3b570022446112d8723312816b1e4afbbb42027753aa82357c6713191bfb491a: Status 404 returned error can't find the container with id 3b570022446112d8723312816b1e4afbbb42027753aa82357c6713191bfb491a Jul 02 21:37:08.271217 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:08.271177 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fsdjr" event={"ID":"f9e06bbd-441d-43a5-afdd-48eb709a5ecd","Type":"ContainerStarted","Data":"3b172523fb3687949db9e11738f912860dca3006b2ef02ddfec3d09da4dbd824"} Jul 02 21:37:08.271677 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:08.271250 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fsdjr" event={"ID":"f9e06bbd-441d-43a5-afdd-48eb709a5ecd","Type":"ContainerStarted","Data":"21ab2d78482095f68c622fb0c8ca2e060214a2d6d81117110137f0936c8150e6"} Jul 02 21:37:08.274278 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:08.274248 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cn45l" event={"ID":"30f00b78-e6a8-48f7-b7b5-801cf436f80b","Type":"ContainerStarted","Data":"6b5196828a98b5b7f9368f799e7f545100e7d2377002bde2b68955845ba05e1f"} Jul 02 21:37:08.275846 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:08.275815 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jgn9h" event={"ID":"42124588-1a15-4e43-a5cb-18c8142bc434","Type":"ContainerStarted","Data":"3b570022446112d8723312816b1e4afbbb42027753aa82357c6713191bfb491a"} Jul 02 21:37:08.277396 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:08.277375 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-j9flr" event={"ID":"d95b88bb-30e2-421a-84a7-96de7cd5d1e1","Type":"ContainerStarted","Data":"87b6866666c8533aecb894bf86a5f9036a2f60af3b405ca6be70185c45142284"} Jul 02 21:37:08.348670 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:08.348629 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2dc606a9-6a0c-49de-be3e-4c090d343d3e-registry-tls\") pod \"image-registry-7c596bcd6b-rzj8c\" (UID: \"2dc606a9-6a0c-49de-be3e-4c090d343d3e\") " pod="openshift-image-registry/image-registry-7c596bcd6b-rzj8c" Jul 02 21:37:08.348841 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:08.348694 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7337a38e-9618-4a34-b207-44dc712dd12b-metrics-tls\") pod \"dns-default-fqj8h\" (UID: \"7337a38e-9618-4a34-b207-44dc712dd12b\") " pod="openshift-dns/dns-default-fqj8h" Jul 02 21:37:08.351558 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:08.351529 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7337a38e-9618-4a34-b207-44dc712dd12b-metrics-tls\") pod \"dns-default-fqj8h\" (UID: \"7337a38e-9618-4a34-b207-44dc712dd12b\") " pod="openshift-dns/dns-default-fqj8h" Jul 02 21:37:08.351817 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:08.351779 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2dc606a9-6a0c-49de-be3e-4c090d343d3e-registry-tls\") pod \"image-registry-7c596bcd6b-rzj8c\" (UID: \"2dc606a9-6a0c-49de-be3e-4c090d343d3e\") " pod="openshift-image-registry/image-registry-7c596bcd6b-rzj8c" Jul 02 21:37:08.576276 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:08.576241 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-wzgxn\"" Jul 02 21:37:08.584191 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:08.584140 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c596bcd6b-rzj8c" Jul 02 21:37:08.589581 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:08.589549 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bv4sq\"" Jul 02 21:37:08.598416 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:08.598376 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fqj8h" Jul 02 21:37:08.744465 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:08.744431 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7c596bcd6b-rzj8c"] Jul 02 21:37:08.747960 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:37:08.747898 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2dc606a9_6a0c_49de_be3e_4c090d343d3e.slice/crio-6d9de92d643f496db3379990099e91626b20e8a5f111bb0072b8c8dabcbf35ca WatchSource:0}: Error finding container 6d9de92d643f496db3379990099e91626b20e8a5f111bb0072b8c8dabcbf35ca: Status 404 returned error can't find the container with id 6d9de92d643f496db3379990099e91626b20e8a5f111bb0072b8c8dabcbf35ca Jul 02 21:37:08.922584 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:08.922497 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fqj8h"] Jul 02 21:37:08.934536 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:37:08.934502 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7337a38e_9618_4a34_b207_44dc712dd12b.slice/crio-0c627331b13c7072f2688308ca300e446de5006365a0695aeb908952c08e8df2 WatchSource:0}: Error finding container 0c627331b13c7072f2688308ca300e446de5006365a0695aeb908952c08e8df2: Status 404 returned error can't find the container with id 0c627331b13c7072f2688308ca300e446de5006365a0695aeb908952c08e8df2 Jul 02 21:37:09.171458 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.171339 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7c596bcd6b-rzj8c"] Jul 02 21:37:09.182148 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.179516 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5468bfc94c-dptkz"] Jul 02 21:37:09.199389 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.199362 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5468bfc94c-dptkz"] Jul 02 21:37:09.199564 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.199544 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5468bfc94c-dptkz" Jul 02 21:37:09.204185 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.203305 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 02 21:37:09.204185 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.203762 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 02 21:37:09.204185 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.203949 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 02 21:37:09.204185 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.203986 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 02 21:37:09.255314 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.255076 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pddb8\" (UniqueName: \"kubernetes.io/projected/93bd8222-3798-4572-b120-8ada8386ff4f-kube-api-access-pddb8\") pod \"klusterlet-addon-workmgr-5468bfc94c-dptkz\" (UID: \"93bd8222-3798-4572-b120-8ada8386ff4f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5468bfc94c-dptkz" Jul 02 21:37:09.255314 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.255128 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/93bd8222-3798-4572-b120-8ada8386ff4f-tmp\") pod \"klusterlet-addon-workmgr-5468bfc94c-dptkz\" (UID: \"93bd8222-3798-4572-b120-8ada8386ff4f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5468bfc94c-dptkz" Jul 02 21:37:09.255314 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.255160 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/93bd8222-3798-4572-b120-8ada8386ff4f-klusterlet-config\") pod \"klusterlet-addon-workmgr-5468bfc94c-dptkz\" (UID: \"93bd8222-3798-4572-b120-8ada8386ff4f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5468bfc94c-dptkz" Jul 02 21:37:09.279813 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.276889 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-f788c7bf6-4fwb6"] Jul 02 21:37:09.305824 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.305549 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7c596bcd6b-rzj8c" Jul 02 21:37:09.305824 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.305590 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-f788c7bf6-4fwb6"] Jul 02 21:37:09.305824 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.305616 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cn45l" event={"ID":"30f00b78-e6a8-48f7-b7b5-801cf436f80b","Type":"ContainerStarted","Data":"570c33a68d892369965c418654151960ffb1fc22b7abafbb66f6c4d2010fb4fc"} Jul 02 21:37:09.305824 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.305645 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cn45l" event={"ID":"30f00b78-e6a8-48f7-b7b5-801cf436f80b","Type":"ContainerStarted","Data":"999575b077d178f130c7ef55f9536d4f3a1da0a8ca9f74e4d4447fbbd42a11b6"} Jul 02 21:37:09.305824 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.305667 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fqj8h" event={"ID":"7337a38e-9618-4a34-b207-44dc712dd12b","Type":"ContainerStarted","Data":"0c627331b13c7072f2688308ca300e446de5006365a0695aeb908952c08e8df2"} Jul 02 21:37:09.305824 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.305693 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c596bcd6b-rzj8c" event={"ID":"2dc606a9-6a0c-49de-be3e-4c090d343d3e","Type":"ContainerStarted","Data":"6e4c6f719b027d67de6190c51309bec482ac8f0696d2548a32b58b1f74a304e8"} Jul 02 21:37:09.305824 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.305710 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c596bcd6b-rzj8c" event={"ID":"2dc606a9-6a0c-49de-be3e-4c090d343d3e","Type":"ContainerStarted","Data":"6d9de92d643f496db3379990099e91626b20e8a5f111bb0072b8c8dabcbf35ca"} Jul 02 21:37:09.306291 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.306218 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-f788c7bf6-4fwb6" Jul 02 21:37:09.311600 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.311528 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-cn45l" podStartSLOduration=66.90420812 podStartE2EDuration="1m8.311507902s" totalImagesPullingTime="1.407299782s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:36:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:37:09.309886678 +0000 UTC m=+68.648857126" watchObservedRunningTime="2026-07-02 21:37:09.311507902 +0000 UTC m=+68.650478339" Jul 02 21:37:09.334313 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.333473 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7c596bcd6b-rzj8c" podStartSLOduration=51.333452191 podStartE2EDuration="51.333452191s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:36:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:37:09.332623488 +0000 UTC m=+68.671593950" watchObservedRunningTime="2026-07-02 21:37:09.333452191 +0000 UTC m=+68.672422626" Jul 02 21:37:09.355629 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.355559 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/93bd8222-3798-4572-b120-8ada8386ff4f-tmp\") pod \"klusterlet-addon-workmgr-5468bfc94c-dptkz\" (UID: \"93bd8222-3798-4572-b120-8ada8386ff4f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5468bfc94c-dptkz" Jul 02 21:37:09.355737 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.355659 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/93bd8222-3798-4572-b120-8ada8386ff4f-klusterlet-config\") pod \"klusterlet-addon-workmgr-5468bfc94c-dptkz\" (UID: \"93bd8222-3798-4572-b120-8ada8386ff4f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5468bfc94c-dptkz" Jul 02 21:37:09.355737 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.355716 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1adcf74b-7adb-4c3e-859d-2481e729caf1-bound-sa-token\") pod \"image-registry-f788c7bf6-4fwb6\" (UID: \"1adcf74b-7adb-4c3e-859d-2481e729caf1\") " pod="openshift-image-registry/image-registry-f788c7bf6-4fwb6" Jul 02 21:37:09.355845 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.355757 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/af3c0081-d6c4-418c-8cfc-84b0248778f1-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-rzzw8\" (UID: \"af3c0081-d6c4-418c-8cfc-84b0248778f1\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-rzzw8" Jul 02 21:37:09.355845 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.355805 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1adcf74b-7adb-4c3e-859d-2481e729caf1-ca-trust-extracted\") pod \"image-registry-f788c7bf6-4fwb6\" (UID: \"1adcf74b-7adb-4c3e-859d-2481e729caf1\") " pod="openshift-image-registry/image-registry-f788c7bf6-4fwb6" Jul 02 21:37:09.355845 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.355833 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1adcf74b-7adb-4c3e-859d-2481e729caf1-registry-certificates\") pod \"image-registry-f788c7bf6-4fwb6\" (UID: \"1adcf74b-7adb-4c3e-859d-2481e729caf1\") " pod="openshift-image-registry/image-registry-f788c7bf6-4fwb6" Jul 02 21:37:09.355997 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.355860 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1adcf74b-7adb-4c3e-859d-2481e729caf1-registry-tls\") pod \"image-registry-f788c7bf6-4fwb6\" (UID: \"1adcf74b-7adb-4c3e-859d-2481e729caf1\") " pod="openshift-image-registry/image-registry-f788c7bf6-4fwb6" Jul 02 21:37:09.355997 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.355888 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2kdv\" (UniqueName: \"kubernetes.io/projected/1adcf74b-7adb-4c3e-859d-2481e729caf1-kube-api-access-r2kdv\") pod \"image-registry-f788c7bf6-4fwb6\" (UID: \"1adcf74b-7adb-4c3e-859d-2481e729caf1\") " pod="openshift-image-registry/image-registry-f788c7bf6-4fwb6" Jul 02 21:37:09.355997 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.355961 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1adcf74b-7adb-4c3e-859d-2481e729caf1-installation-pull-secrets\") pod \"image-registry-f788c7bf6-4fwb6\" (UID: \"1adcf74b-7adb-4c3e-859d-2481e729caf1\") " pod="openshift-image-registry/image-registry-f788c7bf6-4fwb6" Jul 02 21:37:09.355997 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.355967 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/93bd8222-3798-4572-b120-8ada8386ff4f-tmp\") pod \"klusterlet-addon-workmgr-5468bfc94c-dptkz\" (UID: \"93bd8222-3798-4572-b120-8ada8386ff4f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5468bfc94c-dptkz" Jul 02 21:37:09.356307 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.356009 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1adcf74b-7adb-4c3e-859d-2481e729caf1-image-registry-private-configuration\") pod \"image-registry-f788c7bf6-4fwb6\" (UID: \"1adcf74b-7adb-4c3e-859d-2481e729caf1\") " pod="openshift-image-registry/image-registry-f788c7bf6-4fwb6" Jul 02 21:37:09.356307 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.356044 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pddb8\" (UniqueName: \"kubernetes.io/projected/93bd8222-3798-4572-b120-8ada8386ff4f-kube-api-access-pddb8\") pod \"klusterlet-addon-workmgr-5468bfc94c-dptkz\" (UID: \"93bd8222-3798-4572-b120-8ada8386ff4f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5468bfc94c-dptkz" Jul 02 21:37:09.356307 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.356068 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1adcf74b-7adb-4c3e-859d-2481e729caf1-trusted-ca\") pod \"image-registry-f788c7bf6-4fwb6\" (UID: \"1adcf74b-7adb-4c3e-859d-2481e729caf1\") " pod="openshift-image-registry/image-registry-f788c7bf6-4fwb6" Jul 02 21:37:09.360077 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.360032 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/af3c0081-d6c4-418c-8cfc-84b0248778f1-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-rzzw8\" (UID: \"af3c0081-d6c4-418c-8cfc-84b0248778f1\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-rzzw8" Jul 02 21:37:09.363778 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.363751 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/93bd8222-3798-4572-b120-8ada8386ff4f-klusterlet-config\") pod \"klusterlet-addon-workmgr-5468bfc94c-dptkz\" (UID: \"93bd8222-3798-4572-b120-8ada8386ff4f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5468bfc94c-dptkz" Jul 02 21:37:09.370661 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.370619 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pddb8\" (UniqueName: \"kubernetes.io/projected/93bd8222-3798-4572-b120-8ada8386ff4f-kube-api-access-pddb8\") pod \"klusterlet-addon-workmgr-5468bfc94c-dptkz\" (UID: \"93bd8222-3798-4572-b120-8ada8386ff4f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5468bfc94c-dptkz" Jul 02 21:37:09.459641 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.456979 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1adcf74b-7adb-4c3e-859d-2481e729caf1-installation-pull-secrets\") pod \"image-registry-f788c7bf6-4fwb6\" (UID: \"1adcf74b-7adb-4c3e-859d-2481e729caf1\") " pod="openshift-image-registry/image-registry-f788c7bf6-4fwb6" Jul 02 21:37:09.459641 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.457028 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1adcf74b-7adb-4c3e-859d-2481e729caf1-image-registry-private-configuration\") pod \"image-registry-f788c7bf6-4fwb6\" (UID: \"1adcf74b-7adb-4c3e-859d-2481e729caf1\") " pod="openshift-image-registry/image-registry-f788c7bf6-4fwb6" Jul 02 21:37:09.459641 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.457058 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1adcf74b-7adb-4c3e-859d-2481e729caf1-trusted-ca\") pod \"image-registry-f788c7bf6-4fwb6\" (UID: \"1adcf74b-7adb-4c3e-859d-2481e729caf1\") " pod="openshift-image-registry/image-registry-f788c7bf6-4fwb6" Jul 02 21:37:09.459641 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.457106 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1adcf74b-7adb-4c3e-859d-2481e729caf1-bound-sa-token\") pod \"image-registry-f788c7bf6-4fwb6\" (UID: \"1adcf74b-7adb-4c3e-859d-2481e729caf1\") " pod="openshift-image-registry/image-registry-f788c7bf6-4fwb6" Jul 02 21:37:09.459641 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.457139 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1adcf74b-7adb-4c3e-859d-2481e729caf1-ca-trust-extracted\") pod \"image-registry-f788c7bf6-4fwb6\" (UID: \"1adcf74b-7adb-4c3e-859d-2481e729caf1\") " pod="openshift-image-registry/image-registry-f788c7bf6-4fwb6" Jul 02 21:37:09.459641 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.457173 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1adcf74b-7adb-4c3e-859d-2481e729caf1-registry-certificates\") pod \"image-registry-f788c7bf6-4fwb6\" (UID: \"1adcf74b-7adb-4c3e-859d-2481e729caf1\") " pod="openshift-image-registry/image-registry-f788c7bf6-4fwb6" Jul 02 21:37:09.459641 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.457196 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1adcf74b-7adb-4c3e-859d-2481e729caf1-registry-tls\") pod \"image-registry-f788c7bf6-4fwb6\" (UID: \"1adcf74b-7adb-4c3e-859d-2481e729caf1\") " pod="openshift-image-registry/image-registry-f788c7bf6-4fwb6" Jul 02 21:37:09.459641 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.457224 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2kdv\" (UniqueName: \"kubernetes.io/projected/1adcf74b-7adb-4c3e-859d-2481e729caf1-kube-api-access-r2kdv\") pod \"image-registry-f788c7bf6-4fwb6\" (UID: \"1adcf74b-7adb-4c3e-859d-2481e729caf1\") " pod="openshift-image-registry/image-registry-f788c7bf6-4fwb6" Jul 02 21:37:09.459641 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.458296 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1adcf74b-7adb-4c3e-859d-2481e729caf1-ca-trust-extracted\") pod \"image-registry-f788c7bf6-4fwb6\" (UID: \"1adcf74b-7adb-4c3e-859d-2481e729caf1\") " pod="openshift-image-registry/image-registry-f788c7bf6-4fwb6" Jul 02 21:37:09.459641 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.458355 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1adcf74b-7adb-4c3e-859d-2481e729caf1-registry-certificates\") pod \"image-registry-f788c7bf6-4fwb6\" (UID: \"1adcf74b-7adb-4c3e-859d-2481e729caf1\") " pod="openshift-image-registry/image-registry-f788c7bf6-4fwb6" Jul 02 21:37:09.459641 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.458949 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1adcf74b-7adb-4c3e-859d-2481e729caf1-trusted-ca\") pod \"image-registry-f788c7bf6-4fwb6\" (UID: \"1adcf74b-7adb-4c3e-859d-2481e729caf1\") " pod="openshift-image-registry/image-registry-f788c7bf6-4fwb6" Jul 02 21:37:09.462882 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.462861 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1adcf74b-7adb-4c3e-859d-2481e729caf1-image-registry-private-configuration\") pod \"image-registry-f788c7bf6-4fwb6\" (UID: \"1adcf74b-7adb-4c3e-859d-2481e729caf1\") " pod="openshift-image-registry/image-registry-f788c7bf6-4fwb6" Jul 02 21:37:09.462999 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.462945 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1adcf74b-7adb-4c3e-859d-2481e729caf1-installation-pull-secrets\") pod \"image-registry-f788c7bf6-4fwb6\" (UID: \"1adcf74b-7adb-4c3e-859d-2481e729caf1\") " pod="openshift-image-registry/image-registry-f788c7bf6-4fwb6" Jul 02 21:37:09.463269 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.463245 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1adcf74b-7adb-4c3e-859d-2481e729caf1-registry-tls\") pod \"image-registry-f788c7bf6-4fwb6\" (UID: \"1adcf74b-7adb-4c3e-859d-2481e729caf1\") " pod="openshift-image-registry/image-registry-f788c7bf6-4fwb6" Jul 02 21:37:09.473825 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.473799 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2kdv\" (UniqueName: \"kubernetes.io/projected/1adcf74b-7adb-4c3e-859d-2481e729caf1-kube-api-access-r2kdv\") pod \"image-registry-f788c7bf6-4fwb6\" (UID: \"1adcf74b-7adb-4c3e-859d-2481e729caf1\") " pod="openshift-image-registry/image-registry-f788c7bf6-4fwb6" Jul 02 21:37:09.473957 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.473858 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1adcf74b-7adb-4c3e-859d-2481e729caf1-bound-sa-token\") pod \"image-registry-f788c7bf6-4fwb6\" (UID: \"1adcf74b-7adb-4c3e-859d-2481e729caf1\") " pod="openshift-image-registry/image-registry-f788c7bf6-4fwb6" Jul 02 21:37:09.512981 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.512703 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5468bfc94c-dptkz" Jul 02 21:37:09.559298 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.559072 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-gv4ks\"" Jul 02 21:37:09.567693 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.567628 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-rzzw8" Jul 02 21:37:09.621205 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.620928 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-f788c7bf6-4fwb6" Jul 02 21:37:09.722973 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.722752 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5468bfc94c-dptkz"] Jul 02 21:37:09.728846 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:37:09.728813 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93bd8222_3798_4572_b120_8ada8386ff4f.slice/crio-9317da5d1f851925cb714222ecb284f3abca8f664ee06e2e45b5a2b78ace42ed WatchSource:0}: Error finding container 9317da5d1f851925cb714222ecb284f3abca8f664ee06e2e45b5a2b78ace42ed: Status 404 returned error can't find the container with id 9317da5d1f851925cb714222ecb284f3abca8f664ee06e2e45b5a2b78ace42ed Jul 02 21:37:09.780093 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.780007 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-rzzw8"] Jul 02 21:37:09.793091 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:37:09.793059 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf3c0081_d6c4_418c_8cfc_84b0248778f1.slice/crio-10c967dc95afb449e4d913d7b45d05023ca2820f3a3609ca968bb2525c2a019b WatchSource:0}: Error finding container 10c967dc95afb449e4d913d7b45d05023ca2820f3a3609ca968bb2525c2a019b: Status 404 returned error can't find the container with id 10c967dc95afb449e4d913d7b45d05023ca2820f3a3609ca968bb2525c2a019b Jul 02 21:37:09.840836 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:09.840748 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-f788c7bf6-4fwb6"] Jul 02 21:37:09.845262 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:37:09.845229 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1adcf74b_7adb_4c3e_859d_2481e729caf1.slice/crio-a917fab1bcfd497f0a70f3d23744c96fedcaad08fb5599ff2733925d42a6f3ef WatchSource:0}: Error finding container a917fab1bcfd497f0a70f3d23744c96fedcaad08fb5599ff2733925d42a6f3ef: Status 404 returned error can't find the container with id a917fab1bcfd497f0a70f3d23744c96fedcaad08fb5599ff2733925d42a6f3ef Jul 02 21:37:10.301488 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:10.300619 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-f788c7bf6-4fwb6" event={"ID":"1adcf74b-7adb-4c3e-859d-2481e729caf1","Type":"ContainerStarted","Data":"5f4e2aaf8da939ceead8d8362fcd0a3350452eba080a7ead4e1690eabe18aca0"} Jul 02 21:37:10.301488 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:10.300681 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-f788c7bf6-4fwb6" event={"ID":"1adcf74b-7adb-4c3e-859d-2481e729caf1","Type":"ContainerStarted","Data":"a917fab1bcfd497f0a70f3d23744c96fedcaad08fb5599ff2733925d42a6f3ef"} Jul 02 21:37:10.301488 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:10.301438 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-f788c7bf6-4fwb6" Jul 02 21:37:10.304026 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:10.303952 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-rzzw8" event={"ID":"af3c0081-d6c4-418c-8cfc-84b0248778f1","Type":"ContainerStarted","Data":"10c967dc95afb449e4d913d7b45d05023ca2820f3a3609ca968bb2525c2a019b"} Jul 02 21:37:10.306164 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:10.305928 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5468bfc94c-dptkz" event={"ID":"93bd8222-3798-4572-b120-8ada8386ff4f","Type":"ContainerStarted","Data":"9317da5d1f851925cb714222ecb284f3abca8f664ee06e2e45b5a2b78ace42ed"} Jul 02 21:37:10.349365 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:10.348181 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-f788c7bf6-4fwb6" podStartSLOduration=1.348159446 podStartE2EDuration="1.348159446s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:37:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:37:10.323024897 +0000 UTC m=+69.661995334" watchObservedRunningTime="2026-07-02 21:37:10.348159446 +0000 UTC m=+69.687129873" Jul 02 21:37:14.872293 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:14.872119 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-rzs97"] Jul 02 21:37:14.874858 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:14.874834 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-rzs97" Jul 02 21:37:14.877178 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:14.877100 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 02 21:37:14.877298 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:14.877175 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-s88rc\"" Jul 02 21:37:14.884592 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:14.884535 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-rzs97"] Jul 02 21:37:15.002846 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:15.002815 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/8742b41f-9660-4b06-be1b-957b839a327b-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-rzs97\" (UID: \"8742b41f-9660-4b06-be1b-957b839a327b\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-rzs97" Jul 02 21:37:15.103325 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:15.103223 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/8742b41f-9660-4b06-be1b-957b839a327b-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-rzs97\" (UID: \"8742b41f-9660-4b06-be1b-957b839a327b\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-rzs97" Jul 02 21:37:15.106183 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:15.106134 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/8742b41f-9660-4b06-be1b-957b839a327b-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-rzs97\" (UID: \"8742b41f-9660-4b06-be1b-957b839a327b\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-rzs97" Jul 02 21:37:15.187090 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:15.187064 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-rzs97" Jul 02 21:37:15.311007 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:15.310977 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-rzs97"] Jul 02 21:37:15.313865 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:37:15.313837 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8742b41f_9660_4b06_be1b_957b839a327b.slice/crio-32e6c26739d87ce2b9349c586785111c3ac2e67f2c1a7a9dc1bf64eff3fb75a9 WatchSource:0}: Error finding container 32e6c26739d87ce2b9349c586785111c3ac2e67f2c1a7a9dc1bf64eff3fb75a9: Status 404 returned error can't find the container with id 32e6c26739d87ce2b9349c586785111c3ac2e67f2c1a7a9dc1bf64eff3fb75a9 Jul 02 21:37:15.326067 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:15.326040 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-rzs97" event={"ID":"8742b41f-9660-4b06-be1b-957b839a327b","Type":"ContainerStarted","Data":"32e6c26739d87ce2b9349c586785111c3ac2e67f2c1a7a9dc1bf64eff3fb75a9"} Jul 02 21:37:15.327817 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:15.327771 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jgn9h" event={"ID":"42124588-1a15-4e43-a5cb-18c8142bc434","Type":"ContainerStarted","Data":"513bdb505a5061da60b41e7e994c048739e18c0fee2f321ce75a66625455f655"} Jul 02 21:37:15.327918 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:15.327876 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-jgn9h" Jul 02 21:37:15.329209 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:15.329176 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-j9flr" event={"ID":"d95b88bb-30e2-421a-84a7-96de7cd5d1e1","Type":"ContainerStarted","Data":"b7abf97d80deb512279cfe27b3866ef63c392ed67f5c114c4d2523f56b40b7cf"} Jul 02 21:37:15.330948 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:15.330928 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fsdjr" event={"ID":"f9e06bbd-441d-43a5-afdd-48eb709a5ecd","Type":"ContainerStarted","Data":"252c4c36a340d5811b1c3a29230faed55a614f69281434459f8b0dd07275f981"} Jul 02 21:37:15.332609 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:15.332590 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fqj8h" event={"ID":"7337a38e-9618-4a34-b207-44dc712dd12b","Type":"ContainerStarted","Data":"4e0d0031942f360323b7ef75aab48a66de2a8a123073b40a462305ea8dc39c26"} Jul 02 21:37:15.332692 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:15.332630 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fqj8h" event={"ID":"7337a38e-9618-4a34-b207-44dc712dd12b","Type":"ContainerStarted","Data":"6d7d6f8635ed198784e4a928781d33277ff497f4cee66b681682baa47105ed1c"} Jul 02 21:37:15.332692 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:15.332659 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-fqj8h" Jul 02 21:37:15.333879 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:15.333857 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-rzzw8" event={"ID":"af3c0081-d6c4-418c-8cfc-84b0248778f1","Type":"ContainerStarted","Data":"a241644123bdf60a537af429edfa0cca1bdb20d1f4850ffdbe0f6749e256c48f"} Jul 02 21:37:15.347292 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:15.347253 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-jgn9h" podStartSLOduration=66.80138692 podStartE2EDuration="1m14.347240448s" totalImagesPullingTime="7.545853528s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:36:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:37:15.346360567 +0000 UTC m=+74.685331004" watchObservedRunningTime="2026-07-02 21:37:15.347240448 +0000 UTC m=+74.686210901" Jul 02 21:37:15.350698 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:15.350674 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-ftvjj"] Jul 02 21:37:15.355730 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:15.355713 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-ftvjj" Jul 02 21:37:15.358089 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:15.358068 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 02 21:37:15.358089 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:15.358079 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 02 21:37:15.358267 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:15.358172 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-g6kpb\"" Jul 02 21:37:15.364115 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:15.364096 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-ftvjj"] Jul 02 21:37:15.366394 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:15.366350 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-j9flr" podStartSLOduration=66.77229751 podStartE2EDuration="1m14.366334674s" totalImagesPullingTime="7.594037164s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 21:36:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:37:15.365461136 +0000 UTC m=+74.704431574" watchObservedRunningTime="2026-07-02 21:37:15.366334674 +0000 UTC m=+74.705305111" Jul 02 21:37:15.387520 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:15.387481 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-fsdjr" podStartSLOduration=17.177426772 podStartE2EDuration="28.387469641s" totalImagesPullingTime="11.210042869s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 21:36:47 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:37:15.386846249 +0000 UTC m=+74.725816684" watchObservedRunningTime="2026-07-02 21:37:15.387469641 +0000 UTC m=+74.726440076" Jul 02 21:37:15.403209 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:15.403138 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-fqj8h" podStartSLOduration=33.383151524 podStartE2EDuration="39.403126157s" totalImagesPullingTime="6.019974633s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:36:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:37:15.402455875 +0000 UTC m=+74.741426309" watchObservedRunningTime="2026-07-02 21:37:15.403126157 +0000 UTC m=+74.742096592" Jul 02 21:37:15.406044 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:15.406018 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ld9nv\" (UniqueName: \"kubernetes.io/projected/cc6ebdd7-4265-45cc-86d2-b36a76ffe2c0-kube-api-access-ld9nv\") pod \"downloads-685bfb5f64-ftvjj\" (UID: \"cc6ebdd7-4265-45cc-86d2-b36a76ffe2c0\") " pod="openshift-console/downloads-685bfb5f64-ftvjj" Jul 02 21:37:15.416221 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:15.416189 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-rzzw8" podStartSLOduration=33.260744579 podStartE2EDuration="38.416177598s" totalImagesPullingTime="5.155433019s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:36:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:37:15.415574542 +0000 UTC m=+74.754544974" watchObservedRunningTime="2026-07-02 21:37:15.416177598 +0000 UTC m=+74.755148033" Jul 02 21:37:15.506796 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:15.506767 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ld9nv\" (UniqueName: \"kubernetes.io/projected/cc6ebdd7-4265-45cc-86d2-b36a76ffe2c0-kube-api-access-ld9nv\") pod \"downloads-685bfb5f64-ftvjj\" (UID: \"cc6ebdd7-4265-45cc-86d2-b36a76ffe2c0\") " pod="openshift-console/downloads-685bfb5f64-ftvjj" Jul 02 21:37:15.514561 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:15.514538 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ld9nv\" (UniqueName: \"kubernetes.io/projected/cc6ebdd7-4265-45cc-86d2-b36a76ffe2c0-kube-api-access-ld9nv\") pod \"downloads-685bfb5f64-ftvjj\" (UID: \"cc6ebdd7-4265-45cc-86d2-b36a76ffe2c0\") " pod="openshift-console/downloads-685bfb5f64-ftvjj" Jul 02 21:37:15.664652 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:15.664580 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-ftvjj" Jul 02 21:37:15.799780 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:15.799742 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-ftvjj"] Jul 02 21:37:15.802561 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:37:15.802531 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc6ebdd7_4265_45cc_86d2_b36a76ffe2c0.slice/crio-32730f3b9ce4302a6398f5336bb4f6a44cc80a3e52e860672094f5f933487857 WatchSource:0}: Error finding container 32730f3b9ce4302a6398f5336bb4f6a44cc80a3e52e860672094f5f933487857: Status 404 returned error can't find the container with id 32730f3b9ce4302a6398f5336bb4f6a44cc80a3e52e860672094f5f933487857 Jul 02 21:37:16.339582 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:16.339547 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-ftvjj" event={"ID":"cc6ebdd7-4265-45cc-86d2-b36a76ffe2c0","Type":"ContainerStarted","Data":"32730f3b9ce4302a6398f5336bb4f6a44cc80a3e52e860672094f5f933487857"} Jul 02 21:37:19.351642 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:19.351603 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5468bfc94c-dptkz" event={"ID":"93bd8222-3798-4572-b120-8ada8386ff4f","Type":"ContainerStarted","Data":"d4387592ce46c48d76ebf46ef6d3f1e8ef20ec16cbf2f741d4d4b723da4dae6d"} Jul 02 21:37:19.352114 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:19.351841 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5468bfc94c-dptkz" Jul 02 21:37:19.353497 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:19.353440 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-rzs97" event={"ID":"8742b41f-9660-4b06-be1b-957b839a327b","Type":"ContainerStarted","Data":"493dc47e319248a7e946ed7c5ff8e3e93fef810dece04ab0230383862c7fd0f1"} Jul 02 21:37:19.353667 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:19.353640 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-rzs97" Jul 02 21:37:19.354007 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:19.353988 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5468bfc94c-dptkz" Jul 02 21:37:19.359718 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:19.359697 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-rzs97" Jul 02 21:37:19.367498 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:19.367451 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5468bfc94c-dptkz" podStartSLOduration=1.7402170099999998 podStartE2EDuration="10.36743485s" totalImagesPullingTime="8.62721784s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:37:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:37:19.366632905 +0000 UTC m=+78.705603340" watchObservedRunningTime="2026-07-02 21:37:19.36743485 +0000 UTC m=+78.706405287" Jul 02 21:37:19.393931 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:19.393764 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-rzs97" podStartSLOduration=2.370798495 podStartE2EDuration="5.393745871s" totalImagesPullingTime="3.022947376s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:37:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:37:19.393394087 +0000 UTC m=+78.732364523" watchObservedRunningTime="2026-07-02 21:37:19.393745871 +0000 UTC m=+78.732716306" Jul 02 21:37:19.936688 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:19.936656 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-l9rhw"] Jul 02 21:37:19.940157 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:19.940134 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-l9rhw" Jul 02 21:37:19.943604 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:19.943580 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 02 21:37:19.943604 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:19.943591 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 02 21:37:19.943819 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:19.943607 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 02 21:37:19.943819 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:19.943642 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 02 21:37:19.943819 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:19.943660 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-2cbm2\"" Jul 02 21:37:19.943959 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:19.943945 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 02 21:37:19.948737 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:19.948717 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-l9rhw"] Jul 02 21:37:20.045467 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:20.045416 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/efe39bd5-6e24-463f-af81-2f9beab85fe6-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-l9rhw\" (UID: \"efe39bd5-6e24-463f-af81-2f9beab85fe6\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-l9rhw" Jul 02 21:37:20.045667 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:20.045527 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/efe39bd5-6e24-463f-af81-2f9beab85fe6-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-l9rhw\" (UID: \"efe39bd5-6e24-463f-af81-2f9beab85fe6\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-l9rhw" Jul 02 21:37:20.045667 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:20.045555 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/efe39bd5-6e24-463f-af81-2f9beab85fe6-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-l9rhw\" (UID: \"efe39bd5-6e24-463f-af81-2f9beab85fe6\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-l9rhw" Jul 02 21:37:20.045667 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:20.045643 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7t9mj\" (UniqueName: \"kubernetes.io/projected/efe39bd5-6e24-463f-af81-2f9beab85fe6-kube-api-access-7t9mj\") pod \"prometheus-operator-567f7b8b7c-l9rhw\" (UID: \"efe39bd5-6e24-463f-af81-2f9beab85fe6\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-l9rhw" Jul 02 21:37:20.149022 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:20.146998 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/efe39bd5-6e24-463f-af81-2f9beab85fe6-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-l9rhw\" (UID: \"efe39bd5-6e24-463f-af81-2f9beab85fe6\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-l9rhw" Jul 02 21:37:20.149022 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:20.147054 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/efe39bd5-6e24-463f-af81-2f9beab85fe6-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-l9rhw\" (UID: \"efe39bd5-6e24-463f-af81-2f9beab85fe6\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-l9rhw" Jul 02 21:37:20.149022 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:20.147105 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7t9mj\" (UniqueName: \"kubernetes.io/projected/efe39bd5-6e24-463f-af81-2f9beab85fe6-kube-api-access-7t9mj\") pod \"prometheus-operator-567f7b8b7c-l9rhw\" (UID: \"efe39bd5-6e24-463f-af81-2f9beab85fe6\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-l9rhw" Jul 02 21:37:20.149022 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:20.147171 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/efe39bd5-6e24-463f-af81-2f9beab85fe6-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-l9rhw\" (UID: \"efe39bd5-6e24-463f-af81-2f9beab85fe6\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-l9rhw" Jul 02 21:37:20.149022 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:37:20.147929 2569 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 02 21:37:20.149022 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:20.147969 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/efe39bd5-6e24-463f-af81-2f9beab85fe6-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-l9rhw\" (UID: \"efe39bd5-6e24-463f-af81-2f9beab85fe6\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-l9rhw" Jul 02 21:37:20.149022 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:37:20.148021 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efe39bd5-6e24-463f-af81-2f9beab85fe6-prometheus-operator-tls podName:efe39bd5-6e24-463f-af81-2f9beab85fe6 nodeName:}" failed. No retries permitted until 2026-07-02 21:37:20.647999835 +0000 UTC m=+79.986970251 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/efe39bd5-6e24-463f-af81-2f9beab85fe6-prometheus-operator-tls") pod "prometheus-operator-567f7b8b7c-l9rhw" (UID: "efe39bd5-6e24-463f-af81-2f9beab85fe6") : secret "prometheus-operator-tls" not found Jul 02 21:37:20.152760 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:20.152737 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/efe39bd5-6e24-463f-af81-2f9beab85fe6-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-l9rhw\" (UID: \"efe39bd5-6e24-463f-af81-2f9beab85fe6\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-l9rhw" Jul 02 21:37:20.155487 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:20.155465 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7t9mj\" (UniqueName: \"kubernetes.io/projected/efe39bd5-6e24-463f-af81-2f9beab85fe6-kube-api-access-7t9mj\") pod \"prometheus-operator-567f7b8b7c-l9rhw\" (UID: \"efe39bd5-6e24-463f-af81-2f9beab85fe6\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-l9rhw" Jul 02 21:37:20.650110 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:20.650064 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/efe39bd5-6e24-463f-af81-2f9beab85fe6-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-l9rhw\" (UID: \"efe39bd5-6e24-463f-af81-2f9beab85fe6\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-l9rhw" Jul 02 21:37:20.652840 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:20.652805 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/efe39bd5-6e24-463f-af81-2f9beab85fe6-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-l9rhw\" (UID: \"efe39bd5-6e24-463f-af81-2f9beab85fe6\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-l9rhw" Jul 02 21:37:20.850953 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:20.850918 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-l9rhw" Jul 02 21:37:20.986489 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:20.986448 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-l9rhw"] Jul 02 21:37:20.990136 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:37:20.990109 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefe39bd5_6e24_463f_af81_2f9beab85fe6.slice/crio-bf17f07d9e6121645752820ddf6f750466c4812b4910a4994e9e7d9f466aabcf WatchSource:0}: Error finding container bf17f07d9e6121645752820ddf6f750466c4812b4910a4994e9e7d9f466aabcf: Status 404 returned error can't find the container with id bf17f07d9e6121645752820ddf6f750466c4812b4910a4994e9e7d9f466aabcf Jul 02 21:37:21.362509 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:21.362464 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-l9rhw" event={"ID":"efe39bd5-6e24-463f-af81-2f9beab85fe6","Type":"ContainerStarted","Data":"bf17f07d9e6121645752820ddf6f750466c4812b4910a4994e9e7d9f466aabcf"} Jul 02 21:37:22.369647 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:22.369617 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-l9rhw" event={"ID":"efe39bd5-6e24-463f-af81-2f9beab85fe6","Type":"ContainerStarted","Data":"527b997a7177ff906ee38b2b15aa8c2c782f897722d42fec252f3c340b0cf9fd"} Jul 02 21:37:23.374890 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:23.374852 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-l9rhw" event={"ID":"efe39bd5-6e24-463f-af81-2f9beab85fe6","Type":"ContainerStarted","Data":"0670f5b6e10730a73a7300bd4649d89677d0931fb040687f489edeafa64fc0ea"} Jul 02 21:37:23.393247 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:23.393204 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-l9rhw" podStartSLOduration=3.116871503 podStartE2EDuration="4.393190118s" totalImagesPullingTime="1.276318615s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:37:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:37:23.391943535 +0000 UTC m=+82.730913972" watchObservedRunningTime="2026-07-02 21:37:23.393190118 +0000 UTC m=+82.732160553" Jul 02 21:37:25.268821 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:25.267435 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-g6qqq"] Jul 02 21:37:25.272444 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:25.272392 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-g6qqq" Jul 02 21:37:25.275576 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:25.275502 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 02 21:37:25.275950 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:25.275746 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 02 21:37:25.275950 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:25.275913 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-mx7c5\"" Jul 02 21:37:25.285569 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:25.285530 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-g6qqq"] Jul 02 21:37:25.323999 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:25.323971 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-c8jks"] Jul 02 21:37:25.328411 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:25.327385 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-c8jks" Jul 02 21:37:25.331144 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:25.329701 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 02 21:37:25.331144 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:25.329986 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-p4xt9\"" Jul 02 21:37:25.331144 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:25.330197 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 02 21:37:25.331144 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:25.330347 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 02 21:37:25.342422 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:25.341860 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-fqj8h" Jul 02 21:37:25.387811 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:25.387766 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7497r\" (UniqueName: \"kubernetes.io/projected/f9f5712b-72f4-4536-b1fe-2a4240d463f6-kube-api-access-7497r\") pod \"node-exporter-c8jks\" (UID: \"f9f5712b-72f4-4536-b1fe-2a4240d463f6\") " pod="openshift-monitoring/node-exporter-c8jks" Jul 02 21:37:25.387964 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:25.387816 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f9f5712b-72f4-4536-b1fe-2a4240d463f6-node-exporter-textfile\") pod \"node-exporter-c8jks\" (UID: \"f9f5712b-72f4-4536-b1fe-2a4240d463f6\") " pod="openshift-monitoring/node-exporter-c8jks" Jul 02 21:37:25.387964 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:25.387849 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f9f5712b-72f4-4536-b1fe-2a4240d463f6-node-exporter-accelerators-collector-config\") pod \"node-exporter-c8jks\" (UID: \"f9f5712b-72f4-4536-b1fe-2a4240d463f6\") " pod="openshift-monitoring/node-exporter-c8jks" Jul 02 21:37:25.387964 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:25.387884 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f9f5712b-72f4-4536-b1fe-2a4240d463f6-metrics-client-ca\") pod \"node-exporter-c8jks\" (UID: \"f9f5712b-72f4-4536-b1fe-2a4240d463f6\") " pod="openshift-monitoring/node-exporter-c8jks" Jul 02 21:37:25.387964 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:25.387921 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/739024f7-d272-4932-b23d-5d815ade3282-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-g6qqq\" (UID: \"739024f7-d272-4932-b23d-5d815ade3282\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-g6qqq" Jul 02 21:37:25.387964 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:25.387950 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f9f5712b-72f4-4536-b1fe-2a4240d463f6-root\") pod \"node-exporter-c8jks\" (UID: \"f9f5712b-72f4-4536-b1fe-2a4240d463f6\") " pod="openshift-monitoring/node-exporter-c8jks" Jul 02 21:37:25.388204 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:25.387974 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f9f5712b-72f4-4536-b1fe-2a4240d463f6-sys\") pod \"node-exporter-c8jks\" (UID: \"f9f5712b-72f4-4536-b1fe-2a4240d463f6\") " pod="openshift-monitoring/node-exporter-c8jks" Jul 02 21:37:25.388204 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:25.388003 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f9f5712b-72f4-4536-b1fe-2a4240d463f6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-c8jks\" (UID: \"f9f5712b-72f4-4536-b1fe-2a4240d463f6\") " pod="openshift-monitoring/node-exporter-c8jks" Jul 02 21:37:25.388204 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:25.388041 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/739024f7-d272-4932-b23d-5d815ade3282-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-g6qqq\" (UID: \"739024f7-d272-4932-b23d-5d815ade3282\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-g6qqq" Jul 02 21:37:25.388204 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:25.388066 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngl52\" (UniqueName: \"kubernetes.io/projected/739024f7-d272-4932-b23d-5d815ade3282-kube-api-access-ngl52\") pod \"openshift-state-metrics-54cf96dfcf-g6qqq\" (UID: \"739024f7-d272-4932-b23d-5d815ade3282\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-g6qqq" Jul 02 21:37:25.388204 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:25.388092 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/739024f7-d272-4932-b23d-5d815ade3282-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-g6qqq\" (UID: \"739024f7-d272-4932-b23d-5d815ade3282\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-g6qqq" Jul 02 21:37:25.388204 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:25.388122 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f9f5712b-72f4-4536-b1fe-2a4240d463f6-node-exporter-tls\") pod \"node-exporter-c8jks\" (UID: \"f9f5712b-72f4-4536-b1fe-2a4240d463f6\") " pod="openshift-monitoring/node-exporter-c8jks" Jul 02 21:37:25.388204 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:25.388151 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f9f5712b-72f4-4536-b1fe-2a4240d463f6-node-exporter-wtmp\") pod \"node-exporter-c8jks\" (UID: \"f9f5712b-72f4-4536-b1fe-2a4240d463f6\") " pod="openshift-monitoring/node-exporter-c8jks" Jul 02 21:37:25.489803 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:25.488893 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7497r\" (UniqueName: \"kubernetes.io/projected/f9f5712b-72f4-4536-b1fe-2a4240d463f6-kube-api-access-7497r\") pod \"node-exporter-c8jks\" (UID: \"f9f5712b-72f4-4536-b1fe-2a4240d463f6\") " pod="openshift-monitoring/node-exporter-c8jks" Jul 02 21:37:25.489803 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:25.488937 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f9f5712b-72f4-4536-b1fe-2a4240d463f6-node-exporter-textfile\") pod \"node-exporter-c8jks\" (UID: \"f9f5712b-72f4-4536-b1fe-2a4240d463f6\") " pod="openshift-monitoring/node-exporter-c8jks" Jul 02 21:37:25.489803 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:25.488998 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f9f5712b-72f4-4536-b1fe-2a4240d463f6-node-exporter-accelerators-collector-config\") pod \"node-exporter-c8jks\" (UID: \"f9f5712b-72f4-4536-b1fe-2a4240d463f6\") " pod="openshift-monitoring/node-exporter-c8jks" Jul 02 21:37:25.489803 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:25.489025 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f9f5712b-72f4-4536-b1fe-2a4240d463f6-metrics-client-ca\") pod \"node-exporter-c8jks\" (UID: \"f9f5712b-72f4-4536-b1fe-2a4240d463f6\") " pod="openshift-monitoring/node-exporter-c8jks" Jul 02 21:37:25.489803 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:25.489076 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/739024f7-d272-4932-b23d-5d815ade3282-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-g6qqq\" (UID: \"739024f7-d272-4932-b23d-5d815ade3282\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-g6qqq" Jul 02 21:37:25.489803 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:25.489106 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f9f5712b-72f4-4536-b1fe-2a4240d463f6-root\") pod \"node-exporter-c8jks\" (UID: \"f9f5712b-72f4-4536-b1fe-2a4240d463f6\") " pod="openshift-monitoring/node-exporter-c8jks" Jul 02 21:37:25.489803 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:25.489130 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f9f5712b-72f4-4536-b1fe-2a4240d463f6-sys\") pod \"node-exporter-c8jks\" (UID: \"f9f5712b-72f4-4536-b1fe-2a4240d463f6\") " pod="openshift-monitoring/node-exporter-c8jks" Jul 02 21:37:25.489803 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:25.489156 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f9f5712b-72f4-4536-b1fe-2a4240d463f6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-c8jks\" (UID: \"f9f5712b-72f4-4536-b1fe-2a4240d463f6\") " pod="openshift-monitoring/node-exporter-c8jks" Jul 02 21:37:25.489803 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:25.489195 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/739024f7-d272-4932-b23d-5d815ade3282-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-g6qqq\" (UID: \"739024f7-d272-4932-b23d-5d815ade3282\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-g6qqq" Jul 02 21:37:25.489803 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:25.489234 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngl52\" (UniqueName: \"kubernetes.io/projected/739024f7-d272-4932-b23d-5d815ade3282-kube-api-access-ngl52\") pod \"openshift-state-metrics-54cf96dfcf-g6qqq\" (UID: \"739024f7-d272-4932-b23d-5d815ade3282\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-g6qqq" Jul 02 21:37:25.489803 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:25.489265 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/739024f7-d272-4932-b23d-5d815ade3282-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-g6qqq\" (UID: \"739024f7-d272-4932-b23d-5d815ade3282\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-g6qqq" Jul 02 21:37:25.489803 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:25.489299 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f9f5712b-72f4-4536-b1fe-2a4240d463f6-node-exporter-tls\") pod \"node-exporter-c8jks\" (UID: \"f9f5712b-72f4-4536-b1fe-2a4240d463f6\") " pod="openshift-monitoring/node-exporter-c8jks" Jul 02 21:37:25.489803 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:25.489326 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f9f5712b-72f4-4536-b1fe-2a4240d463f6-node-exporter-wtmp\") pod \"node-exporter-c8jks\" (UID: \"f9f5712b-72f4-4536-b1fe-2a4240d463f6\") " pod="openshift-monitoring/node-exporter-c8jks" Jul 02 21:37:25.489803 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:25.489490 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f9f5712b-72f4-4536-b1fe-2a4240d463f6-node-exporter-wtmp\") pod \"node-exporter-c8jks\" (UID: \"f9f5712b-72f4-4536-b1fe-2a4240d463f6\") " pod="openshift-monitoring/node-exporter-c8jks" Jul 02 21:37:25.489803 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:25.489772 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f9f5712b-72f4-4536-b1fe-2a4240d463f6-metrics-client-ca\") pod \"node-exporter-c8jks\" (UID: \"f9f5712b-72f4-4536-b1fe-2a4240d463f6\") " pod="openshift-monitoring/node-exporter-c8jks" Jul 02 21:37:25.490599 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:25.490047 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f9f5712b-72f4-4536-b1fe-2a4240d463f6-node-exporter-textfile\") pod \"node-exporter-c8jks\" (UID: \"f9f5712b-72f4-4536-b1fe-2a4240d463f6\") " pod="openshift-monitoring/node-exporter-c8jks" Jul 02 21:37:25.490599 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:25.490402 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/739024f7-d272-4932-b23d-5d815ade3282-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-g6qqq\" (UID: \"739024f7-d272-4932-b23d-5d815ade3282\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-g6qqq" Jul 02 21:37:25.490599 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:25.490467 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f9f5712b-72f4-4536-b1fe-2a4240d463f6-root\") pod \"node-exporter-c8jks\" (UID: \"f9f5712b-72f4-4536-b1fe-2a4240d463f6\") " pod="openshift-monitoring/node-exporter-c8jks" Jul 02 21:37:25.490599 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:25.490512 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f9f5712b-72f4-4536-b1fe-2a4240d463f6-sys\") pod \"node-exporter-c8jks\" (UID: \"f9f5712b-72f4-4536-b1fe-2a4240d463f6\") " pod="openshift-monitoring/node-exporter-c8jks" Jul 02 21:37:25.491817 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:25.490977 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f9f5712b-72f4-4536-b1fe-2a4240d463f6-node-exporter-accelerators-collector-config\") pod \"node-exporter-c8jks\" (UID: \"f9f5712b-72f4-4536-b1fe-2a4240d463f6\") " pod="openshift-monitoring/node-exporter-c8jks" Jul 02 21:37:25.491817 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:37:25.491607 2569 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 02 21:37:25.491817 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:37:25.491664 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/739024f7-d272-4932-b23d-5d815ade3282-openshift-state-metrics-tls podName:739024f7-d272-4932-b23d-5d815ade3282 nodeName:}" failed. No retries permitted until 2026-07-02 21:37:25.991646401 +0000 UTC m=+85.330616819 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/739024f7-d272-4932-b23d-5d815ade3282-openshift-state-metrics-tls") pod "openshift-state-metrics-54cf96dfcf-g6qqq" (UID: "739024f7-d272-4932-b23d-5d815ade3282") : secret "openshift-state-metrics-tls" not found Jul 02 21:37:25.491817 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:37:25.491665 2569 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 02 21:37:25.491817 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:37:25.491713 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9f5712b-72f4-4536-b1fe-2a4240d463f6-node-exporter-tls podName:f9f5712b-72f4-4536-b1fe-2a4240d463f6 nodeName:}" failed. No retries permitted until 2026-07-02 21:37:25.991702329 +0000 UTC m=+85.330672748 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/f9f5712b-72f4-4536-b1fe-2a4240d463f6-node-exporter-tls") pod "node-exporter-c8jks" (UID: "f9f5712b-72f4-4536-b1fe-2a4240d463f6") : secret "node-exporter-tls" not found Jul 02 21:37:25.495153 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:25.494115 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f9f5712b-72f4-4536-b1fe-2a4240d463f6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-c8jks\" (UID: \"f9f5712b-72f4-4536-b1fe-2a4240d463f6\") " pod="openshift-monitoring/node-exporter-c8jks" Jul 02 21:37:25.495153 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:25.495001 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/739024f7-d272-4932-b23d-5d815ade3282-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-g6qqq\" (UID: \"739024f7-d272-4932-b23d-5d815ade3282\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-g6qqq" Jul 02 21:37:25.504475 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:25.504436 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngl52\" (UniqueName: \"kubernetes.io/projected/739024f7-d272-4932-b23d-5d815ade3282-kube-api-access-ngl52\") pod \"openshift-state-metrics-54cf96dfcf-g6qqq\" (UID: \"739024f7-d272-4932-b23d-5d815ade3282\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-g6qqq" Jul 02 21:37:25.513357 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:25.512541 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7497r\" (UniqueName: \"kubernetes.io/projected/f9f5712b-72f4-4536-b1fe-2a4240d463f6-kube-api-access-7497r\") pod \"node-exporter-c8jks\" (UID: \"f9f5712b-72f4-4536-b1fe-2a4240d463f6\") " pod="openshift-monitoring/node-exporter-c8jks" Jul 02 21:37:25.993930 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:25.993706 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/739024f7-d272-4932-b23d-5d815ade3282-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-g6qqq\" (UID: \"739024f7-d272-4932-b23d-5d815ade3282\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-g6qqq" Jul 02 21:37:25.993930 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:25.993761 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f9f5712b-72f4-4536-b1fe-2a4240d463f6-node-exporter-tls\") pod \"node-exporter-c8jks\" (UID: \"f9f5712b-72f4-4536-b1fe-2a4240d463f6\") " pod="openshift-monitoring/node-exporter-c8jks" Jul 02 21:37:25.994176 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:37:25.993955 2569 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 02 21:37:25.994176 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:37:25.994044 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/739024f7-d272-4932-b23d-5d815ade3282-openshift-state-metrics-tls podName:739024f7-d272-4932-b23d-5d815ade3282 nodeName:}" failed. No retries permitted until 2026-07-02 21:37:26.994019033 +0000 UTC m=+86.332989452 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/739024f7-d272-4932-b23d-5d815ade3282-openshift-state-metrics-tls") pod "openshift-state-metrics-54cf96dfcf-g6qqq" (UID: "739024f7-d272-4932-b23d-5d815ade3282") : secret "openshift-state-metrics-tls" not found Jul 02 21:37:25.996766 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:25.996715 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f9f5712b-72f4-4536-b1fe-2a4240d463f6-node-exporter-tls\") pod \"node-exporter-c8jks\" (UID: \"f9f5712b-72f4-4536-b1fe-2a4240d463f6\") " pod="openshift-monitoring/node-exporter-c8jks" Jul 02 21:37:26.239967 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:26.239657 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-c8jks" Jul 02 21:37:26.255628 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:37:26.255596 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9f5712b_72f4_4536_b1fe_2a4240d463f6.slice/crio-270e5f9d49e3c11eeab5d07fa1080e81ab271c89f9572a0c2c426938762a84a6 WatchSource:0}: Error finding container 270e5f9d49e3c11eeab5d07fa1080e81ab271c89f9572a0c2c426938762a84a6: Status 404 returned error can't find the container with id 270e5f9d49e3c11eeab5d07fa1080e81ab271c89f9572a0c2c426938762a84a6 Jul 02 21:37:26.394975 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:26.394941 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c8jks" event={"ID":"f9f5712b-72f4-4536-b1fe-2a4240d463f6","Type":"ContainerStarted","Data":"270e5f9d49e3c11eeab5d07fa1080e81ab271c89f9572a0c2c426938762a84a6"} Jul 02 21:37:27.002616 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:27.002583 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/739024f7-d272-4932-b23d-5d815ade3282-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-g6qqq\" (UID: \"739024f7-d272-4932-b23d-5d815ade3282\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-g6qqq" Jul 02 21:37:27.005455 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:27.005427 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/739024f7-d272-4932-b23d-5d815ade3282-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-g6qqq\" (UID: \"739024f7-d272-4932-b23d-5d815ade3282\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-g6qqq" Jul 02 21:37:27.086403 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:27.086364 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-g6qqq" Jul 02 21:37:27.261888 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:27.261849 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6fd7d9ffcd-njp9f"] Jul 02 21:37:27.266095 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:27.266072 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6fd7d9ffcd-njp9f" Jul 02 21:37:27.268663 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:27.268641 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 02 21:37:27.269460 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:27.269003 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-3l7lcjmnp90s5\"" Jul 02 21:37:27.269460 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:27.269050 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 02 21:37:27.269460 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:27.269010 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-ss2vm\"" Jul 02 21:37:27.269460 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:27.269307 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 02 21:37:27.270339 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:27.270251 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 02 21:37:27.271208 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:27.270598 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 02 21:37:27.279266 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:27.279246 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6fd7d9ffcd-njp9f"] Jul 02 21:37:27.357983 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:27.357899 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-g6qqq"] Jul 02 21:37:27.361749 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:37:27.361720 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod739024f7_d272_4932_b23d_5d815ade3282.slice/crio-810714e7f8867ba4b9e4282a9f1ce73d5e86f9529efff0b7bc4a326386f88c00 WatchSource:0}: Error finding container 810714e7f8867ba4b9e4282a9f1ce73d5e86f9529efff0b7bc4a326386f88c00: Status 404 returned error can't find the container with id 810714e7f8867ba4b9e4282a9f1ce73d5e86f9529efff0b7bc4a326386f88c00 Jul 02 21:37:27.400830 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:27.400776 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-g6qqq" event={"ID":"739024f7-d272-4932-b23d-5d815ade3282","Type":"ContainerStarted","Data":"810714e7f8867ba4b9e4282a9f1ce73d5e86f9529efff0b7bc4a326386f88c00"} Jul 02 21:37:27.402838 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:27.402747 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c8jks" event={"ID":"f9f5712b-72f4-4536-b1fe-2a4240d463f6","Type":"ContainerStarted","Data":"0782c5805fdf306dd9a7bdf53ee49adfdf515cadff9f24236def8643e9d0d0d2"} Jul 02 21:37:27.405489 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:27.405401 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftngp\" (UniqueName: \"kubernetes.io/projected/57579b3f-0f25-49af-b810-64b3b0e3d3d1-kube-api-access-ftngp\") pod \"thanos-querier-6fd7d9ffcd-njp9f\" (UID: \"57579b3f-0f25-49af-b810-64b3b0e3d3d1\") " pod="openshift-monitoring/thanos-querier-6fd7d9ffcd-njp9f" Jul 02 21:37:27.405489 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:27.405443 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/57579b3f-0f25-49af-b810-64b3b0e3d3d1-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6fd7d9ffcd-njp9f\" (UID: \"57579b3f-0f25-49af-b810-64b3b0e3d3d1\") " pod="openshift-monitoring/thanos-querier-6fd7d9ffcd-njp9f" Jul 02 21:37:27.405489 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:27.405467 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/57579b3f-0f25-49af-b810-64b3b0e3d3d1-metrics-client-ca\") pod \"thanos-querier-6fd7d9ffcd-njp9f\" (UID: \"57579b3f-0f25-49af-b810-64b3b0e3d3d1\") " pod="openshift-monitoring/thanos-querier-6fd7d9ffcd-njp9f" Jul 02 21:37:27.405641 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:27.405496 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/57579b3f-0f25-49af-b810-64b3b0e3d3d1-secret-thanos-querier-tls\") pod \"thanos-querier-6fd7d9ffcd-njp9f\" (UID: \"57579b3f-0f25-49af-b810-64b3b0e3d3d1\") " pod="openshift-monitoring/thanos-querier-6fd7d9ffcd-njp9f" Jul 02 21:37:27.405641 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:27.405521 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/57579b3f-0f25-49af-b810-64b3b0e3d3d1-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6fd7d9ffcd-njp9f\" (UID: \"57579b3f-0f25-49af-b810-64b3b0e3d3d1\") " pod="openshift-monitoring/thanos-querier-6fd7d9ffcd-njp9f" Jul 02 21:37:27.405641 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:27.405575 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/57579b3f-0f25-49af-b810-64b3b0e3d3d1-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6fd7d9ffcd-njp9f\" (UID: \"57579b3f-0f25-49af-b810-64b3b0e3d3d1\") " pod="openshift-monitoring/thanos-querier-6fd7d9ffcd-njp9f" Jul 02 21:37:27.405641 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:27.405619 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/57579b3f-0f25-49af-b810-64b3b0e3d3d1-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6fd7d9ffcd-njp9f\" (UID: \"57579b3f-0f25-49af-b810-64b3b0e3d3d1\") " pod="openshift-monitoring/thanos-querier-6fd7d9ffcd-njp9f" Jul 02 21:37:27.405751 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:27.405648 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/57579b3f-0f25-49af-b810-64b3b0e3d3d1-secret-grpc-tls\") pod \"thanos-querier-6fd7d9ffcd-njp9f\" (UID: \"57579b3f-0f25-49af-b810-64b3b0e3d3d1\") " pod="openshift-monitoring/thanos-querier-6fd7d9ffcd-njp9f" Jul 02 21:37:27.506969 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:27.506934 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/57579b3f-0f25-49af-b810-64b3b0e3d3d1-secret-grpc-tls\") pod \"thanos-querier-6fd7d9ffcd-njp9f\" (UID: \"57579b3f-0f25-49af-b810-64b3b0e3d3d1\") " pod="openshift-monitoring/thanos-querier-6fd7d9ffcd-njp9f" Jul 02 21:37:27.507079 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:27.507013 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftngp\" (UniqueName: \"kubernetes.io/projected/57579b3f-0f25-49af-b810-64b3b0e3d3d1-kube-api-access-ftngp\") pod \"thanos-querier-6fd7d9ffcd-njp9f\" (UID: \"57579b3f-0f25-49af-b810-64b3b0e3d3d1\") " pod="openshift-monitoring/thanos-querier-6fd7d9ffcd-njp9f" Jul 02 21:37:27.507124 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:27.507082 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/57579b3f-0f25-49af-b810-64b3b0e3d3d1-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6fd7d9ffcd-njp9f\" (UID: \"57579b3f-0f25-49af-b810-64b3b0e3d3d1\") " pod="openshift-monitoring/thanos-querier-6fd7d9ffcd-njp9f" Jul 02 21:37:27.507159 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:27.507121 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/57579b3f-0f25-49af-b810-64b3b0e3d3d1-metrics-client-ca\") pod \"thanos-querier-6fd7d9ffcd-njp9f\" (UID: \"57579b3f-0f25-49af-b810-64b3b0e3d3d1\") " pod="openshift-monitoring/thanos-querier-6fd7d9ffcd-njp9f" Jul 02 21:37:27.507190 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:27.507178 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/57579b3f-0f25-49af-b810-64b3b0e3d3d1-secret-thanos-querier-tls\") pod \"thanos-querier-6fd7d9ffcd-njp9f\" (UID: \"57579b3f-0f25-49af-b810-64b3b0e3d3d1\") " pod="openshift-monitoring/thanos-querier-6fd7d9ffcd-njp9f" Jul 02 21:37:27.507221 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:27.507203 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/57579b3f-0f25-49af-b810-64b3b0e3d3d1-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6fd7d9ffcd-njp9f\" (UID: \"57579b3f-0f25-49af-b810-64b3b0e3d3d1\") " pod="openshift-monitoring/thanos-querier-6fd7d9ffcd-njp9f" Jul 02 21:37:27.507251 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:27.507230 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/57579b3f-0f25-49af-b810-64b3b0e3d3d1-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6fd7d9ffcd-njp9f\" (UID: \"57579b3f-0f25-49af-b810-64b3b0e3d3d1\") " pod="openshift-monitoring/thanos-querier-6fd7d9ffcd-njp9f" Jul 02 21:37:27.507589 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:27.507298 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/57579b3f-0f25-49af-b810-64b3b0e3d3d1-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6fd7d9ffcd-njp9f\" (UID: \"57579b3f-0f25-49af-b810-64b3b0e3d3d1\") " pod="openshift-monitoring/thanos-querier-6fd7d9ffcd-njp9f" Jul 02 21:37:27.508192 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:27.508166 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/57579b3f-0f25-49af-b810-64b3b0e3d3d1-metrics-client-ca\") pod \"thanos-querier-6fd7d9ffcd-njp9f\" (UID: \"57579b3f-0f25-49af-b810-64b3b0e3d3d1\") " pod="openshift-monitoring/thanos-querier-6fd7d9ffcd-njp9f" Jul 02 21:37:27.510270 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:27.510223 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/57579b3f-0f25-49af-b810-64b3b0e3d3d1-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6fd7d9ffcd-njp9f\" (UID: \"57579b3f-0f25-49af-b810-64b3b0e3d3d1\") " pod="openshift-monitoring/thanos-querier-6fd7d9ffcd-njp9f" Jul 02 21:37:27.510356 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:27.510307 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/57579b3f-0f25-49af-b810-64b3b0e3d3d1-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6fd7d9ffcd-njp9f\" (UID: \"57579b3f-0f25-49af-b810-64b3b0e3d3d1\") " pod="openshift-monitoring/thanos-querier-6fd7d9ffcd-njp9f" Jul 02 21:37:27.510925 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:27.510862 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/57579b3f-0f25-49af-b810-64b3b0e3d3d1-secret-grpc-tls\") pod \"thanos-querier-6fd7d9ffcd-njp9f\" (UID: \"57579b3f-0f25-49af-b810-64b3b0e3d3d1\") " pod="openshift-monitoring/thanos-querier-6fd7d9ffcd-njp9f" Jul 02 21:37:27.511773 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:27.511733 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/57579b3f-0f25-49af-b810-64b3b0e3d3d1-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6fd7d9ffcd-njp9f\" (UID: \"57579b3f-0f25-49af-b810-64b3b0e3d3d1\") " pod="openshift-monitoring/thanos-querier-6fd7d9ffcd-njp9f" Jul 02 21:37:27.512093 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:27.512069 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/57579b3f-0f25-49af-b810-64b3b0e3d3d1-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6fd7d9ffcd-njp9f\" (UID: \"57579b3f-0f25-49af-b810-64b3b0e3d3d1\") " pod="openshift-monitoring/thanos-querier-6fd7d9ffcd-njp9f" Jul 02 21:37:27.512357 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:27.512334 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/57579b3f-0f25-49af-b810-64b3b0e3d3d1-secret-thanos-querier-tls\") pod \"thanos-querier-6fd7d9ffcd-njp9f\" (UID: \"57579b3f-0f25-49af-b810-64b3b0e3d3d1\") " pod="openshift-monitoring/thanos-querier-6fd7d9ffcd-njp9f" Jul 02 21:37:27.516654 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:27.516632 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftngp\" (UniqueName: \"kubernetes.io/projected/57579b3f-0f25-49af-b810-64b3b0e3d3d1-kube-api-access-ftngp\") pod \"thanos-querier-6fd7d9ffcd-njp9f\" (UID: \"57579b3f-0f25-49af-b810-64b3b0e3d3d1\") " pod="openshift-monitoring/thanos-querier-6fd7d9ffcd-njp9f" Jul 02 21:37:27.612090 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:27.612053 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6fd7d9ffcd-njp9f" Jul 02 21:37:28.409188 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:28.409074 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-g6qqq" event={"ID":"739024f7-d272-4932-b23d-5d815ade3282","Type":"ContainerStarted","Data":"6a6867ab692ec26d62ffa2e8bb9e311578685f28cc2ba552d76a7b1dfa0d06b6"} Jul 02 21:37:28.409188 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:28.409149 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-g6qqq" event={"ID":"739024f7-d272-4932-b23d-5d815ade3282","Type":"ContainerStarted","Data":"44c77d3e303236d4c72d426c8e81f7f9e0b0b65b5ecb3806b80f2366cf86959f"} Jul 02 21:37:28.411175 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:28.411143 2569 generic.go:356] "Generic (PLEG): container finished" podID="f9f5712b-72f4-4536-b1fe-2a4240d463f6" containerID="0782c5805fdf306dd9a7bdf53ee49adfdf515cadff9f24236def8643e9d0d0d2" exitCode=0 Jul 02 21:37:28.411306 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:28.411210 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c8jks" event={"ID":"f9f5712b-72f4-4536-b1fe-2a4240d463f6","Type":"ContainerDied","Data":"0782c5805fdf306dd9a7bdf53ee49adfdf515cadff9f24236def8643e9d0d0d2"} Jul 02 21:37:29.626180 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:29.626140 2569 patch_prober.go:28] interesting pod/image-registry-f788c7bf6-4fwb6 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 02 21:37:29.626704 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:29.626207 2569 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-f788c7bf6-4fwb6" podUID="1adcf74b-7adb-4c3e-859d-2481e729caf1" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 21:37:30.053202 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:30.053167 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-z69cz"] Jul 02 21:37:30.059534 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:30.059511 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-z69cz" Jul 02 21:37:30.061666 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:30.061641 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 02 21:37:30.061917 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:30.061900 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-2q67f\"" Jul 02 21:37:30.067614 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:30.067589 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-z69cz"] Jul 02 21:37:30.134802 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:30.134753 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmdx5\" (UniqueName: \"kubernetes.io/projected/cc6da78f-0511-4162-831c-4a433aaef2a6-kube-api-access-mmdx5\") pod \"monitoring-plugin-d5dcff7cf-z69cz\" (UID: \"cc6da78f-0511-4162-831c-4a433aaef2a6\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-z69cz" Jul 02 21:37:30.134982 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:30.134862 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cc6da78f-0511-4162-831c-4a433aaef2a6-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-z69cz\" (UID: \"cc6da78f-0511-4162-831c-4a433aaef2a6\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-z69cz" Jul 02 21:37:30.235387 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:30.235354 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmdx5\" (UniqueName: \"kubernetes.io/projected/cc6da78f-0511-4162-831c-4a433aaef2a6-kube-api-access-mmdx5\") pod \"monitoring-plugin-d5dcff7cf-z69cz\" (UID: \"cc6da78f-0511-4162-831c-4a433aaef2a6\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-z69cz" Jul 02 21:37:30.235564 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:30.235435 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cc6da78f-0511-4162-831c-4a433aaef2a6-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-z69cz\" (UID: \"cc6da78f-0511-4162-831c-4a433aaef2a6\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-z69cz" Jul 02 21:37:30.235622 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:37:30.235596 2569 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 02 21:37:30.235672 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:37:30.235660 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc6da78f-0511-4162-831c-4a433aaef2a6-monitoring-plugin-cert podName:cc6da78f-0511-4162-831c-4a433aaef2a6 nodeName:}" failed. No retries permitted until 2026-07-02 21:37:30.735638712 +0000 UTC m=+90.074609128 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/cc6da78f-0511-4162-831c-4a433aaef2a6-monitoring-plugin-cert") pod "monitoring-plugin-d5dcff7cf-z69cz" (UID: "cc6da78f-0511-4162-831c-4a433aaef2a6") : secret "monitoring-plugin-cert" not found Jul 02 21:37:30.244144 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:30.244118 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmdx5\" (UniqueName: \"kubernetes.io/projected/cc6da78f-0511-4162-831c-4a433aaef2a6-kube-api-access-mmdx5\") pod \"monitoring-plugin-d5dcff7cf-z69cz\" (UID: \"cc6da78f-0511-4162-831c-4a433aaef2a6\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-z69cz" Jul 02 21:37:30.312443 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:30.312346 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7c596bcd6b-rzj8c" Jul 02 21:37:30.738779 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:30.738686 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cc6da78f-0511-4162-831c-4a433aaef2a6-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-z69cz\" (UID: \"cc6da78f-0511-4162-831c-4a433aaef2a6\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-z69cz" Jul 02 21:37:30.741493 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:30.741465 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cc6da78f-0511-4162-831c-4a433aaef2a6-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-z69cz\" (UID: \"cc6da78f-0511-4162-831c-4a433aaef2a6\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-z69cz" Jul 02 21:37:30.971479 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:30.971448 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-z69cz" Jul 02 21:37:31.534064 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.533513 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 02 21:37:31.625920 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.625878 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 02 21:37:31.626105 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.625973 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.629274 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.628993 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 02 21:37:31.629274 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.629028 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 02 21:37:31.629274 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.629041 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 02 21:37:31.629274 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.629118 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 02 21:37:31.629274 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.629138 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-rb88p\"" Jul 02 21:37:31.629960 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.629774 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 02 21:37:31.629960 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.629836 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 02 21:37:31.630138 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.630117 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 02 21:37:31.630310 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.630294 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 02 21:37:31.630982 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.630951 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 02 21:37:31.631601 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.631580 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 02 21:37:31.632020 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.632002 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 02 21:37:31.632336 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.632317 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 02 21:37:31.632694 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.632675 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-91qhd5buaeu34\"" Jul 02 21:37:31.633315 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.633158 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 02 21:37:31.634709 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.634691 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 02 21:37:31.637998 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.637968 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 02 21:37:31.749167 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.749127 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ab3c0820-2dc6-402b-b845-eae86ad75d46-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.749620 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.749198 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhwcf\" (UniqueName: \"kubernetes.io/projected/ab3c0820-2dc6-402b-b845-eae86ad75d46-kube-api-access-lhwcf\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.749620 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.749277 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ab3c0820-2dc6-402b-b845-eae86ad75d46-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.749620 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.749308 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ab3c0820-2dc6-402b-b845-eae86ad75d46-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.749620 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.749347 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ab3c0820-2dc6-402b-b845-eae86ad75d46-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.749620 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.749376 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ab3c0820-2dc6-402b-b845-eae86ad75d46-config\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.749620 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.749415 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ab3c0820-2dc6-402b-b845-eae86ad75d46-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.749620 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.749439 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ab3c0820-2dc6-402b-b845-eae86ad75d46-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.749620 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.749472 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ab3c0820-2dc6-402b-b845-eae86ad75d46-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.749620 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.749529 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ab3c0820-2dc6-402b-b845-eae86ad75d46-web-config\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.749620 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.749580 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ab3c0820-2dc6-402b-b845-eae86ad75d46-config-out\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.749620 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.749609 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab3c0820-2dc6-402b-b845-eae86ad75d46-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.750134 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.749651 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab3c0820-2dc6-402b-b845-eae86ad75d46-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.750134 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.749676 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ab3c0820-2dc6-402b-b845-eae86ad75d46-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.750134 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.749708 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ab3c0820-2dc6-402b-b845-eae86ad75d46-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.750134 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.749760 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ab3c0820-2dc6-402b-b845-eae86ad75d46-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.750134 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.749807 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ab3c0820-2dc6-402b-b845-eae86ad75d46-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.750134 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.749838 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ab3c0820-2dc6-402b-b845-eae86ad75d46-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.750134 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.749877 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab3c0820-2dc6-402b-b845-eae86ad75d46-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.750134 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.749903 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ab3c0820-2dc6-402b-b845-eae86ad75d46-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.854927 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.854837 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ab3c0820-2dc6-402b-b845-eae86ad75d46-config-out\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.855091 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.854979 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab3c0820-2dc6-402b-b845-eae86ad75d46-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.855163 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.855097 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab3c0820-2dc6-402b-b845-eae86ad75d46-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.855220 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.855156 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ab3c0820-2dc6-402b-b845-eae86ad75d46-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.855624 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.855201 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ab3c0820-2dc6-402b-b845-eae86ad75d46-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.855759 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.855667 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ab3c0820-2dc6-402b-b845-eae86ad75d46-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.855759 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.855700 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ab3c0820-2dc6-402b-b845-eae86ad75d46-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.855759 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.855736 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ab3c0820-2dc6-402b-b845-eae86ad75d46-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.855958 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.855767 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab3c0820-2dc6-402b-b845-eae86ad75d46-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.855958 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.855835 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ab3c0820-2dc6-402b-b845-eae86ad75d46-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.855958 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.855879 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ab3c0820-2dc6-402b-b845-eae86ad75d46-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.856278 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.856256 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab3c0820-2dc6-402b-b845-eae86ad75d46-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.856416 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.856393 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab3c0820-2dc6-402b-b845-eae86ad75d46-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.856843 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.856823 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ab3c0820-2dc6-402b-b845-eae86ad75d46-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.857461 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.857435 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ab3c0820-2dc6-402b-b845-eae86ad75d46-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.857553 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.857493 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhwcf\" (UniqueName: \"kubernetes.io/projected/ab3c0820-2dc6-402b-b845-eae86ad75d46-kube-api-access-lhwcf\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.857553 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.857538 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ab3c0820-2dc6-402b-b845-eae86ad75d46-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.857666 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.857568 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ab3c0820-2dc6-402b-b845-eae86ad75d46-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.857666 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.857614 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ab3c0820-2dc6-402b-b845-eae86ad75d46-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.857666 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.857648 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ab3c0820-2dc6-402b-b845-eae86ad75d46-config\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.857833 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.857678 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ab3c0820-2dc6-402b-b845-eae86ad75d46-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.857833 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.857703 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ab3c0820-2dc6-402b-b845-eae86ad75d46-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.857833 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.857741 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ab3c0820-2dc6-402b-b845-eae86ad75d46-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.857833 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.857771 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ab3c0820-2dc6-402b-b845-eae86ad75d46-web-config\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.858045 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.857862 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ab3c0820-2dc6-402b-b845-eae86ad75d46-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.858045 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.857928 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab3c0820-2dc6-402b-b845-eae86ad75d46-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.859344 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.859093 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ab3c0820-2dc6-402b-b845-eae86ad75d46-config-out\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.859344 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.859331 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ab3c0820-2dc6-402b-b845-eae86ad75d46-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.859802 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.859767 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ab3c0820-2dc6-402b-b845-eae86ad75d46-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.862646 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.862170 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ab3c0820-2dc6-402b-b845-eae86ad75d46-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.862646 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.862318 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ab3c0820-2dc6-402b-b845-eae86ad75d46-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.862646 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.862310 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ab3c0820-2dc6-402b-b845-eae86ad75d46-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.862646 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.862412 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ab3c0820-2dc6-402b-b845-eae86ad75d46-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.862646 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.862460 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ab3c0820-2dc6-402b-b845-eae86ad75d46-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.862646 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.862611 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ab3c0820-2dc6-402b-b845-eae86ad75d46-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.862646 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.862628 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/ab3c0820-2dc6-402b-b845-eae86ad75d46-config\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.863226 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.863147 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ab3c0820-2dc6-402b-b845-eae86ad75d46-web-config\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.863226 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.863177 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ab3c0820-2dc6-402b-b845-eae86ad75d46-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.863538 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.863502 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ab3c0820-2dc6-402b-b845-eae86ad75d46-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.867083 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.867065 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhwcf\" (UniqueName: \"kubernetes.io/projected/ab3c0820-2dc6-402b-b845-eae86ad75d46-kube-api-access-lhwcf\") pod \"prometheus-k8s-0\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:31.938852 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:31.938512 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:32.318992 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:32.318961 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-f788c7bf6-4fwb6" Jul 02 21:37:35.177136 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:35.177112 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-z69cz"] Jul 02 21:37:35.181460 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:37:35.181410 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc6da78f_0511_4162_831c_4a433aaef2a6.slice/crio-f0103a9cdd7f027a29767ce33e1a0c78f1689feea1b4252be027d109014a399c WatchSource:0}: Error finding container f0103a9cdd7f027a29767ce33e1a0c78f1689feea1b4252be027d109014a399c: Status 404 returned error can't find the container with id f0103a9cdd7f027a29767ce33e1a0c78f1689feea1b4252be027d109014a399c Jul 02 21:37:35.204691 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:35.204665 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 02 21:37:35.206644 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:37:35.206621 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab3c0820_2dc6_402b_b845_eae86ad75d46.slice/crio-069c68d068e18a3bd056dd4231832d665b7d5abd7189dfe9425641abd251591d WatchSource:0}: Error finding container 069c68d068e18a3bd056dd4231832d665b7d5abd7189dfe9425641abd251591d: Status 404 returned error can't find the container with id 069c68d068e18a3bd056dd4231832d665b7d5abd7189dfe9425641abd251591d Jul 02 21:37:35.221191 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:35.221171 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6fd7d9ffcd-njp9f"] Jul 02 21:37:35.340772 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:35.340700 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7c596bcd6b-rzj8c" podUID="2dc606a9-6a0c-49de-be3e-4c090d343d3e" containerName="registry" containerID="cri-o://6e4c6f719b027d67de6190c51309bec482ac8f0696d2548a32b58b1f74a304e8" gracePeriod=30 Jul 02 21:37:35.421736 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:37:35.421707 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57579b3f_0f25_49af_b810_64b3b0e3d3d1.slice/crio-f97918f1ef2fdf1d6e0cd6490e4f0f449e32957e660fa0dd24ebe5be997de558 WatchSource:0}: Error finding container f97918f1ef2fdf1d6e0cd6490e4f0f449e32957e660fa0dd24ebe5be997de558: Status 404 returned error can't find the container with id f97918f1ef2fdf1d6e0cd6490e4f0f449e32957e660fa0dd24ebe5be997de558 Jul 02 21:37:35.443286 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:35.443258 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c8jks" event={"ID":"f9f5712b-72f4-4536-b1fe-2a4240d463f6","Type":"ContainerStarted","Data":"93c7c863358859ff6a149127583ab7fbf6c2e7a1ad35dd8eae4936d0c7f1ff69"} Jul 02 21:37:35.443400 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:35.443319 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c8jks" event={"ID":"f9f5712b-72f4-4536-b1fe-2a4240d463f6","Type":"ContainerStarted","Data":"6200b84718974f56a860d4211b35d74d197d77624ceadea8669e25b45eaa3f03"} Jul 02 21:37:35.444711 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:35.444684 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ab3c0820-2dc6-402b-b845-eae86ad75d46","Type":"ContainerStarted","Data":"069c68d068e18a3bd056dd4231832d665b7d5abd7189dfe9425641abd251591d"} Jul 02 21:37:35.446171 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:35.446144 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-z69cz" event={"ID":"cc6da78f-0511-4162-831c-4a433aaef2a6","Type":"ContainerStarted","Data":"f0103a9cdd7f027a29767ce33e1a0c78f1689feea1b4252be027d109014a399c"} Jul 02 21:37:35.461051 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:35.461001 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-c8jks" podStartSLOduration=9.386534861 podStartE2EDuration="10.460984035s" totalImagesPullingTime="1.001478545s" totalInitContainerRuntime="72.970629ms" isStatefulPod=true podCreationTimestamp="2026-07-02 21:37:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:37:35.459028614 +0000 UTC m=+94.797999050" watchObservedRunningTime="2026-07-02 21:37:35.460984035 +0000 UTC m=+94.799954469" Jul 02 21:37:35.614415 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:35.614391 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c596bcd6b-rzj8c" Jul 02 21:37:35.798045 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:35.798003 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2dc606a9-6a0c-49de-be3e-4c090d343d3e-registry-tls\") pod \"2dc606a9-6a0c-49de-be3e-4c090d343d3e\" (UID: \"2dc606a9-6a0c-49de-be3e-4c090d343d3e\") " Jul 02 21:37:35.798304 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:35.798076 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2dc606a9-6a0c-49de-be3e-4c090d343d3e-bound-sa-token\") pod \"2dc606a9-6a0c-49de-be3e-4c090d343d3e\" (UID: \"2dc606a9-6a0c-49de-be3e-4c090d343d3e\") " Jul 02 21:37:35.798304 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:35.798124 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2dc606a9-6a0c-49de-be3e-4c090d343d3e-ca-trust-extracted\") pod \"2dc606a9-6a0c-49de-be3e-4c090d343d3e\" (UID: \"2dc606a9-6a0c-49de-be3e-4c090d343d3e\") " Jul 02 21:37:35.798304 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:35.798156 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2dc606a9-6a0c-49de-be3e-4c090d343d3e-image-registry-private-configuration\") pod \"2dc606a9-6a0c-49de-be3e-4c090d343d3e\" (UID: \"2dc606a9-6a0c-49de-be3e-4c090d343d3e\") " Jul 02 21:37:35.798304 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:35.798191 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x9xps\" (UniqueName: \"kubernetes.io/projected/2dc606a9-6a0c-49de-be3e-4c090d343d3e-kube-api-access-x9xps\") pod \"2dc606a9-6a0c-49de-be3e-4c090d343d3e\" (UID: \"2dc606a9-6a0c-49de-be3e-4c090d343d3e\") " Jul 02 21:37:35.798304 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:35.798228 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2dc606a9-6a0c-49de-be3e-4c090d343d3e-installation-pull-secrets\") pod \"2dc606a9-6a0c-49de-be3e-4c090d343d3e\" (UID: \"2dc606a9-6a0c-49de-be3e-4c090d343d3e\") " Jul 02 21:37:35.798304 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:35.798267 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2dc606a9-6a0c-49de-be3e-4c090d343d3e-trusted-ca\") pod \"2dc606a9-6a0c-49de-be3e-4c090d343d3e\" (UID: \"2dc606a9-6a0c-49de-be3e-4c090d343d3e\") " Jul 02 21:37:35.798633 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:35.798326 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2dc606a9-6a0c-49de-be3e-4c090d343d3e-registry-certificates\") pod \"2dc606a9-6a0c-49de-be3e-4c090d343d3e\" (UID: \"2dc606a9-6a0c-49de-be3e-4c090d343d3e\") " Jul 02 21:37:35.799051 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:35.799003 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2dc606a9-6a0c-49de-be3e-4c090d343d3e-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "2dc606a9-6a0c-49de-be3e-4c090d343d3e" (UID: "2dc606a9-6a0c-49de-be3e-4c090d343d3e"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 21:37:35.799448 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:35.799397 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2dc606a9-6a0c-49de-be3e-4c090d343d3e-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "2dc606a9-6a0c-49de-be3e-4c090d343d3e" (UID: "2dc606a9-6a0c-49de-be3e-4c090d343d3e"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 21:37:35.801889 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:35.801842 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2dc606a9-6a0c-49de-be3e-4c090d343d3e-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "2dc606a9-6a0c-49de-be3e-4c090d343d3e" (UID: "2dc606a9-6a0c-49de-be3e-4c090d343d3e"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 21:37:35.802099 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:35.802039 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2dc606a9-6a0c-49de-be3e-4c090d343d3e-kube-api-access-x9xps" (OuterVolumeSpecName: "kube-api-access-x9xps") pod "2dc606a9-6a0c-49de-be3e-4c090d343d3e" (UID: "2dc606a9-6a0c-49de-be3e-4c090d343d3e"). InnerVolumeSpecName "kube-api-access-x9xps". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 21:37:35.802099 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:35.802137 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2dc606a9-6a0c-49de-be3e-4c090d343d3e-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "2dc606a9-6a0c-49de-be3e-4c090d343d3e" (UID: "2dc606a9-6a0c-49de-be3e-4c090d343d3e"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 21:37:35.802540 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:35.802507 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2dc606a9-6a0c-49de-be3e-4c090d343d3e-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "2dc606a9-6a0c-49de-be3e-4c090d343d3e" (UID: "2dc606a9-6a0c-49de-be3e-4c090d343d3e"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 21:37:35.802862 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:35.802837 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2dc606a9-6a0c-49de-be3e-4c090d343d3e-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "2dc606a9-6a0c-49de-be3e-4c090d343d3e" (UID: "2dc606a9-6a0c-49de-be3e-4c090d343d3e"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 21:37:35.811521 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:35.811493 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2dc606a9-6a0c-49de-be3e-4c090d343d3e-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "2dc606a9-6a0c-49de-be3e-4c090d343d3e" (UID: "2dc606a9-6a0c-49de-be3e-4c090d343d3e"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 21:37:35.899696 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:35.899617 2569 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2dc606a9-6a0c-49de-be3e-4c090d343d3e-registry-certificates\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 21:37:35.899696 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:35.899653 2569 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2dc606a9-6a0c-49de-be3e-4c090d343d3e-registry-tls\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 21:37:35.899696 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:35.899662 2569 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2dc606a9-6a0c-49de-be3e-4c090d343d3e-bound-sa-token\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 21:37:35.899696 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:35.899670 2569 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2dc606a9-6a0c-49de-be3e-4c090d343d3e-ca-trust-extracted\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 21:37:35.899696 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:35.899679 2569 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2dc606a9-6a0c-49de-be3e-4c090d343d3e-image-registry-private-configuration\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 21:37:35.899696 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:35.899689 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x9xps\" (UniqueName: \"kubernetes.io/projected/2dc606a9-6a0c-49de-be3e-4c090d343d3e-kube-api-access-x9xps\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 21:37:35.899696 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:35.899698 2569 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2dc606a9-6a0c-49de-be3e-4c090d343d3e-installation-pull-secrets\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 21:37:35.900133 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:35.899711 2569 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2dc606a9-6a0c-49de-be3e-4c090d343d3e-trusted-ca\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 21:37:36.462063 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:36.459072 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-ftvjj" event={"ID":"cc6ebdd7-4265-45cc-86d2-b36a76ffe2c0","Type":"ContainerStarted","Data":"72162b057f631c9ea52b48e95c4c9c70c27cfe0bccafbb40706417d7a4b347a2"} Jul 02 21:37:36.462523 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:36.462474 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-ftvjj" Jul 02 21:37:36.466279 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:36.466249 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6fd7d9ffcd-njp9f" event={"ID":"57579b3f-0f25-49af-b810-64b3b0e3d3d1","Type":"ContainerStarted","Data":"f97918f1ef2fdf1d6e0cd6490e4f0f449e32957e660fa0dd24ebe5be997de558"} Jul 02 21:37:36.469359 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:36.469133 2569 generic.go:356] "Generic (PLEG): container finished" podID="2dc606a9-6a0c-49de-be3e-4c090d343d3e" containerID="6e4c6f719b027d67de6190c51309bec482ac8f0696d2548a32b58b1f74a304e8" exitCode=0 Jul 02 21:37:36.469909 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:36.469640 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c596bcd6b-rzj8c" event={"ID":"2dc606a9-6a0c-49de-be3e-4c090d343d3e","Type":"ContainerDied","Data":"6e4c6f719b027d67de6190c51309bec482ac8f0696d2548a32b58b1f74a304e8"} Jul 02 21:37:36.469909 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:36.469724 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c596bcd6b-rzj8c" event={"ID":"2dc606a9-6a0c-49de-be3e-4c090d343d3e","Type":"ContainerDied","Data":"6d9de92d643f496db3379990099e91626b20e8a5f111bb0072b8c8dabcbf35ca"} Jul 02 21:37:36.469909 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:36.469762 2569 scope.go:117] "RemoveContainer" containerID="6e4c6f719b027d67de6190c51309bec482ac8f0696d2548a32b58b1f74a304e8" Jul 02 21:37:36.472258 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:36.470818 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c596bcd6b-rzj8c" Jul 02 21:37:36.478178 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:36.478071 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-ftvjj" Jul 02 21:37:36.480406 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:36.479249 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-ftvjj" podStartSLOduration=1.7994605639999999 podStartE2EDuration="21.479231581s" totalImagesPullingTime="19.679771017s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:37:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:37:36.476694752 +0000 UTC m=+95.815665192" watchObservedRunningTime="2026-07-02 21:37:36.479231581 +0000 UTC m=+95.818202018" Jul 02 21:37:36.522646 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:36.522611 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7c596bcd6b-rzj8c"] Jul 02 21:37:36.526196 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:36.526145 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7c596bcd6b-rzj8c"] Jul 02 21:37:36.967134 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:36.966712 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2dc606a9-6a0c-49de-be3e-4c090d343d3e" path="/var/lib/kubelet/pods/2dc606a9-6a0c-49de-be3e-4c090d343d3e/volumes" Jul 02 21:37:37.400521 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:37.400443 2569 scope.go:117] "RemoveContainer" containerID="6e4c6f719b027d67de6190c51309bec482ac8f0696d2548a32b58b1f74a304e8" Jul 02 21:37:37.400948 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:37:37.400909 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e4c6f719b027d67de6190c51309bec482ac8f0696d2548a32b58b1f74a304e8\": container with ID starting with 6e4c6f719b027d67de6190c51309bec482ac8f0696d2548a32b58b1f74a304e8 not found: ID does not exist" containerID="6e4c6f719b027d67de6190c51309bec482ac8f0696d2548a32b58b1f74a304e8" Jul 02 21:37:37.401083 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:37.400957 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e4c6f719b027d67de6190c51309bec482ac8f0696d2548a32b58b1f74a304e8"} err="failed to get container status \"6e4c6f719b027d67de6190c51309bec482ac8f0696d2548a32b58b1f74a304e8\": rpc error: code = NotFound desc = could not find container \"6e4c6f719b027d67de6190c51309bec482ac8f0696d2548a32b58b1f74a304e8\": container with ID starting with 6e4c6f719b027d67de6190c51309bec482ac8f0696d2548a32b58b1f74a304e8 not found: ID does not exist" Jul 02 21:37:37.485830 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:37.485620 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-g6qqq" event={"ID":"739024f7-d272-4932-b23d-5d815ade3282","Type":"ContainerStarted","Data":"488ce7d211c9b63b663e87727c3634000c24561bc9437824605f04772680f929"} Jul 02 21:37:39.496760 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:39.496666 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6fd7d9ffcd-njp9f" event={"ID":"57579b3f-0f25-49af-b810-64b3b0e3d3d1","Type":"ContainerStarted","Data":"de3233461509cc8e1764e4054cda7fd3fac2251f91cb51fc0b3cafba49900236"} Jul 02 21:37:39.498537 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:39.498457 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-z69cz" event={"ID":"cc6da78f-0511-4162-831c-4a433aaef2a6","Type":"ContainerStarted","Data":"a3641f92a7ddfd2bb0bb773639d615463cb999b6269ca86ac175c3334864caf8"} Jul 02 21:37:39.502000 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:39.501590 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ab3c0820-2dc6-402b-b845-eae86ad75d46","Type":"ContainerStarted","Data":"4083ee39b13639009eff5623a7f398e9de4bfc682e0fd699a179934d5e6d1262"} Jul 02 21:37:39.541138 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:39.541046 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-g6qqq" podStartSLOduration=5.656684095 podStartE2EDuration="14.540996817s" totalImagesPullingTime="8.884312722s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:37:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:37:37.505098751 +0000 UTC m=+96.844069188" watchObservedRunningTime="2026-07-02 21:37:39.540996817 +0000 UTC m=+98.879967255" Jul 02 21:37:40.511974 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:40.511651 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6fd7d9ffcd-njp9f" event={"ID":"57579b3f-0f25-49af-b810-64b3b0e3d3d1","Type":"ContainerStarted","Data":"30a2c3740cfb39c3299e78e4f7ad4dca8a104498aa383082faabf12db3b826b7"} Jul 02 21:37:40.511974 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:40.511751 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6fd7d9ffcd-njp9f" event={"ID":"57579b3f-0f25-49af-b810-64b3b0e3d3d1","Type":"ContainerStarted","Data":"5b5384874e7a3987173e9bacf423b2d25671e1766e93c412bd858f1f94a43a01"} Jul 02 21:37:40.514163 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:40.513716 2569 generic.go:356] "Generic (PLEG): container finished" podID="ab3c0820-2dc6-402b-b845-eae86ad75d46" containerID="4083ee39b13639009eff5623a7f398e9de4bfc682e0fd699a179934d5e6d1262" exitCode=0 Jul 02 21:37:40.514419 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:40.514297 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ab3c0820-2dc6-402b-b845-eae86ad75d46","Type":"ContainerDied","Data":"4083ee39b13639009eff5623a7f398e9de4bfc682e0fd699a179934d5e6d1262"} Jul 02 21:37:40.514419 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:40.514377 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-z69cz" Jul 02 21:37:40.520542 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:40.520488 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-z69cz" Jul 02 21:37:40.537520 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:40.537468 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-z69cz" podStartSLOduration=6.431572804 podStartE2EDuration="10.537451442s" totalImagesPullingTime="4.105878638s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:37:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:37:39.554806836 +0000 UTC m=+98.893777273" watchObservedRunningTime="2026-07-02 21:37:40.537451442 +0000 UTC m=+99.876421878" Jul 02 21:37:41.526851 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:41.526000 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6fd7d9ffcd-njp9f" event={"ID":"57579b3f-0f25-49af-b810-64b3b0e3d3d1","Type":"ContainerStarted","Data":"4eb1510d34e9604f5a0b006a4c7589ea0009bf662bb1ac63b70ac86e9e6875e1"} Jul 02 21:37:41.526851 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:41.526094 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6fd7d9ffcd-njp9f" event={"ID":"57579b3f-0f25-49af-b810-64b3b0e3d3d1","Type":"ContainerStarted","Data":"9d0b651e3fe1210bfb26aa3bd201df51080c6ef31be362ac3bd28a1ec39c120a"} Jul 02 21:37:41.526851 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:41.526143 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6fd7d9ffcd-njp9f" event={"ID":"57579b3f-0f25-49af-b810-64b3b0e3d3d1","Type":"ContainerStarted","Data":"e82cf44cb811bddb0f3192ef0d3a3860d8125db03c14104e59704801716228a8"} Jul 02 21:37:41.526851 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:41.526316 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6fd7d9ffcd-njp9f" Jul 02 21:37:44.543562 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:44.543489 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ab3c0820-2dc6-402b-b845-eae86ad75d46","Type":"ContainerStarted","Data":"562349de69ebe9323c0318788249f1143fe13487f597996c67f838e19777941f"} Jul 02 21:37:44.543908 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:44.543575 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ab3c0820-2dc6-402b-b845-eae86ad75d46","Type":"ContainerStarted","Data":"59679e4aba2a9a94b4412190386a648b7dd6e371c15fa6bb2884fd4d151976dc"} Jul 02 21:37:44.543908 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:44.543603 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ab3c0820-2dc6-402b-b845-eae86ad75d46","Type":"ContainerStarted","Data":"ffb9c3a2757384933124acf4f890c186c55202281a29e944daa1e99108774f41"} Jul 02 21:37:44.543908 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:44.543627 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ab3c0820-2dc6-402b-b845-eae86ad75d46","Type":"ContainerStarted","Data":"994ffbd2a36fe6076996a91c08a2b3e3c3f8d007f6810bb03ca1c8bc9a4edf91"} Jul 02 21:37:44.543908 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:44.543650 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ab3c0820-2dc6-402b-b845-eae86ad75d46","Type":"ContainerStarted","Data":"c7729839da28689f661acfd091f3a98790de2ff973e0c72992773f42ff25912b"} Jul 02 21:37:44.543908 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:44.543673 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ab3c0820-2dc6-402b-b845-eae86ad75d46","Type":"ContainerStarted","Data":"cb369aeea922de2f8d9c77fa6c887b92f4883d33d6f2301f4e264f0f3aa70eca"} Jul 02 21:37:44.569996 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:44.569945 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=5.454197692 podStartE2EDuration="13.569924703s" totalImagesPullingTime="8.093060656s" totalInitContainerRuntime="22.666355ms" isStatefulPod=false podCreationTimestamp="2026-07-02 21:37:31 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:37:44.568632895 +0000 UTC m=+103.907603366" watchObservedRunningTime="2026-07-02 21:37:44.569924703 +0000 UTC m=+103.908895139" Jul 02 21:37:44.570318 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:44.570295 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6fd7d9ffcd-njp9f" podStartSLOduration=12.358276007 podStartE2EDuration="17.570287662s" totalImagesPullingTime="5.212011655s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:37:27 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:37:41.554946397 +0000 UTC m=+100.893916835" watchObservedRunningTime="2026-07-02 21:37:44.570287662 +0000 UTC m=+103.909258096" Jul 02 21:37:46.342744 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:46.342718 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-jgn9h" Jul 02 21:37:46.939644 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:46.939607 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:37:47.537339 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:37:47.537312 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6fd7d9ffcd-njp9f" Jul 02 21:38:31.939974 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:31.939922 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:31.958246 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:31.958218 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:32.741934 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:32.741908 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:49.974143 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:49.974106 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 02 21:38:49.974647 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:49.974401 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ab3c0820-2dc6-402b-b845-eae86ad75d46" containerName="prometheus" containerID="cri-o://cb369aeea922de2f8d9c77fa6c887b92f4883d33d6f2301f4e264f0f3aa70eca" gracePeriod=600 Jul 02 21:38:49.974647 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:49.974402 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ab3c0820-2dc6-402b-b845-eae86ad75d46" containerName="kube-rbac-proxy" containerID="cri-o://59679e4aba2a9a94b4412190386a648b7dd6e371c15fa6bb2884fd4d151976dc" gracePeriod=600 Jul 02 21:38:49.974647 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:49.974432 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ab3c0820-2dc6-402b-b845-eae86ad75d46" containerName="thanos-sidecar" containerID="cri-o://994ffbd2a36fe6076996a91c08a2b3e3c3f8d007f6810bb03ca1c8bc9a4edf91" gracePeriod=600 Jul 02 21:38:49.974647 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:49.974470 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ab3c0820-2dc6-402b-b845-eae86ad75d46" containerName="config-reloader" containerID="cri-o://c7729839da28689f661acfd091f3a98790de2ff973e0c72992773f42ff25912b" gracePeriod=600 Jul 02 21:38:49.974647 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:49.974517 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ab3c0820-2dc6-402b-b845-eae86ad75d46" containerName="kube-rbac-proxy-thanos" containerID="cri-o://562349de69ebe9323c0318788249f1143fe13487f597996c67f838e19777941f" gracePeriod=600 Jul 02 21:38:49.974647 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:49.974575 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ab3c0820-2dc6-402b-b845-eae86ad75d46" containerName="kube-rbac-proxy-web" containerID="cri-o://ffb9c3a2757384933124acf4f890c186c55202281a29e944daa1e99108774f41" gracePeriod=600 Jul 02 21:38:50.230379 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.229985 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:50.343964 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.343926 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ab3c0820-2dc6-402b-b845-eae86ad75d46-secret-kube-rbac-proxy\") pod \"ab3c0820-2dc6-402b-b845-eae86ad75d46\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " Jul 02 21:38:50.343964 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.343963 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ab3c0820-2dc6-402b-b845-eae86ad75d46-prometheus-k8s-rulefiles-2\") pod \"ab3c0820-2dc6-402b-b845-eae86ad75d46\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " Jul 02 21:38:50.344173 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.343995 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab3c0820-2dc6-402b-b845-eae86ad75d46-configmap-serving-certs-ca-bundle\") pod \"ab3c0820-2dc6-402b-b845-eae86ad75d46\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " Jul 02 21:38:50.344173 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.344014 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ab3c0820-2dc6-402b-b845-eae86ad75d46-configmap-metrics-client-ca\") pod \"ab3c0820-2dc6-402b-b845-eae86ad75d46\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " Jul 02 21:38:50.344173 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.344032 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab3c0820-2dc6-402b-b845-eae86ad75d46-prometheus-trusted-ca-bundle\") pod \"ab3c0820-2dc6-402b-b845-eae86ad75d46\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " Jul 02 21:38:50.344173 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.344061 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ab3c0820-2dc6-402b-b845-eae86ad75d46-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"ab3c0820-2dc6-402b-b845-eae86ad75d46\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " Jul 02 21:38:50.344173 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.344081 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ab3c0820-2dc6-402b-b845-eae86ad75d46-prometheus-k8s-rulefiles-1\") pod \"ab3c0820-2dc6-402b-b845-eae86ad75d46\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " Jul 02 21:38:50.344173 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.344096 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ab3c0820-2dc6-402b-b845-eae86ad75d46-prometheus-k8s-rulefiles-0\") pod \"ab3c0820-2dc6-402b-b845-eae86ad75d46\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " Jul 02 21:38:50.344173 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.344119 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ab3c0820-2dc6-402b-b845-eae86ad75d46-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"ab3c0820-2dc6-402b-b845-eae86ad75d46\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " Jul 02 21:38:50.344173 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.344150 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ab3c0820-2dc6-402b-b845-eae86ad75d46-config-out\") pod \"ab3c0820-2dc6-402b-b845-eae86ad75d46\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " Jul 02 21:38:50.344173 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.344175 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ab3c0820-2dc6-402b-b845-eae86ad75d46-web-config\") pod \"ab3c0820-2dc6-402b-b845-eae86ad75d46\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " Jul 02 21:38:50.344581 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.344199 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ab3c0820-2dc6-402b-b845-eae86ad75d46-secret-grpc-tls\") pod \"ab3c0820-2dc6-402b-b845-eae86ad75d46\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " Jul 02 21:38:50.344581 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.344235 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ab3c0820-2dc6-402b-b845-eae86ad75d46-thanos-prometheus-http-client-file\") pod \"ab3c0820-2dc6-402b-b845-eae86ad75d46\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " Jul 02 21:38:50.344581 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.344279 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lhwcf\" (UniqueName: \"kubernetes.io/projected/ab3c0820-2dc6-402b-b845-eae86ad75d46-kube-api-access-lhwcf\") pod \"ab3c0820-2dc6-402b-b845-eae86ad75d46\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " Jul 02 21:38:50.344581 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.344303 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab3c0820-2dc6-402b-b845-eae86ad75d46-configmap-kubelet-serving-ca-bundle\") pod \"ab3c0820-2dc6-402b-b845-eae86ad75d46\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " Jul 02 21:38:50.344581 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.344354 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ab3c0820-2dc6-402b-b845-eae86ad75d46-prometheus-k8s-db\") pod \"ab3c0820-2dc6-402b-b845-eae86ad75d46\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " Jul 02 21:38:50.344581 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.344397 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ab3c0820-2dc6-402b-b845-eae86ad75d46-config\") pod \"ab3c0820-2dc6-402b-b845-eae86ad75d46\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " Jul 02 21:38:50.344581 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.344395 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab3c0820-2dc6-402b-b845-eae86ad75d46-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "ab3c0820-2dc6-402b-b845-eae86ad75d46" (UID: "ab3c0820-2dc6-402b-b845-eae86ad75d46"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 21:38:50.344581 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.344433 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ab3c0820-2dc6-402b-b845-eae86ad75d46-tls-assets\") pod \"ab3c0820-2dc6-402b-b845-eae86ad75d46\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " Jul 02 21:38:50.344581 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.344454 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab3c0820-2dc6-402b-b845-eae86ad75d46-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "ab3c0820-2dc6-402b-b845-eae86ad75d46" (UID: "ab3c0820-2dc6-402b-b845-eae86ad75d46"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 21:38:50.344581 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.344466 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ab3c0820-2dc6-402b-b845-eae86ad75d46-secret-metrics-client-certs\") pod \"ab3c0820-2dc6-402b-b845-eae86ad75d46\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " Jul 02 21:38:50.344581 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.344455 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab3c0820-2dc6-402b-b845-eae86ad75d46-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "ab3c0820-2dc6-402b-b845-eae86ad75d46" (UID: "ab3c0820-2dc6-402b-b845-eae86ad75d46"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 21:38:50.344581 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.344502 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ab3c0820-2dc6-402b-b845-eae86ad75d46-secret-prometheus-k8s-tls\") pod \"ab3c0820-2dc6-402b-b845-eae86ad75d46\" (UID: \"ab3c0820-2dc6-402b-b845-eae86ad75d46\") " Jul 02 21:38:50.345124 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.344745 2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ab3c0820-2dc6-402b-b845-eae86ad75d46-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 21:38:50.345124 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.344745 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab3c0820-2dc6-402b-b845-eae86ad75d46-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "ab3c0820-2dc6-402b-b845-eae86ad75d46" (UID: "ab3c0820-2dc6-402b-b845-eae86ad75d46"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 21:38:50.345124 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.344764 2569 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab3c0820-2dc6-402b-b845-eae86ad75d46-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 21:38:50.345124 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.344779 2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab3c0820-2dc6-402b-b845-eae86ad75d46-prometheus-trusted-ca-bundle\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 21:38:50.345124 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.344860 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab3c0820-2dc6-402b-b845-eae86ad75d46-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "ab3c0820-2dc6-402b-b845-eae86ad75d46" (UID: "ab3c0820-2dc6-402b-b845-eae86ad75d46"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 21:38:50.345124 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.344876 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab3c0820-2dc6-402b-b845-eae86ad75d46-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "ab3c0820-2dc6-402b-b845-eae86ad75d46" (UID: "ab3c0820-2dc6-402b-b845-eae86ad75d46"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 21:38:50.346941 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.346905 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab3c0820-2dc6-402b-b845-eae86ad75d46-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "ab3c0820-2dc6-402b-b845-eae86ad75d46" (UID: "ab3c0820-2dc6-402b-b845-eae86ad75d46"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 21:38:50.347738 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.347667 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab3c0820-2dc6-402b-b845-eae86ad75d46-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "ab3c0820-2dc6-402b-b845-eae86ad75d46" (UID: "ab3c0820-2dc6-402b-b845-eae86ad75d46"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 21:38:50.347738 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.347709 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab3c0820-2dc6-402b-b845-eae86ad75d46-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "ab3c0820-2dc6-402b-b845-eae86ad75d46" (UID: "ab3c0820-2dc6-402b-b845-eae86ad75d46"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 21:38:50.347911 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.347773 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab3c0820-2dc6-402b-b845-eae86ad75d46-config" (OuterVolumeSpecName: "config") pod "ab3c0820-2dc6-402b-b845-eae86ad75d46" (UID: "ab3c0820-2dc6-402b-b845-eae86ad75d46"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 21:38:50.347911 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.347883 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab3c0820-2dc6-402b-b845-eae86ad75d46-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "ab3c0820-2dc6-402b-b845-eae86ad75d46" (UID: "ab3c0820-2dc6-402b-b845-eae86ad75d46"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 21:38:50.348042 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.347934 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab3c0820-2dc6-402b-b845-eae86ad75d46-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "ab3c0820-2dc6-402b-b845-eae86ad75d46" (UID: "ab3c0820-2dc6-402b-b845-eae86ad75d46"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 21:38:50.348042 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.347961 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab3c0820-2dc6-402b-b845-eae86ad75d46-kube-api-access-lhwcf" (OuterVolumeSpecName: "kube-api-access-lhwcf") pod "ab3c0820-2dc6-402b-b845-eae86ad75d46" (UID: "ab3c0820-2dc6-402b-b845-eae86ad75d46"). InnerVolumeSpecName "kube-api-access-lhwcf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 21:38:50.348239 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.348135 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab3c0820-2dc6-402b-b845-eae86ad75d46-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "ab3c0820-2dc6-402b-b845-eae86ad75d46" (UID: "ab3c0820-2dc6-402b-b845-eae86ad75d46"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 21:38:50.348239 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.348201 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab3c0820-2dc6-402b-b845-eae86ad75d46-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "ab3c0820-2dc6-402b-b845-eae86ad75d46" (UID: "ab3c0820-2dc6-402b-b845-eae86ad75d46"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 21:38:50.348389 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.348358 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab3c0820-2dc6-402b-b845-eae86ad75d46-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "ab3c0820-2dc6-402b-b845-eae86ad75d46" (UID: "ab3c0820-2dc6-402b-b845-eae86ad75d46"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 21:38:50.348893 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.348873 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab3c0820-2dc6-402b-b845-eae86ad75d46-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "ab3c0820-2dc6-402b-b845-eae86ad75d46" (UID: "ab3c0820-2dc6-402b-b845-eae86ad75d46"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 21:38:50.349368 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.349354 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab3c0820-2dc6-402b-b845-eae86ad75d46-config-out" (OuterVolumeSpecName: "config-out") pod "ab3c0820-2dc6-402b-b845-eae86ad75d46" (UID: "ab3c0820-2dc6-402b-b845-eae86ad75d46"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 21:38:50.349493 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.349475 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab3c0820-2dc6-402b-b845-eae86ad75d46-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "ab3c0820-2dc6-402b-b845-eae86ad75d46" (UID: "ab3c0820-2dc6-402b-b845-eae86ad75d46"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 21:38:50.359896 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.359875 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab3c0820-2dc6-402b-b845-eae86ad75d46-web-config" (OuterVolumeSpecName: "web-config") pod "ab3c0820-2dc6-402b-b845-eae86ad75d46" (UID: "ab3c0820-2dc6-402b-b845-eae86ad75d46"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 21:38:50.445465 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.445430 2569 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/ab3c0820-2dc6-402b-b845-eae86ad75d46-config\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 21:38:50.445465 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.445459 2569 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ab3c0820-2dc6-402b-b845-eae86ad75d46-tls-assets\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 21:38:50.445465 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.445471 2569 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ab3c0820-2dc6-402b-b845-eae86ad75d46-secret-metrics-client-certs\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 21:38:50.445686 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.445480 2569 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ab3c0820-2dc6-402b-b845-eae86ad75d46-secret-prometheus-k8s-tls\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 21:38:50.445686 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.445490 2569 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ab3c0820-2dc6-402b-b845-eae86ad75d46-secret-kube-rbac-proxy\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 21:38:50.445686 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.445498 2569 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ab3c0820-2dc6-402b-b845-eae86ad75d46-configmap-metrics-client-ca\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 21:38:50.445686 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.445508 2569 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ab3c0820-2dc6-402b-b845-eae86ad75d46-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 21:38:50.445686 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.445517 2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ab3c0820-2dc6-402b-b845-eae86ad75d46-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 21:38:50.445686 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.445525 2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ab3c0820-2dc6-402b-b845-eae86ad75d46-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 21:38:50.445686 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.445534 2569 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ab3c0820-2dc6-402b-b845-eae86ad75d46-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 21:38:50.445686 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.445542 2569 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ab3c0820-2dc6-402b-b845-eae86ad75d46-config-out\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 21:38:50.445686 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.445550 2569 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ab3c0820-2dc6-402b-b845-eae86ad75d46-web-config\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 21:38:50.445686 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.445558 2569 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ab3c0820-2dc6-402b-b845-eae86ad75d46-secret-grpc-tls\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 21:38:50.445686 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.445566 2569 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ab3c0820-2dc6-402b-b845-eae86ad75d46-thanos-prometheus-http-client-file\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 21:38:50.445686 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.445575 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lhwcf\" (UniqueName: \"kubernetes.io/projected/ab3c0820-2dc6-402b-b845-eae86ad75d46-kube-api-access-lhwcf\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 21:38:50.445686 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.445583 2569 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab3c0820-2dc6-402b-b845-eae86ad75d46-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 21:38:50.445686 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.445592 2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ab3c0820-2dc6-402b-b845-eae86ad75d46-prometheus-k8s-db\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 21:38:50.795939 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.795908 2569 generic.go:356] "Generic (PLEG): container finished" podID="ab3c0820-2dc6-402b-b845-eae86ad75d46" containerID="562349de69ebe9323c0318788249f1143fe13487f597996c67f838e19777941f" exitCode=0 Jul 02 21:38:50.795939 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.795934 2569 generic.go:356] "Generic (PLEG): container finished" podID="ab3c0820-2dc6-402b-b845-eae86ad75d46" containerID="59679e4aba2a9a94b4412190386a648b7dd6e371c15fa6bb2884fd4d151976dc" exitCode=0 Jul 02 21:38:50.795939 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.795940 2569 generic.go:356] "Generic (PLEG): container finished" podID="ab3c0820-2dc6-402b-b845-eae86ad75d46" containerID="ffb9c3a2757384933124acf4f890c186c55202281a29e944daa1e99108774f41" exitCode=0 Jul 02 21:38:50.795939 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.795948 2569 generic.go:356] "Generic (PLEG): container finished" podID="ab3c0820-2dc6-402b-b845-eae86ad75d46" containerID="994ffbd2a36fe6076996a91c08a2b3e3c3f8d007f6810bb03ca1c8bc9a4edf91" exitCode=0 Jul 02 21:38:50.796199 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.795955 2569 generic.go:356] "Generic (PLEG): container finished" podID="ab3c0820-2dc6-402b-b845-eae86ad75d46" containerID="c7729839da28689f661acfd091f3a98790de2ff973e0c72992773f42ff25912b" exitCode=0 Jul 02 21:38:50.796199 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.795961 2569 generic.go:356] "Generic (PLEG): container finished" podID="ab3c0820-2dc6-402b-b845-eae86ad75d46" containerID="cb369aeea922de2f8d9c77fa6c887b92f4883d33d6f2301f4e264f0f3aa70eca" exitCode=0 Jul 02 21:38:50.796199 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.795984 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:50.796199 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.795998 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ab3c0820-2dc6-402b-b845-eae86ad75d46","Type":"ContainerDied","Data":"562349de69ebe9323c0318788249f1143fe13487f597996c67f838e19777941f"} Jul 02 21:38:50.796199 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.796080 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ab3c0820-2dc6-402b-b845-eae86ad75d46","Type":"ContainerDied","Data":"59679e4aba2a9a94b4412190386a648b7dd6e371c15fa6bb2884fd4d151976dc"} Jul 02 21:38:50.796199 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.796126 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ab3c0820-2dc6-402b-b845-eae86ad75d46","Type":"ContainerDied","Data":"ffb9c3a2757384933124acf4f890c186c55202281a29e944daa1e99108774f41"} Jul 02 21:38:50.796199 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.796144 2569 scope.go:117] "RemoveContainer" containerID="562349de69ebe9323c0318788249f1143fe13487f597996c67f838e19777941f" Jul 02 21:38:50.796199 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.796170 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ab3c0820-2dc6-402b-b845-eae86ad75d46","Type":"ContainerDied","Data":"994ffbd2a36fe6076996a91c08a2b3e3c3f8d007f6810bb03ca1c8bc9a4edf91"} Jul 02 21:38:50.796445 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.796211 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ab3c0820-2dc6-402b-b845-eae86ad75d46","Type":"ContainerDied","Data":"c7729839da28689f661acfd091f3a98790de2ff973e0c72992773f42ff25912b"} Jul 02 21:38:50.796445 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.796252 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ab3c0820-2dc6-402b-b845-eae86ad75d46","Type":"ContainerDied","Data":"cb369aeea922de2f8d9c77fa6c887b92f4883d33d6f2301f4e264f0f3aa70eca"} Jul 02 21:38:50.796445 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.796294 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ab3c0820-2dc6-402b-b845-eae86ad75d46","Type":"ContainerDied","Data":"069c68d068e18a3bd056dd4231832d665b7d5abd7189dfe9425641abd251591d"} Jul 02 21:38:50.803609 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.803593 2569 scope.go:117] "RemoveContainer" containerID="59679e4aba2a9a94b4412190386a648b7dd6e371c15fa6bb2884fd4d151976dc" Jul 02 21:38:50.810932 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.810916 2569 scope.go:117] "RemoveContainer" containerID="ffb9c3a2757384933124acf4f890c186c55202281a29e944daa1e99108774f41" Jul 02 21:38:50.817286 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.817263 2569 scope.go:117] "RemoveContainer" containerID="994ffbd2a36fe6076996a91c08a2b3e3c3f8d007f6810bb03ca1c8bc9a4edf91" Jul 02 21:38:50.824498 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.822193 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 02 21:38:50.826392 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.824883 2569 scope.go:117] "RemoveContainer" containerID="c7729839da28689f661acfd091f3a98790de2ff973e0c72992773f42ff25912b" Jul 02 21:38:50.829139 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.829116 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 02 21:38:50.833618 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.833590 2569 scope.go:117] "RemoveContainer" containerID="cb369aeea922de2f8d9c77fa6c887b92f4883d33d6f2301f4e264f0f3aa70eca" Jul 02 21:38:50.840772 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.840756 2569 scope.go:117] "RemoveContainer" containerID="4083ee39b13639009eff5623a7f398e9de4bfc682e0fd699a179934d5e6d1262" Jul 02 21:38:50.847036 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.847020 2569 scope.go:117] "RemoveContainer" containerID="562349de69ebe9323c0318788249f1143fe13487f597996c67f838e19777941f" Jul 02 21:38:50.847269 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:38:50.847251 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"562349de69ebe9323c0318788249f1143fe13487f597996c67f838e19777941f\": container with ID starting with 562349de69ebe9323c0318788249f1143fe13487f597996c67f838e19777941f not found: ID does not exist" containerID="562349de69ebe9323c0318788249f1143fe13487f597996c67f838e19777941f" Jul 02 21:38:50.847309 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.847277 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"562349de69ebe9323c0318788249f1143fe13487f597996c67f838e19777941f"} err="failed to get container status \"562349de69ebe9323c0318788249f1143fe13487f597996c67f838e19777941f\": rpc error: code = NotFound desc = could not find container \"562349de69ebe9323c0318788249f1143fe13487f597996c67f838e19777941f\": container with ID starting with 562349de69ebe9323c0318788249f1143fe13487f597996c67f838e19777941f not found: ID does not exist" Jul 02 21:38:50.847309 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.847297 2569 scope.go:117] "RemoveContainer" containerID="59679e4aba2a9a94b4412190386a648b7dd6e371c15fa6bb2884fd4d151976dc" Jul 02 21:38:50.847541 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:38:50.847519 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59679e4aba2a9a94b4412190386a648b7dd6e371c15fa6bb2884fd4d151976dc\": container with ID starting with 59679e4aba2a9a94b4412190386a648b7dd6e371c15fa6bb2884fd4d151976dc not found: ID does not exist" containerID="59679e4aba2a9a94b4412190386a648b7dd6e371c15fa6bb2884fd4d151976dc" Jul 02 21:38:50.847648 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.847545 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59679e4aba2a9a94b4412190386a648b7dd6e371c15fa6bb2884fd4d151976dc"} err="failed to get container status \"59679e4aba2a9a94b4412190386a648b7dd6e371c15fa6bb2884fd4d151976dc\": rpc error: code = NotFound desc = could not find container \"59679e4aba2a9a94b4412190386a648b7dd6e371c15fa6bb2884fd4d151976dc\": container with ID starting with 59679e4aba2a9a94b4412190386a648b7dd6e371c15fa6bb2884fd4d151976dc not found: ID does not exist" Jul 02 21:38:50.847648 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.847566 2569 scope.go:117] "RemoveContainer" containerID="ffb9c3a2757384933124acf4f890c186c55202281a29e944daa1e99108774f41" Jul 02 21:38:50.847923 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:38:50.847901 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ffb9c3a2757384933124acf4f890c186c55202281a29e944daa1e99108774f41\": container with ID starting with ffb9c3a2757384933124acf4f890c186c55202281a29e944daa1e99108774f41 not found: ID does not exist" containerID="ffb9c3a2757384933124acf4f890c186c55202281a29e944daa1e99108774f41" Jul 02 21:38:50.847999 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.847930 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffb9c3a2757384933124acf4f890c186c55202281a29e944daa1e99108774f41"} err="failed to get container status \"ffb9c3a2757384933124acf4f890c186c55202281a29e944daa1e99108774f41\": rpc error: code = NotFound desc = could not find container \"ffb9c3a2757384933124acf4f890c186c55202281a29e944daa1e99108774f41\": container with ID starting with ffb9c3a2757384933124acf4f890c186c55202281a29e944daa1e99108774f41 not found: ID does not exist" Jul 02 21:38:50.847999 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.847946 2569 scope.go:117] "RemoveContainer" containerID="994ffbd2a36fe6076996a91c08a2b3e3c3f8d007f6810bb03ca1c8bc9a4edf91" Jul 02 21:38:50.848212 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:38:50.848195 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"994ffbd2a36fe6076996a91c08a2b3e3c3f8d007f6810bb03ca1c8bc9a4edf91\": container with ID starting with 994ffbd2a36fe6076996a91c08a2b3e3c3f8d007f6810bb03ca1c8bc9a4edf91 not found: ID does not exist" containerID="994ffbd2a36fe6076996a91c08a2b3e3c3f8d007f6810bb03ca1c8bc9a4edf91" Jul 02 21:38:50.848278 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.848218 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"994ffbd2a36fe6076996a91c08a2b3e3c3f8d007f6810bb03ca1c8bc9a4edf91"} err="failed to get container status \"994ffbd2a36fe6076996a91c08a2b3e3c3f8d007f6810bb03ca1c8bc9a4edf91\": rpc error: code = NotFound desc = could not find container \"994ffbd2a36fe6076996a91c08a2b3e3c3f8d007f6810bb03ca1c8bc9a4edf91\": container with ID starting with 994ffbd2a36fe6076996a91c08a2b3e3c3f8d007f6810bb03ca1c8bc9a4edf91 not found: ID does not exist" Jul 02 21:38:50.848278 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.848233 2569 scope.go:117] "RemoveContainer" containerID="c7729839da28689f661acfd091f3a98790de2ff973e0c72992773f42ff25912b" Jul 02 21:38:50.848452 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:38:50.848432 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7729839da28689f661acfd091f3a98790de2ff973e0c72992773f42ff25912b\": container with ID starting with c7729839da28689f661acfd091f3a98790de2ff973e0c72992773f42ff25912b not found: ID does not exist" containerID="c7729839da28689f661acfd091f3a98790de2ff973e0c72992773f42ff25912b" Jul 02 21:38:50.848488 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.848458 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7729839da28689f661acfd091f3a98790de2ff973e0c72992773f42ff25912b"} err="failed to get container status \"c7729839da28689f661acfd091f3a98790de2ff973e0c72992773f42ff25912b\": rpc error: code = NotFound desc = could not find container \"c7729839da28689f661acfd091f3a98790de2ff973e0c72992773f42ff25912b\": container with ID starting with c7729839da28689f661acfd091f3a98790de2ff973e0c72992773f42ff25912b not found: ID does not exist" Jul 02 21:38:50.848488 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.848476 2569 scope.go:117] "RemoveContainer" containerID="cb369aeea922de2f8d9c77fa6c887b92f4883d33d6f2301f4e264f0f3aa70eca" Jul 02 21:38:50.848693 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:38:50.848678 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb369aeea922de2f8d9c77fa6c887b92f4883d33d6f2301f4e264f0f3aa70eca\": container with ID starting with cb369aeea922de2f8d9c77fa6c887b92f4883d33d6f2301f4e264f0f3aa70eca not found: ID does not exist" containerID="cb369aeea922de2f8d9c77fa6c887b92f4883d33d6f2301f4e264f0f3aa70eca" Jul 02 21:38:50.848735 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.848696 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb369aeea922de2f8d9c77fa6c887b92f4883d33d6f2301f4e264f0f3aa70eca"} err="failed to get container status \"cb369aeea922de2f8d9c77fa6c887b92f4883d33d6f2301f4e264f0f3aa70eca\": rpc error: code = NotFound desc = could not find container \"cb369aeea922de2f8d9c77fa6c887b92f4883d33d6f2301f4e264f0f3aa70eca\": container with ID starting with cb369aeea922de2f8d9c77fa6c887b92f4883d33d6f2301f4e264f0f3aa70eca not found: ID does not exist" Jul 02 21:38:50.848735 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.848717 2569 scope.go:117] "RemoveContainer" containerID="4083ee39b13639009eff5623a7f398e9de4bfc682e0fd699a179934d5e6d1262" Jul 02 21:38:50.849040 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:38:50.849024 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4083ee39b13639009eff5623a7f398e9de4bfc682e0fd699a179934d5e6d1262\": container with ID starting with 4083ee39b13639009eff5623a7f398e9de4bfc682e0fd699a179934d5e6d1262 not found: ID does not exist" containerID="4083ee39b13639009eff5623a7f398e9de4bfc682e0fd699a179934d5e6d1262" Jul 02 21:38:50.849108 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.849046 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4083ee39b13639009eff5623a7f398e9de4bfc682e0fd699a179934d5e6d1262"} err="failed to get container status \"4083ee39b13639009eff5623a7f398e9de4bfc682e0fd699a179934d5e6d1262\": rpc error: code = NotFound desc = could not find container \"4083ee39b13639009eff5623a7f398e9de4bfc682e0fd699a179934d5e6d1262\": container with ID starting with 4083ee39b13639009eff5623a7f398e9de4bfc682e0fd699a179934d5e6d1262 not found: ID does not exist" Jul 02 21:38:50.849108 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.849066 2569 scope.go:117] "RemoveContainer" containerID="562349de69ebe9323c0318788249f1143fe13487f597996c67f838e19777941f" Jul 02 21:38:50.849304 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.849288 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"562349de69ebe9323c0318788249f1143fe13487f597996c67f838e19777941f"} err="failed to get container status \"562349de69ebe9323c0318788249f1143fe13487f597996c67f838e19777941f\": rpc error: code = NotFound desc = could not find container \"562349de69ebe9323c0318788249f1143fe13487f597996c67f838e19777941f\": container with ID starting with 562349de69ebe9323c0318788249f1143fe13487f597996c67f838e19777941f not found: ID does not exist" Jul 02 21:38:50.849348 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.849305 2569 scope.go:117] "RemoveContainer" containerID="59679e4aba2a9a94b4412190386a648b7dd6e371c15fa6bb2884fd4d151976dc" Jul 02 21:38:50.849517 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.849500 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59679e4aba2a9a94b4412190386a648b7dd6e371c15fa6bb2884fd4d151976dc"} err="failed to get container status \"59679e4aba2a9a94b4412190386a648b7dd6e371c15fa6bb2884fd4d151976dc\": rpc error: code = NotFound desc = could not find container \"59679e4aba2a9a94b4412190386a648b7dd6e371c15fa6bb2884fd4d151976dc\": container with ID starting with 59679e4aba2a9a94b4412190386a648b7dd6e371c15fa6bb2884fd4d151976dc not found: ID does not exist" Jul 02 21:38:50.849559 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.849517 2569 scope.go:117] "RemoveContainer" containerID="ffb9c3a2757384933124acf4f890c186c55202281a29e944daa1e99108774f41" Jul 02 21:38:50.849724 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.849707 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffb9c3a2757384933124acf4f890c186c55202281a29e944daa1e99108774f41"} err="failed to get container status \"ffb9c3a2757384933124acf4f890c186c55202281a29e944daa1e99108774f41\": rpc error: code = NotFound desc = could not find container \"ffb9c3a2757384933124acf4f890c186c55202281a29e944daa1e99108774f41\": container with ID starting with ffb9c3a2757384933124acf4f890c186c55202281a29e944daa1e99108774f41 not found: ID does not exist" Jul 02 21:38:50.849809 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.849725 2569 scope.go:117] "RemoveContainer" containerID="994ffbd2a36fe6076996a91c08a2b3e3c3f8d007f6810bb03ca1c8bc9a4edf91" Jul 02 21:38:50.849975 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.849958 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"994ffbd2a36fe6076996a91c08a2b3e3c3f8d007f6810bb03ca1c8bc9a4edf91"} err="failed to get container status \"994ffbd2a36fe6076996a91c08a2b3e3c3f8d007f6810bb03ca1c8bc9a4edf91\": rpc error: code = NotFound desc = could not find container \"994ffbd2a36fe6076996a91c08a2b3e3c3f8d007f6810bb03ca1c8bc9a4edf91\": container with ID starting with 994ffbd2a36fe6076996a91c08a2b3e3c3f8d007f6810bb03ca1c8bc9a4edf91 not found: ID does not exist" Jul 02 21:38:50.850042 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.849977 2569 scope.go:117] "RemoveContainer" containerID="c7729839da28689f661acfd091f3a98790de2ff973e0c72992773f42ff25912b" Jul 02 21:38:50.850194 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.850178 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7729839da28689f661acfd091f3a98790de2ff973e0c72992773f42ff25912b"} err="failed to get container status \"c7729839da28689f661acfd091f3a98790de2ff973e0c72992773f42ff25912b\": rpc error: code = NotFound desc = could not find container \"c7729839da28689f661acfd091f3a98790de2ff973e0c72992773f42ff25912b\": container with ID starting with c7729839da28689f661acfd091f3a98790de2ff973e0c72992773f42ff25912b not found: ID does not exist" Jul 02 21:38:50.850252 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.850195 2569 scope.go:117] "RemoveContainer" containerID="cb369aeea922de2f8d9c77fa6c887b92f4883d33d6f2301f4e264f0f3aa70eca" Jul 02 21:38:50.850410 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.850393 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb369aeea922de2f8d9c77fa6c887b92f4883d33d6f2301f4e264f0f3aa70eca"} err="failed to get container status \"cb369aeea922de2f8d9c77fa6c887b92f4883d33d6f2301f4e264f0f3aa70eca\": rpc error: code = NotFound desc = could not find container \"cb369aeea922de2f8d9c77fa6c887b92f4883d33d6f2301f4e264f0f3aa70eca\": container with ID starting with cb369aeea922de2f8d9c77fa6c887b92f4883d33d6f2301f4e264f0f3aa70eca not found: ID does not exist" Jul 02 21:38:50.850459 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.850413 2569 scope.go:117] "RemoveContainer" containerID="4083ee39b13639009eff5623a7f398e9de4bfc682e0fd699a179934d5e6d1262" Jul 02 21:38:50.850618 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.850599 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4083ee39b13639009eff5623a7f398e9de4bfc682e0fd699a179934d5e6d1262"} err="failed to get container status \"4083ee39b13639009eff5623a7f398e9de4bfc682e0fd699a179934d5e6d1262\": rpc error: code = NotFound desc = could not find container \"4083ee39b13639009eff5623a7f398e9de4bfc682e0fd699a179934d5e6d1262\": container with ID starting with 4083ee39b13639009eff5623a7f398e9de4bfc682e0fd699a179934d5e6d1262 not found: ID does not exist" Jul 02 21:38:50.850659 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.850619 2569 scope.go:117] "RemoveContainer" containerID="562349de69ebe9323c0318788249f1143fe13487f597996c67f838e19777941f" Jul 02 21:38:50.850856 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.850838 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"562349de69ebe9323c0318788249f1143fe13487f597996c67f838e19777941f"} err="failed to get container status \"562349de69ebe9323c0318788249f1143fe13487f597996c67f838e19777941f\": rpc error: code = NotFound desc = could not find container \"562349de69ebe9323c0318788249f1143fe13487f597996c67f838e19777941f\": container with ID starting with 562349de69ebe9323c0318788249f1143fe13487f597996c67f838e19777941f not found: ID does not exist" Jul 02 21:38:50.850934 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.850858 2569 scope.go:117] "RemoveContainer" containerID="59679e4aba2a9a94b4412190386a648b7dd6e371c15fa6bb2884fd4d151976dc" Jul 02 21:38:50.851058 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.851042 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59679e4aba2a9a94b4412190386a648b7dd6e371c15fa6bb2884fd4d151976dc"} err="failed to get container status \"59679e4aba2a9a94b4412190386a648b7dd6e371c15fa6bb2884fd4d151976dc\": rpc error: code = NotFound desc = could not find container \"59679e4aba2a9a94b4412190386a648b7dd6e371c15fa6bb2884fd4d151976dc\": container with ID starting with 59679e4aba2a9a94b4412190386a648b7dd6e371c15fa6bb2884fd4d151976dc not found: ID does not exist" Jul 02 21:38:50.851123 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.851059 2569 scope.go:117] "RemoveContainer" containerID="ffb9c3a2757384933124acf4f890c186c55202281a29e944daa1e99108774f41" Jul 02 21:38:50.851261 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.851246 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffb9c3a2757384933124acf4f890c186c55202281a29e944daa1e99108774f41"} err="failed to get container status \"ffb9c3a2757384933124acf4f890c186c55202281a29e944daa1e99108774f41\": rpc error: code = NotFound desc = could not find container \"ffb9c3a2757384933124acf4f890c186c55202281a29e944daa1e99108774f41\": container with ID starting with ffb9c3a2757384933124acf4f890c186c55202281a29e944daa1e99108774f41 not found: ID does not exist" Jul 02 21:38:50.851320 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.851262 2569 scope.go:117] "RemoveContainer" containerID="994ffbd2a36fe6076996a91c08a2b3e3c3f8d007f6810bb03ca1c8bc9a4edf91" Jul 02 21:38:50.851460 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.851444 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"994ffbd2a36fe6076996a91c08a2b3e3c3f8d007f6810bb03ca1c8bc9a4edf91"} err="failed to get container status \"994ffbd2a36fe6076996a91c08a2b3e3c3f8d007f6810bb03ca1c8bc9a4edf91\": rpc error: code = NotFound desc = could not find container \"994ffbd2a36fe6076996a91c08a2b3e3c3f8d007f6810bb03ca1c8bc9a4edf91\": container with ID starting with 994ffbd2a36fe6076996a91c08a2b3e3c3f8d007f6810bb03ca1c8bc9a4edf91 not found: ID does not exist" Jul 02 21:38:50.851525 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.851462 2569 scope.go:117] "RemoveContainer" containerID="c7729839da28689f661acfd091f3a98790de2ff973e0c72992773f42ff25912b" Jul 02 21:38:50.851649 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.851633 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7729839da28689f661acfd091f3a98790de2ff973e0c72992773f42ff25912b"} err="failed to get container status \"c7729839da28689f661acfd091f3a98790de2ff973e0c72992773f42ff25912b\": rpc error: code = NotFound desc = could not find container \"c7729839da28689f661acfd091f3a98790de2ff973e0c72992773f42ff25912b\": container with ID starting with c7729839da28689f661acfd091f3a98790de2ff973e0c72992773f42ff25912b not found: ID does not exist" Jul 02 21:38:50.851712 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.851650 2569 scope.go:117] "RemoveContainer" containerID="cb369aeea922de2f8d9c77fa6c887b92f4883d33d6f2301f4e264f0f3aa70eca" Jul 02 21:38:50.851929 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.851908 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb369aeea922de2f8d9c77fa6c887b92f4883d33d6f2301f4e264f0f3aa70eca"} err="failed to get container status \"cb369aeea922de2f8d9c77fa6c887b92f4883d33d6f2301f4e264f0f3aa70eca\": rpc error: code = NotFound desc = could not find container \"cb369aeea922de2f8d9c77fa6c887b92f4883d33d6f2301f4e264f0f3aa70eca\": container with ID starting with cb369aeea922de2f8d9c77fa6c887b92f4883d33d6f2301f4e264f0f3aa70eca not found: ID does not exist" Jul 02 21:38:50.851990 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.851929 2569 scope.go:117] "RemoveContainer" containerID="4083ee39b13639009eff5623a7f398e9de4bfc682e0fd699a179934d5e6d1262" Jul 02 21:38:50.852127 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.852110 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4083ee39b13639009eff5623a7f398e9de4bfc682e0fd699a179934d5e6d1262"} err="failed to get container status \"4083ee39b13639009eff5623a7f398e9de4bfc682e0fd699a179934d5e6d1262\": rpc error: code = NotFound desc = could not find container \"4083ee39b13639009eff5623a7f398e9de4bfc682e0fd699a179934d5e6d1262\": container with ID starting with 4083ee39b13639009eff5623a7f398e9de4bfc682e0fd699a179934d5e6d1262 not found: ID does not exist" Jul 02 21:38:50.852177 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.852127 2569 scope.go:117] "RemoveContainer" containerID="562349de69ebe9323c0318788249f1143fe13487f597996c67f838e19777941f" Jul 02 21:38:50.852298 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.852284 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"562349de69ebe9323c0318788249f1143fe13487f597996c67f838e19777941f"} err="failed to get container status \"562349de69ebe9323c0318788249f1143fe13487f597996c67f838e19777941f\": rpc error: code = NotFound desc = could not find container \"562349de69ebe9323c0318788249f1143fe13487f597996c67f838e19777941f\": container with ID starting with 562349de69ebe9323c0318788249f1143fe13487f597996c67f838e19777941f not found: ID does not exist" Jul 02 21:38:50.852338 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.852298 2569 scope.go:117] "RemoveContainer" containerID="59679e4aba2a9a94b4412190386a648b7dd6e371c15fa6bb2884fd4d151976dc" Jul 02 21:38:50.852505 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.852487 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59679e4aba2a9a94b4412190386a648b7dd6e371c15fa6bb2884fd4d151976dc"} err="failed to get container status \"59679e4aba2a9a94b4412190386a648b7dd6e371c15fa6bb2884fd4d151976dc\": rpc error: code = NotFound desc = could not find container \"59679e4aba2a9a94b4412190386a648b7dd6e371c15fa6bb2884fd4d151976dc\": container with ID starting with 59679e4aba2a9a94b4412190386a648b7dd6e371c15fa6bb2884fd4d151976dc not found: ID does not exist" Jul 02 21:38:50.852570 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.852506 2569 scope.go:117] "RemoveContainer" containerID="ffb9c3a2757384933124acf4f890c186c55202281a29e944daa1e99108774f41" Jul 02 21:38:50.852708 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.852693 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffb9c3a2757384933124acf4f890c186c55202281a29e944daa1e99108774f41"} err="failed to get container status \"ffb9c3a2757384933124acf4f890c186c55202281a29e944daa1e99108774f41\": rpc error: code = NotFound desc = could not find container \"ffb9c3a2757384933124acf4f890c186c55202281a29e944daa1e99108774f41\": container with ID starting with ffb9c3a2757384933124acf4f890c186c55202281a29e944daa1e99108774f41 not found: ID does not exist" Jul 02 21:38:50.852772 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.852710 2569 scope.go:117] "RemoveContainer" containerID="994ffbd2a36fe6076996a91c08a2b3e3c3f8d007f6810bb03ca1c8bc9a4edf91" Jul 02 21:38:50.852938 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.852920 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"994ffbd2a36fe6076996a91c08a2b3e3c3f8d007f6810bb03ca1c8bc9a4edf91"} err="failed to get container status \"994ffbd2a36fe6076996a91c08a2b3e3c3f8d007f6810bb03ca1c8bc9a4edf91\": rpc error: code = NotFound desc = could not find container \"994ffbd2a36fe6076996a91c08a2b3e3c3f8d007f6810bb03ca1c8bc9a4edf91\": container with ID starting with 994ffbd2a36fe6076996a91c08a2b3e3c3f8d007f6810bb03ca1c8bc9a4edf91 not found: ID does not exist" Jul 02 21:38:50.852938 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.852937 2569 scope.go:117] "RemoveContainer" containerID="c7729839da28689f661acfd091f3a98790de2ff973e0c72992773f42ff25912b" Jul 02 21:38:50.853107 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.853090 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7729839da28689f661acfd091f3a98790de2ff973e0c72992773f42ff25912b"} err="failed to get container status \"c7729839da28689f661acfd091f3a98790de2ff973e0c72992773f42ff25912b\": rpc error: code = NotFound desc = could not find container \"c7729839da28689f661acfd091f3a98790de2ff973e0c72992773f42ff25912b\": container with ID starting with c7729839da28689f661acfd091f3a98790de2ff973e0c72992773f42ff25912b not found: ID does not exist" Jul 02 21:38:50.853154 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.853108 2569 scope.go:117] "RemoveContainer" containerID="cb369aeea922de2f8d9c77fa6c887b92f4883d33d6f2301f4e264f0f3aa70eca" Jul 02 21:38:50.853334 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.853317 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb369aeea922de2f8d9c77fa6c887b92f4883d33d6f2301f4e264f0f3aa70eca"} err="failed to get container status \"cb369aeea922de2f8d9c77fa6c887b92f4883d33d6f2301f4e264f0f3aa70eca\": rpc error: code = NotFound desc = could not find container \"cb369aeea922de2f8d9c77fa6c887b92f4883d33d6f2301f4e264f0f3aa70eca\": container with ID starting with cb369aeea922de2f8d9c77fa6c887b92f4883d33d6f2301f4e264f0f3aa70eca not found: ID does not exist" Jul 02 21:38:50.853379 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.853335 2569 scope.go:117] "RemoveContainer" containerID="4083ee39b13639009eff5623a7f398e9de4bfc682e0fd699a179934d5e6d1262" Jul 02 21:38:50.853535 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.853521 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4083ee39b13639009eff5623a7f398e9de4bfc682e0fd699a179934d5e6d1262"} err="failed to get container status \"4083ee39b13639009eff5623a7f398e9de4bfc682e0fd699a179934d5e6d1262\": rpc error: code = NotFound desc = could not find container \"4083ee39b13639009eff5623a7f398e9de4bfc682e0fd699a179934d5e6d1262\": container with ID starting with 4083ee39b13639009eff5623a7f398e9de4bfc682e0fd699a179934d5e6d1262 not found: ID does not exist" Jul 02 21:38:50.853571 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.853536 2569 scope.go:117] "RemoveContainer" containerID="562349de69ebe9323c0318788249f1143fe13487f597996c67f838e19777941f" Jul 02 21:38:50.853735 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.853719 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"562349de69ebe9323c0318788249f1143fe13487f597996c67f838e19777941f"} err="failed to get container status \"562349de69ebe9323c0318788249f1143fe13487f597996c67f838e19777941f\": rpc error: code = NotFound desc = could not find container \"562349de69ebe9323c0318788249f1143fe13487f597996c67f838e19777941f\": container with ID starting with 562349de69ebe9323c0318788249f1143fe13487f597996c67f838e19777941f not found: ID does not exist" Jul 02 21:38:50.853774 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.853736 2569 scope.go:117] "RemoveContainer" containerID="59679e4aba2a9a94b4412190386a648b7dd6e371c15fa6bb2884fd4d151976dc" Jul 02 21:38:50.853958 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.853938 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59679e4aba2a9a94b4412190386a648b7dd6e371c15fa6bb2884fd4d151976dc"} err="failed to get container status \"59679e4aba2a9a94b4412190386a648b7dd6e371c15fa6bb2884fd4d151976dc\": rpc error: code = NotFound desc = could not find container \"59679e4aba2a9a94b4412190386a648b7dd6e371c15fa6bb2884fd4d151976dc\": container with ID starting with 59679e4aba2a9a94b4412190386a648b7dd6e371c15fa6bb2884fd4d151976dc not found: ID does not exist" Jul 02 21:38:50.854003 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.853960 2569 scope.go:117] "RemoveContainer" containerID="ffb9c3a2757384933124acf4f890c186c55202281a29e944daa1e99108774f41" Jul 02 21:38:50.854164 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.854149 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffb9c3a2757384933124acf4f890c186c55202281a29e944daa1e99108774f41"} err="failed to get container status \"ffb9c3a2757384933124acf4f890c186c55202281a29e944daa1e99108774f41\": rpc error: code = NotFound desc = could not find container \"ffb9c3a2757384933124acf4f890c186c55202281a29e944daa1e99108774f41\": container with ID starting with ffb9c3a2757384933124acf4f890c186c55202281a29e944daa1e99108774f41 not found: ID does not exist" Jul 02 21:38:50.854213 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.854166 2569 scope.go:117] "RemoveContainer" containerID="994ffbd2a36fe6076996a91c08a2b3e3c3f8d007f6810bb03ca1c8bc9a4edf91" Jul 02 21:38:50.854340 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.854322 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"994ffbd2a36fe6076996a91c08a2b3e3c3f8d007f6810bb03ca1c8bc9a4edf91"} err="failed to get container status \"994ffbd2a36fe6076996a91c08a2b3e3c3f8d007f6810bb03ca1c8bc9a4edf91\": rpc error: code = NotFound desc = could not find container \"994ffbd2a36fe6076996a91c08a2b3e3c3f8d007f6810bb03ca1c8bc9a4edf91\": container with ID starting with 994ffbd2a36fe6076996a91c08a2b3e3c3f8d007f6810bb03ca1c8bc9a4edf91 not found: ID does not exist" Jul 02 21:38:50.854381 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.854339 2569 scope.go:117] "RemoveContainer" containerID="c7729839da28689f661acfd091f3a98790de2ff973e0c72992773f42ff25912b" Jul 02 21:38:50.854506 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.854489 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7729839da28689f661acfd091f3a98790de2ff973e0c72992773f42ff25912b"} err="failed to get container status \"c7729839da28689f661acfd091f3a98790de2ff973e0c72992773f42ff25912b\": rpc error: code = NotFound desc = could not find container \"c7729839da28689f661acfd091f3a98790de2ff973e0c72992773f42ff25912b\": container with ID starting with c7729839da28689f661acfd091f3a98790de2ff973e0c72992773f42ff25912b not found: ID does not exist" Jul 02 21:38:50.854545 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.854508 2569 scope.go:117] "RemoveContainer" containerID="cb369aeea922de2f8d9c77fa6c887b92f4883d33d6f2301f4e264f0f3aa70eca" Jul 02 21:38:50.854680 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.854664 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb369aeea922de2f8d9c77fa6c887b92f4883d33d6f2301f4e264f0f3aa70eca"} err="failed to get container status \"cb369aeea922de2f8d9c77fa6c887b92f4883d33d6f2301f4e264f0f3aa70eca\": rpc error: code = NotFound desc = could not find container \"cb369aeea922de2f8d9c77fa6c887b92f4883d33d6f2301f4e264f0f3aa70eca\": container with ID starting with cb369aeea922de2f8d9c77fa6c887b92f4883d33d6f2301f4e264f0f3aa70eca not found: ID does not exist" Jul 02 21:38:50.854719 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.854681 2569 scope.go:117] "RemoveContainer" containerID="4083ee39b13639009eff5623a7f398e9de4bfc682e0fd699a179934d5e6d1262" Jul 02 21:38:50.854897 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.854884 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4083ee39b13639009eff5623a7f398e9de4bfc682e0fd699a179934d5e6d1262"} err="failed to get container status \"4083ee39b13639009eff5623a7f398e9de4bfc682e0fd699a179934d5e6d1262\": rpc error: code = NotFound desc = could not find container \"4083ee39b13639009eff5623a7f398e9de4bfc682e0fd699a179934d5e6d1262\": container with ID starting with 4083ee39b13639009eff5623a7f398e9de4bfc682e0fd699a179934d5e6d1262 not found: ID does not exist" Jul 02 21:38:50.854942 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.854897 2569 scope.go:117] "RemoveContainer" containerID="562349de69ebe9323c0318788249f1143fe13487f597996c67f838e19777941f" Jul 02 21:38:50.855091 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.855076 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"562349de69ebe9323c0318788249f1143fe13487f597996c67f838e19777941f"} err="failed to get container status \"562349de69ebe9323c0318788249f1143fe13487f597996c67f838e19777941f\": rpc error: code = NotFound desc = could not find container \"562349de69ebe9323c0318788249f1143fe13487f597996c67f838e19777941f\": container with ID starting with 562349de69ebe9323c0318788249f1143fe13487f597996c67f838e19777941f not found: ID does not exist" Jul 02 21:38:50.855130 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.855091 2569 scope.go:117] "RemoveContainer" containerID="59679e4aba2a9a94b4412190386a648b7dd6e371c15fa6bb2884fd4d151976dc" Jul 02 21:38:50.855271 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.855252 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59679e4aba2a9a94b4412190386a648b7dd6e371c15fa6bb2884fd4d151976dc"} err="failed to get container status \"59679e4aba2a9a94b4412190386a648b7dd6e371c15fa6bb2884fd4d151976dc\": rpc error: code = NotFound desc = could not find container \"59679e4aba2a9a94b4412190386a648b7dd6e371c15fa6bb2884fd4d151976dc\": container with ID starting with 59679e4aba2a9a94b4412190386a648b7dd6e371c15fa6bb2884fd4d151976dc not found: ID does not exist" Jul 02 21:38:50.855310 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.855271 2569 scope.go:117] "RemoveContainer" containerID="ffb9c3a2757384933124acf4f890c186c55202281a29e944daa1e99108774f41" Jul 02 21:38:50.855508 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.855484 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffb9c3a2757384933124acf4f890c186c55202281a29e944daa1e99108774f41"} err="failed to get container status \"ffb9c3a2757384933124acf4f890c186c55202281a29e944daa1e99108774f41\": rpc error: code = NotFound desc = could not find container \"ffb9c3a2757384933124acf4f890c186c55202281a29e944daa1e99108774f41\": container with ID starting with ffb9c3a2757384933124acf4f890c186c55202281a29e944daa1e99108774f41 not found: ID does not exist" Jul 02 21:38:50.855508 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.855507 2569 scope.go:117] "RemoveContainer" containerID="994ffbd2a36fe6076996a91c08a2b3e3c3f8d007f6810bb03ca1c8bc9a4edf91" Jul 02 21:38:50.855837 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.855816 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"994ffbd2a36fe6076996a91c08a2b3e3c3f8d007f6810bb03ca1c8bc9a4edf91"} err="failed to get container status \"994ffbd2a36fe6076996a91c08a2b3e3c3f8d007f6810bb03ca1c8bc9a4edf91\": rpc error: code = NotFound desc = could not find container \"994ffbd2a36fe6076996a91c08a2b3e3c3f8d007f6810bb03ca1c8bc9a4edf91\": container with ID starting with 994ffbd2a36fe6076996a91c08a2b3e3c3f8d007f6810bb03ca1c8bc9a4edf91 not found: ID does not exist" Jul 02 21:38:50.855837 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.855838 2569 scope.go:117] "RemoveContainer" containerID="c7729839da28689f661acfd091f3a98790de2ff973e0c72992773f42ff25912b" Jul 02 21:38:50.856120 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.856078 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7729839da28689f661acfd091f3a98790de2ff973e0c72992773f42ff25912b"} err="failed to get container status \"c7729839da28689f661acfd091f3a98790de2ff973e0c72992773f42ff25912b\": rpc error: code = NotFound desc = could not find container \"c7729839da28689f661acfd091f3a98790de2ff973e0c72992773f42ff25912b\": container with ID starting with c7729839da28689f661acfd091f3a98790de2ff973e0c72992773f42ff25912b not found: ID does not exist" Jul 02 21:38:50.856213 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.856121 2569 scope.go:117] "RemoveContainer" containerID="cb369aeea922de2f8d9c77fa6c887b92f4883d33d6f2301f4e264f0f3aa70eca" Jul 02 21:38:50.856379 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.856362 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb369aeea922de2f8d9c77fa6c887b92f4883d33d6f2301f4e264f0f3aa70eca"} err="failed to get container status \"cb369aeea922de2f8d9c77fa6c887b92f4883d33d6f2301f4e264f0f3aa70eca\": rpc error: code = NotFound desc = could not find container \"cb369aeea922de2f8d9c77fa6c887b92f4883d33d6f2301f4e264f0f3aa70eca\": container with ID starting with cb369aeea922de2f8d9c77fa6c887b92f4883d33d6f2301f4e264f0f3aa70eca not found: ID does not exist" Jul 02 21:38:50.856447 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.856381 2569 scope.go:117] "RemoveContainer" containerID="4083ee39b13639009eff5623a7f398e9de4bfc682e0fd699a179934d5e6d1262" Jul 02 21:38:50.856650 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.856631 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4083ee39b13639009eff5623a7f398e9de4bfc682e0fd699a179934d5e6d1262"} err="failed to get container status \"4083ee39b13639009eff5623a7f398e9de4bfc682e0fd699a179934d5e6d1262\": rpc error: code = NotFound desc = could not find container \"4083ee39b13639009eff5623a7f398e9de4bfc682e0fd699a179934d5e6d1262\": container with ID starting with 4083ee39b13639009eff5623a7f398e9de4bfc682e0fd699a179934d5e6d1262 not found: ID does not exist" Jul 02 21:38:50.857365 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.857345 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 02 21:38:50.857677 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.857661 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab3c0820-2dc6-402b-b845-eae86ad75d46" containerName="thanos-sidecar" Jul 02 21:38:50.857747 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.857679 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab3c0820-2dc6-402b-b845-eae86ad75d46" containerName="thanos-sidecar" Jul 02 21:38:50.857747 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.857692 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab3c0820-2dc6-402b-b845-eae86ad75d46" containerName="kube-rbac-proxy-web" Jul 02 21:38:50.857747 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.857701 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab3c0820-2dc6-402b-b845-eae86ad75d46" containerName="kube-rbac-proxy-web" Jul 02 21:38:50.857747 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.857711 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab3c0820-2dc6-402b-b845-eae86ad75d46" containerName="kube-rbac-proxy-thanos" Jul 02 21:38:50.857747 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.857720 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab3c0820-2dc6-402b-b845-eae86ad75d46" containerName="kube-rbac-proxy-thanos" Jul 02 21:38:50.857747 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.857738 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab3c0820-2dc6-402b-b845-eae86ad75d46" containerName="prometheus" Jul 02 21:38:50.857747 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.857746 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab3c0820-2dc6-402b-b845-eae86ad75d46" containerName="prometheus" Jul 02 21:38:50.858117 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.857755 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab3c0820-2dc6-402b-b845-eae86ad75d46" containerName="config-reloader" Jul 02 21:38:50.858117 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.857763 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab3c0820-2dc6-402b-b845-eae86ad75d46" containerName="config-reloader" Jul 02 21:38:50.858117 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.857780 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab3c0820-2dc6-402b-b845-eae86ad75d46" containerName="kube-rbac-proxy" Jul 02 21:38:50.858117 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.857815 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab3c0820-2dc6-402b-b845-eae86ad75d46" containerName="kube-rbac-proxy" Jul 02 21:38:50.858117 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.857826 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab3c0820-2dc6-402b-b845-eae86ad75d46" containerName="init-config-reloader" Jul 02 21:38:50.858117 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.857835 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab3c0820-2dc6-402b-b845-eae86ad75d46" containerName="init-config-reloader" Jul 02 21:38:50.858117 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.857847 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2dc606a9-6a0c-49de-be3e-4c090d343d3e" containerName="registry" Jul 02 21:38:50.858117 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.857855 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="2dc606a9-6a0c-49de-be3e-4c090d343d3e" containerName="registry" Jul 02 21:38:50.858117 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.857971 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab3c0820-2dc6-402b-b845-eae86ad75d46" containerName="kube-rbac-proxy-thanos" Jul 02 21:38:50.858117 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.857984 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab3c0820-2dc6-402b-b845-eae86ad75d46" containerName="kube-rbac-proxy" Jul 02 21:38:50.858117 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.857992 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab3c0820-2dc6-402b-b845-eae86ad75d46" containerName="prometheus" Jul 02 21:38:50.858117 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.858001 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="2dc606a9-6a0c-49de-be3e-4c090d343d3e" containerName="registry" Jul 02 21:38:50.858117 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.858011 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab3c0820-2dc6-402b-b845-eae86ad75d46" containerName="config-reloader" Jul 02 21:38:50.858117 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.858019 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab3c0820-2dc6-402b-b845-eae86ad75d46" containerName="kube-rbac-proxy-web" Jul 02 21:38:50.858117 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.858026 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab3c0820-2dc6-402b-b845-eae86ad75d46" containerName="thanos-sidecar" Jul 02 21:38:50.861698 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.861681 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:50.864590 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.864573 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-rb88p\"" Jul 02 21:38:50.864709 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.864657 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 02 21:38:50.864850 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.864830 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 02 21:38:50.864850 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.864835 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 02 21:38:50.865051 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.864920 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 02 21:38:50.865051 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.865042 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 02 21:38:50.865168 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.865149 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 02 21:38:50.865244 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.865229 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 02 21:38:50.865301 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.865227 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 02 21:38:50.865423 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.865355 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-91qhd5buaeu34\"" Jul 02 21:38:50.865474 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.865440 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 02 21:38:50.865648 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.865635 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 02 21:38:50.866479 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.866465 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 02 21:38:50.866685 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.866668 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 02 21:38:50.866806 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.866676 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 02 21:38:50.871615 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.871588 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 02 21:38:50.873312 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.873296 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 02 21:38:50.879962 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.879943 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 02 21:38:50.948900 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.948859 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/066af47b-7dcb-49e6-b963-4cf7576bce1f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:50.948900 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.948904 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/066af47b-7dcb-49e6-b963-4cf7576bce1f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:50.949092 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.948923 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/066af47b-7dcb-49e6-b963-4cf7576bce1f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:50.949092 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.948998 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/066af47b-7dcb-49e6-b963-4cf7576bce1f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:50.949092 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.949034 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/066af47b-7dcb-49e6-b963-4cf7576bce1f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:50.949187 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.949103 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/066af47b-7dcb-49e6-b963-4cf7576bce1f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:50.949187 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.949130 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/066af47b-7dcb-49e6-b963-4cf7576bce1f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:50.949187 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.949149 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-788hs\" (UniqueName: \"kubernetes.io/projected/066af47b-7dcb-49e6-b963-4cf7576bce1f-kube-api-access-788hs\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:50.949187 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.949173 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/066af47b-7dcb-49e6-b963-4cf7576bce1f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:50.949299 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.949194 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/066af47b-7dcb-49e6-b963-4cf7576bce1f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:50.949299 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.949252 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/066af47b-7dcb-49e6-b963-4cf7576bce1f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:50.949299 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.949279 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/066af47b-7dcb-49e6-b963-4cf7576bce1f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:50.949391 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.949300 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/066af47b-7dcb-49e6-b963-4cf7576bce1f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:50.949391 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.949344 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/066af47b-7dcb-49e6-b963-4cf7576bce1f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:50.949391 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.949366 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/066af47b-7dcb-49e6-b963-4cf7576bce1f-config\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:50.949391 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.949381 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/066af47b-7dcb-49e6-b963-4cf7576bce1f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:50.949506 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.949404 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/066af47b-7dcb-49e6-b963-4cf7576bce1f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:50.949506 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.949449 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/066af47b-7dcb-49e6-b963-4cf7576bce1f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:50.949506 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.949471 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/066af47b-7dcb-49e6-b963-4cf7576bce1f-config-out\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:50.949506 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.949492 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/066af47b-7dcb-49e6-b963-4cf7576bce1f-web-config\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:50.954945 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:50.954924 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab3c0820-2dc6-402b-b845-eae86ad75d46" path="/var/lib/kubelet/pods/ab3c0820-2dc6-402b-b845-eae86ad75d46/volumes" Jul 02 21:38:51.050750 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:51.050648 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/066af47b-7dcb-49e6-b963-4cf7576bce1f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:51.050750 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:51.050708 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/066af47b-7dcb-49e6-b963-4cf7576bce1f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:51.050750 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:51.050738 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/066af47b-7dcb-49e6-b963-4cf7576bce1f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:51.051324 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:51.050762 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/066af47b-7dcb-49e6-b963-4cf7576bce1f-config\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:51.051324 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:51.050810 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/066af47b-7dcb-49e6-b963-4cf7576bce1f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:51.051324 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:51.050835 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/066af47b-7dcb-49e6-b963-4cf7576bce1f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:51.051324 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:51.050883 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/066af47b-7dcb-49e6-b963-4cf7576bce1f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:51.051324 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:51.050908 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/066af47b-7dcb-49e6-b963-4cf7576bce1f-config-out\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:51.051324 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:51.050938 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/066af47b-7dcb-49e6-b963-4cf7576bce1f-web-config\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:51.051324 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:51.050971 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/066af47b-7dcb-49e6-b963-4cf7576bce1f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:51.051324 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:51.050996 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/066af47b-7dcb-49e6-b963-4cf7576bce1f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:51.051324 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:51.051019 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/066af47b-7dcb-49e6-b963-4cf7576bce1f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:51.051324 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:51.051044 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/066af47b-7dcb-49e6-b963-4cf7576bce1f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:51.051324 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:51.051067 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/066af47b-7dcb-49e6-b963-4cf7576bce1f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:51.051324 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:51.051107 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/066af47b-7dcb-49e6-b963-4cf7576bce1f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:51.051324 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:51.051142 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/066af47b-7dcb-49e6-b963-4cf7576bce1f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:51.051324 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:51.051170 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-788hs\" (UniqueName: \"kubernetes.io/projected/066af47b-7dcb-49e6-b963-4cf7576bce1f-kube-api-access-788hs\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:51.051324 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:51.051197 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/066af47b-7dcb-49e6-b963-4cf7576bce1f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:51.051324 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:51.051235 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/066af47b-7dcb-49e6-b963-4cf7576bce1f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:51.051324 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:51.051270 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/066af47b-7dcb-49e6-b963-4cf7576bce1f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:51.052167 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:51.051304 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/066af47b-7dcb-49e6-b963-4cf7576bce1f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:51.052167 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:51.051587 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/066af47b-7dcb-49e6-b963-4cf7576bce1f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:51.052167 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:51.051725 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/066af47b-7dcb-49e6-b963-4cf7576bce1f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:51.052167 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:51.051964 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/066af47b-7dcb-49e6-b963-4cf7576bce1f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:51.052782 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:51.052464 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/066af47b-7dcb-49e6-b963-4cf7576bce1f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:51.052924 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:51.052814 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/066af47b-7dcb-49e6-b963-4cf7576bce1f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:51.053927 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:51.053897 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/066af47b-7dcb-49e6-b963-4cf7576bce1f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:51.054020 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:51.053954 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/066af47b-7dcb-49e6-b963-4cf7576bce1f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:51.054020 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:51.053969 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/066af47b-7dcb-49e6-b963-4cf7576bce1f-config\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:51.055834 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:51.054205 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/066af47b-7dcb-49e6-b963-4cf7576bce1f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:51.055834 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:51.054545 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/066af47b-7dcb-49e6-b963-4cf7576bce1f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:51.055834 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:51.054655 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/066af47b-7dcb-49e6-b963-4cf7576bce1f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:51.055834 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:51.055152 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/066af47b-7dcb-49e6-b963-4cf7576bce1f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:51.056063 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:51.055878 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/066af47b-7dcb-49e6-b963-4cf7576bce1f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:51.056226 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:51.056203 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/066af47b-7dcb-49e6-b963-4cf7576bce1f-config-out\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:51.056434 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:51.056413 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/066af47b-7dcb-49e6-b963-4cf7576bce1f-web-config\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:51.056554 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:51.056534 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/066af47b-7dcb-49e6-b963-4cf7576bce1f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:51.056899 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:51.056882 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/066af47b-7dcb-49e6-b963-4cf7576bce1f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:51.057471 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:51.057453 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/066af47b-7dcb-49e6-b963-4cf7576bce1f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:51.060420 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:51.060402 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-788hs\" (UniqueName: \"kubernetes.io/projected/066af47b-7dcb-49e6-b963-4cf7576bce1f-kube-api-access-788hs\") pod \"prometheus-k8s-0\" (UID: \"066af47b-7dcb-49e6-b963-4cf7576bce1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:51.173442 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:51.173409 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:38:51.311624 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:51.311597 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 02 21:38:51.312855 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:38:51.312830 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod066af47b_7dcb_49e6_b963_4cf7576bce1f.slice/crio-efa5b450f85f2979f2e20e134c17073c391600ed049c24ff4a816212f30f903d WatchSource:0}: Error finding container efa5b450f85f2979f2e20e134c17073c391600ed049c24ff4a816212f30f903d: Status 404 returned error can't find the container with id efa5b450f85f2979f2e20e134c17073c391600ed049c24ff4a816212f30f903d Jul 02 21:38:51.802567 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:51.802528 2569 generic.go:356] "Generic (PLEG): container finished" podID="066af47b-7dcb-49e6-b963-4cf7576bce1f" containerID="a3f5225272ec448e13cbbae716d7eb85be2609cc5f0034cecf49d0d9bc180b1e" exitCode=0 Jul 02 21:38:51.802723 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:51.802613 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"066af47b-7dcb-49e6-b963-4cf7576bce1f","Type":"ContainerDied","Data":"a3f5225272ec448e13cbbae716d7eb85be2609cc5f0034cecf49d0d9bc180b1e"} Jul 02 21:38:51.802723 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:51.802686 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"066af47b-7dcb-49e6-b963-4cf7576bce1f","Type":"ContainerStarted","Data":"efa5b450f85f2979f2e20e134c17073c391600ed049c24ff4a816212f30f903d"} Jul 02 21:38:52.809723 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:52.809638 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"066af47b-7dcb-49e6-b963-4cf7576bce1f","Type":"ContainerStarted","Data":"b5c1b23fd91a05ad358b8e94357cd220c2ee9876b8a4278afea3053e280c3f1a"} Jul 02 21:38:52.810196 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:52.809754 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"066af47b-7dcb-49e6-b963-4cf7576bce1f","Type":"ContainerStarted","Data":"22b3c93fc644f7ca79afc2b2a221e0132a908712a0e127ae3ba1c137cb2a27c0"} Jul 02 21:38:52.810196 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:52.809817 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"066af47b-7dcb-49e6-b963-4cf7576bce1f","Type":"ContainerStarted","Data":"2c5e12e2b433bcc27eba5912b0654da4b27c69bd5f1bb90b5245b2c9914bd745"} Jul 02 21:38:52.810196 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:52.809862 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"066af47b-7dcb-49e6-b963-4cf7576bce1f","Type":"ContainerStarted","Data":"1f129fc24a426dbf396d1a2897e6ddb2ef35a3a4d5bf769bd707264c0b22b3d0"} Jul 02 21:38:52.810196 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:52.809915 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"066af47b-7dcb-49e6-b963-4cf7576bce1f","Type":"ContainerStarted","Data":"92ebb6c19ad240d27f1dc2056417fa733ccae5fc8497e27a221622df84a19904"} Jul 02 21:38:52.810196 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:52.809958 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"066af47b-7dcb-49e6-b963-4cf7576bce1f","Type":"ContainerStarted","Data":"2333eb2f2c35cd61167e82f0c9378f43f576ea317cb40163480c753b32ebdc8c"} Jul 02 21:38:52.839727 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:52.839677 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.820867213 podStartE2EDuration="2.839662449s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.795236ms" isStatefulPod=false podCreationTimestamp="2026-07-02 21:38:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:38:52.837176242 +0000 UTC m=+172.176146678" watchObservedRunningTime="2026-07-02 21:38:52.839662449 +0000 UTC m=+172.178632884" Jul 02 21:38:56.174046 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:38:56.174015 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:39:51.174408 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:39:51.174372 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:39:51.189538 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:39:51.189514 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:39:52.072983 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:39:52.072954 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 21:40:25.037262 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:40:25.037184 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jvmm7"] Jul 02 21:40:25.039291 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:40:25.039274 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jvmm7" Jul 02 21:40:25.041621 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:40:25.041599 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 02 21:40:25.041737 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:40:25.041628 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 02 21:40:25.042485 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:40:25.042469 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-7c9bd\"" Jul 02 21:40:25.049043 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:40:25.049024 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jvmm7"] Jul 02 21:40:25.109400 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:40:25.109370 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c8efb66e-8222-4daf-9fc9-6b41de55c4b3-tmp\") pod \"openshift-lws-operator-fd8ccff4c-jvmm7\" (UID: \"c8efb66e-8222-4daf-9fc9-6b41de55c4b3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jvmm7" Jul 02 21:40:25.109508 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:40:25.109410 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drgc5\" (UniqueName: \"kubernetes.io/projected/c8efb66e-8222-4daf-9fc9-6b41de55c4b3-kube-api-access-drgc5\") pod \"openshift-lws-operator-fd8ccff4c-jvmm7\" (UID: \"c8efb66e-8222-4daf-9fc9-6b41de55c4b3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jvmm7" Jul 02 21:40:25.210839 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:40:25.210783 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c8efb66e-8222-4daf-9fc9-6b41de55c4b3-tmp\") pod \"openshift-lws-operator-fd8ccff4c-jvmm7\" (UID: \"c8efb66e-8222-4daf-9fc9-6b41de55c4b3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jvmm7" Jul 02 21:40:25.210973 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:40:25.210843 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drgc5\" (UniqueName: \"kubernetes.io/projected/c8efb66e-8222-4daf-9fc9-6b41de55c4b3-kube-api-access-drgc5\") pod \"openshift-lws-operator-fd8ccff4c-jvmm7\" (UID: \"c8efb66e-8222-4daf-9fc9-6b41de55c4b3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jvmm7" Jul 02 21:40:25.211179 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:40:25.211161 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c8efb66e-8222-4daf-9fc9-6b41de55c4b3-tmp\") pod \"openshift-lws-operator-fd8ccff4c-jvmm7\" (UID: \"c8efb66e-8222-4daf-9fc9-6b41de55c4b3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jvmm7" Jul 02 21:40:25.219676 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:40:25.219649 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-drgc5\" (UniqueName: \"kubernetes.io/projected/c8efb66e-8222-4daf-9fc9-6b41de55c4b3-kube-api-access-drgc5\") pod \"openshift-lws-operator-fd8ccff4c-jvmm7\" (UID: \"c8efb66e-8222-4daf-9fc9-6b41de55c4b3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jvmm7" Jul 02 21:40:25.357943 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:40:25.357879 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jvmm7" Jul 02 21:40:25.481890 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:40:25.481860 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jvmm7"] Jul 02 21:40:25.485495 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:40:25.485456 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc8efb66e_8222_4daf_9fc9_6b41de55c4b3.slice/crio-184531f63a0b80887ce6a30e63c9344e868b202e77a548ea032a800827583c6b WatchSource:0}: Error finding container 184531f63a0b80887ce6a30e63c9344e868b202e77a548ea032a800827583c6b: Status 404 returned error can't find the container with id 184531f63a0b80887ce6a30e63c9344e868b202e77a548ea032a800827583c6b Jul 02 21:40:26.182390 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:40:26.182358 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jvmm7" event={"ID":"c8efb66e-8222-4daf-9fc9-6b41de55c4b3","Type":"ContainerStarted","Data":"184531f63a0b80887ce6a30e63c9344e868b202e77a548ea032a800827583c6b"} Jul 02 21:40:29.196273 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:40:29.196237 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jvmm7" event={"ID":"c8efb66e-8222-4daf-9fc9-6b41de55c4b3","Type":"ContainerStarted","Data":"6c5250bd88fcda9dbfdd0ec158cc50e6abf7559238151a92a4e9b4a60d9fe0b4"} Jul 02 21:40:29.216956 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:40:29.216912 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jvmm7" podStartSLOduration=1.433762008 podStartE2EDuration="4.216898641s" totalImagesPullingTime="2.783136633s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:40:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:40:29.2155868 +0000 UTC m=+268.554557236" watchObservedRunningTime="2026-07-02 21:40:29.216898641 +0000 UTC m=+268.555869075" Jul 02 21:40:49.307551 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:40:49.307509 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-bcc4b86cc-8md8v"] Jul 02 21:40:49.310104 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:40:49.310083 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-bcc4b86cc-8md8v" Jul 02 21:40:49.314896 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:40:49.314877 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 02 21:40:49.315007 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:40:49.314947 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 02 21:40:49.315851 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:40:49.315836 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-gm24m\"" Jul 02 21:40:49.316264 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:40:49.316248 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 02 21:40:49.319336 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:40:49.319317 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 02 21:40:49.334206 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:40:49.334183 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-bcc4b86cc-8md8v"] Jul 02 21:40:49.411083 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:40:49.411055 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8fa1f2a1-ceb0-4696-928b-bc09a5a4b5b6-manifests\") pod \"opendatahub-operator-controller-manager-bcc4b86cc-8md8v\" (UID: \"8fa1f2a1-ceb0-4696-928b-bc09a5a4b5b6\") " pod="opendatahub/opendatahub-operator-controller-manager-bcc4b86cc-8md8v" Jul 02 21:40:49.411243 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:40:49.411100 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8fa1f2a1-ceb0-4696-928b-bc09a5a4b5b6-webhook-cert\") pod \"opendatahub-operator-controller-manager-bcc4b86cc-8md8v\" (UID: \"8fa1f2a1-ceb0-4696-928b-bc09a5a4b5b6\") " pod="opendatahub/opendatahub-operator-controller-manager-bcc4b86cc-8md8v" Jul 02 21:40:49.411243 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:40:49.411155 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8fa1f2a1-ceb0-4696-928b-bc09a5a4b5b6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-bcc4b86cc-8md8v\" (UID: \"8fa1f2a1-ceb0-4696-928b-bc09a5a4b5b6\") " pod="opendatahub/opendatahub-operator-controller-manager-bcc4b86cc-8md8v" Jul 02 21:40:49.411243 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:40:49.411199 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8fa1f2a1-ceb0-4696-928b-bc09a5a4b5b6-tmp\") pod \"opendatahub-operator-controller-manager-bcc4b86cc-8md8v\" (UID: \"8fa1f2a1-ceb0-4696-928b-bc09a5a4b5b6\") " pod="opendatahub/opendatahub-operator-controller-manager-bcc4b86cc-8md8v" Jul 02 21:40:49.411342 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:40:49.411247 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5pm2\" (UniqueName: \"kubernetes.io/projected/8fa1f2a1-ceb0-4696-928b-bc09a5a4b5b6-kube-api-access-d5pm2\") pod \"opendatahub-operator-controller-manager-bcc4b86cc-8md8v\" (UID: \"8fa1f2a1-ceb0-4696-928b-bc09a5a4b5b6\") " pod="opendatahub/opendatahub-operator-controller-manager-bcc4b86cc-8md8v" Jul 02 21:40:49.511990 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:40:49.511963 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8fa1f2a1-ceb0-4696-928b-bc09a5a4b5b6-manifests\") pod \"opendatahub-operator-controller-manager-bcc4b86cc-8md8v\" (UID: \"8fa1f2a1-ceb0-4696-928b-bc09a5a4b5b6\") " pod="opendatahub/opendatahub-operator-controller-manager-bcc4b86cc-8md8v" Jul 02 21:40:49.512173 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:40:49.512007 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8fa1f2a1-ceb0-4696-928b-bc09a5a4b5b6-webhook-cert\") pod \"opendatahub-operator-controller-manager-bcc4b86cc-8md8v\" (UID: \"8fa1f2a1-ceb0-4696-928b-bc09a5a4b5b6\") " pod="opendatahub/opendatahub-operator-controller-manager-bcc4b86cc-8md8v" Jul 02 21:40:49.512173 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:40:49.512035 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8fa1f2a1-ceb0-4696-928b-bc09a5a4b5b6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-bcc4b86cc-8md8v\" (UID: \"8fa1f2a1-ceb0-4696-928b-bc09a5a4b5b6\") " pod="opendatahub/opendatahub-operator-controller-manager-bcc4b86cc-8md8v" Jul 02 21:40:49.512173 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:40:49.512059 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8fa1f2a1-ceb0-4696-928b-bc09a5a4b5b6-tmp\") pod \"opendatahub-operator-controller-manager-bcc4b86cc-8md8v\" (UID: \"8fa1f2a1-ceb0-4696-928b-bc09a5a4b5b6\") " pod="opendatahub/opendatahub-operator-controller-manager-bcc4b86cc-8md8v" Jul 02 21:40:49.512173 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:40:49.512096 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5pm2\" (UniqueName: \"kubernetes.io/projected/8fa1f2a1-ceb0-4696-928b-bc09a5a4b5b6-kube-api-access-d5pm2\") pod \"opendatahub-operator-controller-manager-bcc4b86cc-8md8v\" (UID: \"8fa1f2a1-ceb0-4696-928b-bc09a5a4b5b6\") " pod="opendatahub/opendatahub-operator-controller-manager-bcc4b86cc-8md8v" Jul 02 21:40:49.512397 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:40:49.512347 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8fa1f2a1-ceb0-4696-928b-bc09a5a4b5b6-manifests\") pod \"opendatahub-operator-controller-manager-bcc4b86cc-8md8v\" (UID: \"8fa1f2a1-ceb0-4696-928b-bc09a5a4b5b6\") " pod="opendatahub/opendatahub-operator-controller-manager-bcc4b86cc-8md8v" Jul 02 21:40:49.514403 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:40:49.514380 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8fa1f2a1-ceb0-4696-928b-bc09a5a4b5b6-tmp\") pod \"opendatahub-operator-controller-manager-bcc4b86cc-8md8v\" (UID: \"8fa1f2a1-ceb0-4696-928b-bc09a5a4b5b6\") " pod="opendatahub/opendatahub-operator-controller-manager-bcc4b86cc-8md8v" Jul 02 21:40:49.514546 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:40:49.514528 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8fa1f2a1-ceb0-4696-928b-bc09a5a4b5b6-webhook-cert\") pod \"opendatahub-operator-controller-manager-bcc4b86cc-8md8v\" (UID: \"8fa1f2a1-ceb0-4696-928b-bc09a5a4b5b6\") " pod="opendatahub/opendatahub-operator-controller-manager-bcc4b86cc-8md8v" Jul 02 21:40:49.514768 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:40:49.514750 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8fa1f2a1-ceb0-4696-928b-bc09a5a4b5b6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-bcc4b86cc-8md8v\" (UID: \"8fa1f2a1-ceb0-4696-928b-bc09a5a4b5b6\") " pod="opendatahub/opendatahub-operator-controller-manager-bcc4b86cc-8md8v" Jul 02 21:40:49.520621 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:40:49.520596 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5pm2\" (UniqueName: \"kubernetes.io/projected/8fa1f2a1-ceb0-4696-928b-bc09a5a4b5b6-kube-api-access-d5pm2\") pod \"opendatahub-operator-controller-manager-bcc4b86cc-8md8v\" (UID: \"8fa1f2a1-ceb0-4696-928b-bc09a5a4b5b6\") " pod="opendatahub/opendatahub-operator-controller-manager-bcc4b86cc-8md8v" Jul 02 21:40:49.620488 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:40:49.620403 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-bcc4b86cc-8md8v" Jul 02 21:40:49.751971 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:40:49.751942 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-bcc4b86cc-8md8v"] Jul 02 21:40:49.755767 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:40:49.755735 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fa1f2a1_ceb0_4696_928b_bc09a5a4b5b6.slice/crio-aa074ba4c03324b30bfdf4747c9706a26a74e2154add3ca36978f96e3fa64a79 WatchSource:0}: Error finding container aa074ba4c03324b30bfdf4747c9706a26a74e2154add3ca36978f96e3fa64a79: Status 404 returned error can't find the container with id aa074ba4c03324b30bfdf4747c9706a26a74e2154add3ca36978f96e3fa64a79 Jul 02 21:40:50.276879 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:40:50.276839 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bcc4b86cc-8md8v" event={"ID":"8fa1f2a1-ceb0-4696-928b-bc09a5a4b5b6","Type":"ContainerStarted","Data":"aa074ba4c03324b30bfdf4747c9706a26a74e2154add3ca36978f96e3fa64a79"} Jul 02 21:40:53.293284 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:40:53.293251 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bcc4b86cc-8md8v" event={"ID":"8fa1f2a1-ceb0-4696-928b-bc09a5a4b5b6","Type":"ContainerStarted","Data":"afbdf21123dd292e1a53267518e81267e5924f365bd847bb69daf9a767e597ea"} Jul 02 21:40:54.299298 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:40:54.299260 2569 generic.go:356] "Generic (PLEG): container finished" podID="8fa1f2a1-ceb0-4696-928b-bc09a5a4b5b6" containerID="afbdf21123dd292e1a53267518e81267e5924f365bd847bb69daf9a767e597ea" exitCode=0 Jul 02 21:40:54.299667 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:40:54.299344 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bcc4b86cc-8md8v" event={"ID":"8fa1f2a1-ceb0-4696-928b-bc09a5a4b5b6","Type":"ContainerDied","Data":"afbdf21123dd292e1a53267518e81267e5924f365bd847bb69daf9a767e597ea"} Jul 02 21:40:55.304993 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:40:55.304956 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bcc4b86cc-8md8v" event={"ID":"8fa1f2a1-ceb0-4696-928b-bc09a5a4b5b6","Type":"ContainerStarted","Data":"328e482bf3780d8548ef5279c4b28d62b9412df3634ccf7d2966ab8acc240386"} Jul 02 21:40:55.305437 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:40:55.305108 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-bcc4b86cc-8md8v" Jul 02 21:40:55.326903 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:40:55.326847 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-bcc4b86cc-8md8v" podStartSLOduration=1.896811823 podStartE2EDuration="6.32683127s" totalImagesPullingTime="3.393390069s" totalInitContainerRuntime="1.036629378s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:40:49 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:40:55.325758428 +0000 UTC m=+294.664728862" watchObservedRunningTime="2026-07-02 21:40:55.32683127 +0000 UTC m=+294.665801761" Jul 02 21:41:01.036111 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:01.036080 2569 kubelet.go:1648] "Image garbage collection succeeded" Jul 02 21:41:06.311329 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:06.311293 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-bcc4b86cc-8md8v" Jul 02 21:41:09.000435 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:09.000404 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-8697fd7f55-c7p8v"] Jul 02 21:41:09.002572 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:09.002550 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-8697fd7f55-c7p8v" Jul 02 21:41:09.008206 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:09.008183 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 02 21:41:09.008465 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:09.008443 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 02 21:41:09.008668 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:09.008533 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 02 21:41:09.008668 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:09.008550 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-tkjpv\"" Jul 02 21:41:09.008668 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:09.008548 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 02 21:41:09.025640 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:09.025616 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-8697fd7f55-c7p8v"] Jul 02 21:41:09.083154 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:09.083118 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5515c578-d7e1-4942-bc9a-722589fa6a30-tls-certs\") pod \"kube-auth-proxy-8697fd7f55-c7p8v\" (UID: \"5515c578-d7e1-4942-bc9a-722589fa6a30\") " pod="openshift-ingress/kube-auth-proxy-8697fd7f55-c7p8v" Jul 02 21:41:09.083347 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:09.083163 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmfkq\" (UniqueName: \"kubernetes.io/projected/5515c578-d7e1-4942-bc9a-722589fa6a30-kube-api-access-hmfkq\") pod \"kube-auth-proxy-8697fd7f55-c7p8v\" (UID: \"5515c578-d7e1-4942-bc9a-722589fa6a30\") " pod="openshift-ingress/kube-auth-proxy-8697fd7f55-c7p8v" Jul 02 21:41:09.083347 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:09.083208 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5515c578-d7e1-4942-bc9a-722589fa6a30-tmp\") pod \"kube-auth-proxy-8697fd7f55-c7p8v\" (UID: \"5515c578-d7e1-4942-bc9a-722589fa6a30\") " pod="openshift-ingress/kube-auth-proxy-8697fd7f55-c7p8v" Jul 02 21:41:09.184062 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:09.184026 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmfkq\" (UniqueName: \"kubernetes.io/projected/5515c578-d7e1-4942-bc9a-722589fa6a30-kube-api-access-hmfkq\") pod \"kube-auth-proxy-8697fd7f55-c7p8v\" (UID: \"5515c578-d7e1-4942-bc9a-722589fa6a30\") " pod="openshift-ingress/kube-auth-proxy-8697fd7f55-c7p8v" Jul 02 21:41:09.184223 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:09.184076 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5515c578-d7e1-4942-bc9a-722589fa6a30-tmp\") pod \"kube-auth-proxy-8697fd7f55-c7p8v\" (UID: \"5515c578-d7e1-4942-bc9a-722589fa6a30\") " pod="openshift-ingress/kube-auth-proxy-8697fd7f55-c7p8v" Jul 02 21:41:09.184223 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:09.184119 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5515c578-d7e1-4942-bc9a-722589fa6a30-tls-certs\") pod \"kube-auth-proxy-8697fd7f55-c7p8v\" (UID: \"5515c578-d7e1-4942-bc9a-722589fa6a30\") " pod="openshift-ingress/kube-auth-proxy-8697fd7f55-c7p8v" Jul 02 21:41:09.186343 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:09.186314 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5515c578-d7e1-4942-bc9a-722589fa6a30-tmp\") pod \"kube-auth-proxy-8697fd7f55-c7p8v\" (UID: \"5515c578-d7e1-4942-bc9a-722589fa6a30\") " pod="openshift-ingress/kube-auth-proxy-8697fd7f55-c7p8v" Jul 02 21:41:09.186451 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:09.186418 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5515c578-d7e1-4942-bc9a-722589fa6a30-tls-certs\") pod \"kube-auth-proxy-8697fd7f55-c7p8v\" (UID: \"5515c578-d7e1-4942-bc9a-722589fa6a30\") " pod="openshift-ingress/kube-auth-proxy-8697fd7f55-c7p8v" Jul 02 21:41:09.196216 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:09.196191 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmfkq\" (UniqueName: \"kubernetes.io/projected/5515c578-d7e1-4942-bc9a-722589fa6a30-kube-api-access-hmfkq\") pod \"kube-auth-proxy-8697fd7f55-c7p8v\" (UID: \"5515c578-d7e1-4942-bc9a-722589fa6a30\") " pod="openshift-ingress/kube-auth-proxy-8697fd7f55-c7p8v" Jul 02 21:41:09.224397 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:09.224370 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-qsvm7"] Jul 02 21:41:09.226589 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:09.226574 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-qsvm7" Jul 02 21:41:09.229436 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:09.229417 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 02 21:41:09.229520 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:09.229487 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-6zrbj\"" Jul 02 21:41:09.242418 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:09.242394 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-qsvm7"] Jul 02 21:41:09.284737 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:09.284669 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fkzz\" (UniqueName: \"kubernetes.io/projected/9abe791f-664b-45a5-ab2f-b6aa86f63503-kube-api-access-8fkzz\") pod \"odh-model-controller-7998c67dc4-qsvm7\" (UID: \"9abe791f-664b-45a5-ab2f-b6aa86f63503\") " pod="opendatahub/odh-model-controller-7998c67dc4-qsvm7" Jul 02 21:41:09.284737 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:09.284730 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9abe791f-664b-45a5-ab2f-b6aa86f63503-cert\") pod \"odh-model-controller-7998c67dc4-qsvm7\" (UID: \"9abe791f-664b-45a5-ab2f-b6aa86f63503\") " pod="opendatahub/odh-model-controller-7998c67dc4-qsvm7" Jul 02 21:41:09.311966 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:09.311937 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-8697fd7f55-c7p8v" Jul 02 21:41:09.385544 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:09.385512 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fkzz\" (UniqueName: \"kubernetes.io/projected/9abe791f-664b-45a5-ab2f-b6aa86f63503-kube-api-access-8fkzz\") pod \"odh-model-controller-7998c67dc4-qsvm7\" (UID: \"9abe791f-664b-45a5-ab2f-b6aa86f63503\") " pod="opendatahub/odh-model-controller-7998c67dc4-qsvm7" Jul 02 21:41:09.385677 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:09.385574 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9abe791f-664b-45a5-ab2f-b6aa86f63503-cert\") pod \"odh-model-controller-7998c67dc4-qsvm7\" (UID: \"9abe791f-664b-45a5-ab2f-b6aa86f63503\") " pod="opendatahub/odh-model-controller-7998c67dc4-qsvm7" Jul 02 21:41:09.385728 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:41:09.385713 2569 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 02 21:41:09.385950 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:41:09.385781 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9abe791f-664b-45a5-ab2f-b6aa86f63503-cert podName:9abe791f-664b-45a5-ab2f-b6aa86f63503 nodeName:}" failed. No retries permitted until 2026-07-02 21:41:09.885761875 +0000 UTC m=+309.224732301 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/9abe791f-664b-45a5-ab2f-b6aa86f63503-cert") pod "odh-model-controller-7998c67dc4-qsvm7" (UID: "9abe791f-664b-45a5-ab2f-b6aa86f63503") : secret "odh-model-controller-webhook-cert" not found Jul 02 21:41:09.393699 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:09.393668 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fkzz\" (UniqueName: \"kubernetes.io/projected/9abe791f-664b-45a5-ab2f-b6aa86f63503-kube-api-access-8fkzz\") pod \"odh-model-controller-7998c67dc4-qsvm7\" (UID: \"9abe791f-664b-45a5-ab2f-b6aa86f63503\") " pod="opendatahub/odh-model-controller-7998c67dc4-qsvm7" Jul 02 21:41:09.450387 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:09.450361 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-8697fd7f55-c7p8v"] Jul 02 21:41:09.452527 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:41:09.452498 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5515c578_d7e1_4942_bc9a_722589fa6a30.slice/crio-f69b1af6f9c3aafe08c64e1970d7d1d3de2c4719e2f269fc742b57a6005a0448 WatchSource:0}: Error finding container f69b1af6f9c3aafe08c64e1970d7d1d3de2c4719e2f269fc742b57a6005a0448: Status 404 returned error can't find the container with id f69b1af6f9c3aafe08c64e1970d7d1d3de2c4719e2f269fc742b57a6005a0448 Jul 02 21:41:09.454334 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:09.454316 2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 21:41:09.888997 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:09.888964 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9abe791f-664b-45a5-ab2f-b6aa86f63503-cert\") pod \"odh-model-controller-7998c67dc4-qsvm7\" (UID: \"9abe791f-664b-45a5-ab2f-b6aa86f63503\") " pod="opendatahub/odh-model-controller-7998c67dc4-qsvm7" Jul 02 21:41:09.891278 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:09.891245 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9abe791f-664b-45a5-ab2f-b6aa86f63503-cert\") pod \"odh-model-controller-7998c67dc4-qsvm7\" (UID: \"9abe791f-664b-45a5-ab2f-b6aa86f63503\") " pod="opendatahub/odh-model-controller-7998c67dc4-qsvm7" Jul 02 21:41:10.136422 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:10.136384 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-qsvm7" Jul 02 21:41:10.312857 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:10.312833 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-qsvm7"] Jul 02 21:41:10.315496 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:41:10.315468 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9abe791f_664b_45a5_ab2f_b6aa86f63503.slice/crio-28b72527f99ee65f9bf24a39b630d15565db629ce1a261c45534433cb3250c33 WatchSource:0}: Error finding container 28b72527f99ee65f9bf24a39b630d15565db629ce1a261c45534433cb3250c33: Status 404 returned error can't find the container with id 28b72527f99ee65f9bf24a39b630d15565db629ce1a261c45534433cb3250c33 Jul 02 21:41:10.335437 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:10.335414 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-94g9b"] Jul 02 21:41:10.338157 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:10.338139 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-94g9b" Jul 02 21:41:10.341962 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:10.341943 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 02 21:41:10.342065 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:10.341973 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-nsj85\"" Jul 02 21:41:10.365917 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:10.365891 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-94g9b"] Jul 02 21:41:10.371358 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:10.371330 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-qsvm7" event={"ID":"9abe791f-664b-45a5-ab2f-b6aa86f63503","Type":"ContainerStarted","Data":"28b72527f99ee65f9bf24a39b630d15565db629ce1a261c45534433cb3250c33"} Jul 02 21:41:10.372485 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:10.372435 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-8697fd7f55-c7p8v" event={"ID":"5515c578-d7e1-4942-bc9a-722589fa6a30","Type":"ContainerStarted","Data":"f69b1af6f9c3aafe08c64e1970d7d1d3de2c4719e2f269fc742b57a6005a0448"} Jul 02 21:41:10.391556 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:10.391533 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d61dd72b-1715-411c-affd-b4bff8c76012-tls-certs\") pod \"model-serving-api-54d89bd5fc-94g9b\" (UID: \"d61dd72b-1715-411c-affd-b4bff8c76012\") " pod="opendatahub/model-serving-api-54d89bd5fc-94g9b" Jul 02 21:41:10.391649 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:10.391587 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mnhs\" (UniqueName: \"kubernetes.io/projected/d61dd72b-1715-411c-affd-b4bff8c76012-kube-api-access-9mnhs\") pod \"model-serving-api-54d89bd5fc-94g9b\" (UID: \"d61dd72b-1715-411c-affd-b4bff8c76012\") " pod="opendatahub/model-serving-api-54d89bd5fc-94g9b" Jul 02 21:41:10.463773 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:10.463695 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-zqg5d"] Jul 02 21:41:10.465961 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:10.465945 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zqg5d" Jul 02 21:41:10.468615 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:10.468587 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 02 21:41:10.469023 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:10.468877 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 02 21:41:10.469023 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:10.468906 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-f2mq5\"" Jul 02 21:41:10.469023 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:10.468989 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 02 21:41:10.469023 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:10.469004 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 02 21:41:10.485232 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:10.485210 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-zqg5d"] Jul 02 21:41:10.492310 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:10.492291 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07c52afa-aaed-4f8f-996b-ad240001f420-istio-token\") pod \"istiod-openshift-gateway-7574459656-zqg5d\" (UID: \"07c52afa-aaed-4f8f-996b-ad240001f420\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zqg5d" Jul 02 21:41:10.492395 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:10.492320 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/07c52afa-aaed-4f8f-996b-ad240001f420-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-zqg5d\" (UID: \"07c52afa-aaed-4f8f-996b-ad240001f420\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zqg5d" Jul 02 21:41:10.492395 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:10.492340 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7ljm\" (UniqueName: \"kubernetes.io/projected/07c52afa-aaed-4f8f-996b-ad240001f420-kube-api-access-q7ljm\") pod \"istiod-openshift-gateway-7574459656-zqg5d\" (UID: \"07c52afa-aaed-4f8f-996b-ad240001f420\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zqg5d" Jul 02 21:41:10.492462 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:10.492406 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mnhs\" (UniqueName: \"kubernetes.io/projected/d61dd72b-1715-411c-affd-b4bff8c76012-kube-api-access-9mnhs\") pod \"model-serving-api-54d89bd5fc-94g9b\" (UID: \"d61dd72b-1715-411c-affd-b4bff8c76012\") " pod="opendatahub/model-serving-api-54d89bd5fc-94g9b" Jul 02 21:41:10.492462 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:10.492429 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/07c52afa-aaed-4f8f-996b-ad240001f420-local-certs\") pod \"istiod-openshift-gateway-7574459656-zqg5d\" (UID: \"07c52afa-aaed-4f8f-996b-ad240001f420\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zqg5d" Jul 02 21:41:10.492545 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:10.492531 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d61dd72b-1715-411c-affd-b4bff8c76012-tls-certs\") pod \"model-serving-api-54d89bd5fc-94g9b\" (UID: \"d61dd72b-1715-411c-affd-b4bff8c76012\") " pod="opendatahub/model-serving-api-54d89bd5fc-94g9b" Jul 02 21:41:10.492580 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:10.492569 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/07c52afa-aaed-4f8f-996b-ad240001f420-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-zqg5d\" (UID: \"07c52afa-aaed-4f8f-996b-ad240001f420\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zqg5d" Jul 02 21:41:10.492613 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:10.492603 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/07c52afa-aaed-4f8f-996b-ad240001f420-cacerts\") pod \"istiod-openshift-gateway-7574459656-zqg5d\" (UID: \"07c52afa-aaed-4f8f-996b-ad240001f420\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zqg5d" Jul 02 21:41:10.492645 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:10.492620 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/07c52afa-aaed-4f8f-996b-ad240001f420-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-zqg5d\" (UID: \"07c52afa-aaed-4f8f-996b-ad240001f420\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zqg5d" Jul 02 21:41:10.494934 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:10.494915 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d61dd72b-1715-411c-affd-b4bff8c76012-tls-certs\") pod \"model-serving-api-54d89bd5fc-94g9b\" (UID: \"d61dd72b-1715-411c-affd-b4bff8c76012\") " pod="opendatahub/model-serving-api-54d89bd5fc-94g9b" Jul 02 21:41:10.504059 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:10.504038 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mnhs\" (UniqueName: \"kubernetes.io/projected/d61dd72b-1715-411c-affd-b4bff8c76012-kube-api-access-9mnhs\") pod \"model-serving-api-54d89bd5fc-94g9b\" (UID: \"d61dd72b-1715-411c-affd-b4bff8c76012\") " pod="opendatahub/model-serving-api-54d89bd5fc-94g9b" Jul 02 21:41:10.593110 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:10.593074 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/07c52afa-aaed-4f8f-996b-ad240001f420-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-zqg5d\" (UID: \"07c52afa-aaed-4f8f-996b-ad240001f420\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zqg5d" Jul 02 21:41:10.593276 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:10.593122 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/07c52afa-aaed-4f8f-996b-ad240001f420-cacerts\") pod \"istiod-openshift-gateway-7574459656-zqg5d\" (UID: \"07c52afa-aaed-4f8f-996b-ad240001f420\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zqg5d" Jul 02 21:41:10.593276 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:10.593168 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/07c52afa-aaed-4f8f-996b-ad240001f420-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-zqg5d\" (UID: \"07c52afa-aaed-4f8f-996b-ad240001f420\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zqg5d" Jul 02 21:41:10.593276 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:10.593229 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07c52afa-aaed-4f8f-996b-ad240001f420-istio-token\") pod \"istiod-openshift-gateway-7574459656-zqg5d\" (UID: \"07c52afa-aaed-4f8f-996b-ad240001f420\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zqg5d" Jul 02 21:41:10.593276 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:10.593256 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/07c52afa-aaed-4f8f-996b-ad240001f420-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-zqg5d\" (UID: \"07c52afa-aaed-4f8f-996b-ad240001f420\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zqg5d" Jul 02 21:41:10.593479 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:10.593284 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7ljm\" (UniqueName: \"kubernetes.io/projected/07c52afa-aaed-4f8f-996b-ad240001f420-kube-api-access-q7ljm\") pod \"istiod-openshift-gateway-7574459656-zqg5d\" (UID: \"07c52afa-aaed-4f8f-996b-ad240001f420\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zqg5d" Jul 02 21:41:10.593479 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:10.593317 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/07c52afa-aaed-4f8f-996b-ad240001f420-local-certs\") pod \"istiod-openshift-gateway-7574459656-zqg5d\" (UID: \"07c52afa-aaed-4f8f-996b-ad240001f420\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zqg5d" Jul 02 21:41:10.594257 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:10.594206 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/07c52afa-aaed-4f8f-996b-ad240001f420-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-zqg5d\" (UID: \"07c52afa-aaed-4f8f-996b-ad240001f420\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zqg5d" Jul 02 21:41:10.596030 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:10.595980 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/07c52afa-aaed-4f8f-996b-ad240001f420-local-certs\") pod \"istiod-openshift-gateway-7574459656-zqg5d\" (UID: \"07c52afa-aaed-4f8f-996b-ad240001f420\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zqg5d" Jul 02 21:41:10.596140 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:10.596099 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/07c52afa-aaed-4f8f-996b-ad240001f420-cacerts\") pod \"istiod-openshift-gateway-7574459656-zqg5d\" (UID: \"07c52afa-aaed-4f8f-996b-ad240001f420\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zqg5d" Jul 02 21:41:10.596424 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:10.596393 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/07c52afa-aaed-4f8f-996b-ad240001f420-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-zqg5d\" (UID: \"07c52afa-aaed-4f8f-996b-ad240001f420\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zqg5d" Jul 02 21:41:10.596617 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:10.596597 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/07c52afa-aaed-4f8f-996b-ad240001f420-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-zqg5d\" (UID: \"07c52afa-aaed-4f8f-996b-ad240001f420\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zqg5d" Jul 02 21:41:10.601715 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:10.601691 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07c52afa-aaed-4f8f-996b-ad240001f420-istio-token\") pod \"istiod-openshift-gateway-7574459656-zqg5d\" (UID: \"07c52afa-aaed-4f8f-996b-ad240001f420\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zqg5d" Jul 02 21:41:10.601837 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:10.601738 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7ljm\" (UniqueName: \"kubernetes.io/projected/07c52afa-aaed-4f8f-996b-ad240001f420-kube-api-access-q7ljm\") pod \"istiod-openshift-gateway-7574459656-zqg5d\" (UID: \"07c52afa-aaed-4f8f-996b-ad240001f420\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zqg5d" Jul 02 21:41:10.648600 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:10.648570 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-94g9b" Jul 02 21:41:10.775334 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:10.775300 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zqg5d" Jul 02 21:41:10.789652 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:10.789628 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-94g9b"] Jul 02 21:41:10.791965 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:41:10.791934 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd61dd72b_1715_411c_affd_b4bff8c76012.slice/crio-95f6d709895d86e93dd5e58976a13a3cfa739ec0b57a635ff671f7e0d3a5b507 WatchSource:0}: Error finding container 95f6d709895d86e93dd5e58976a13a3cfa739ec0b57a635ff671f7e0d3a5b507: Status 404 returned error can't find the container with id 95f6d709895d86e93dd5e58976a13a3cfa739ec0b57a635ff671f7e0d3a5b507 Jul 02 21:41:10.928575 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:10.928551 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-zqg5d"] Jul 02 21:41:10.933610 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:41:10.933582 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07c52afa_aaed_4f8f_996b_ad240001f420.slice/crio-9c786ac4a73d99902054650aacb8d723eb579b402521558f5cff0e9274707ee3 WatchSource:0}: Error finding container 9c786ac4a73d99902054650aacb8d723eb579b402521558f5cff0e9274707ee3: Status 404 returned error can't find the container with id 9c786ac4a73d99902054650aacb8d723eb579b402521558f5cff0e9274707ee3 Jul 02 21:41:11.380241 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:11.380198 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-94g9b" event={"ID":"d61dd72b-1715-411c-affd-b4bff8c76012","Type":"ContainerStarted","Data":"95f6d709895d86e93dd5e58976a13a3cfa739ec0b57a635ff671f7e0d3a5b507"} Jul 02 21:41:11.381666 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:11.381605 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zqg5d" event={"ID":"07c52afa-aaed-4f8f-996b-ad240001f420","Type":"ContainerStarted","Data":"9c786ac4a73d99902054650aacb8d723eb579b402521558f5cff0e9274707ee3"} Jul 02 21:41:15.404165 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:15.404093 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-qsvm7" event={"ID":"9abe791f-664b-45a5-ab2f-b6aa86f63503","Type":"ContainerStarted","Data":"e7171774e0a8446c987a7fae6c845821bac0c221714f43f45dbaa113264676e0"} Jul 02 21:41:15.404622 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:15.404238 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-qsvm7" Jul 02 21:41:15.405962 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:15.405928 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-8697fd7f55-c7p8v" event={"ID":"5515c578-d7e1-4942-bc9a-722589fa6a30","Type":"ContainerStarted","Data":"e0ca74b7d945666d9383f7b39de24a62efdeb059c715a2735dc945f6cdbcc8f3"} Jul 02 21:41:15.407612 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:15.407587 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-94g9b" event={"ID":"d61dd72b-1715-411c-affd-b4bff8c76012","Type":"ContainerStarted","Data":"197cc0ecffb67db42e097e0761beb8db24cd9edc3422bdec78050de8706a1759"} Jul 02 21:41:15.407756 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:15.407738 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-94g9b" Jul 02 21:41:15.419096 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:15.419051 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-qsvm7" podStartSLOduration=2.269377627 podStartE2EDuration="6.419034273s" totalImagesPullingTime="4.149656646s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:41:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:41:15.418076977 +0000 UTC m=+314.757047427" watchObservedRunningTime="2026-07-02 21:41:15.419034273 +0000 UTC m=+314.758004713" Jul 02 21:41:15.433258 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:15.433217 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-8697fd7f55-c7p8v" podStartSLOduration=2.50247728 podStartE2EDuration="7.43320442s" totalImagesPullingTime="4.93072714s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:41:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:41:15.432446528 +0000 UTC m=+314.771416953" watchObservedRunningTime="2026-07-02 21:41:15.43320442 +0000 UTC m=+314.772174855" Jul 02 21:41:15.448364 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:15.448323 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-94g9b" podStartSLOduration=1.857558537 podStartE2EDuration="5.448309264s" totalImagesPullingTime="3.590750727s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:41:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:41:15.447048931 +0000 UTC m=+314.786019378" watchObservedRunningTime="2026-07-02 21:41:15.448309264 +0000 UTC m=+314.787279699" Jul 02 21:41:16.368411 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:16.368373 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Jul 02 21:41:16.368468 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:16.368447 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Jul 02 21:41:17.419517 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:17.419478 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zqg5d" event={"ID":"07c52afa-aaed-4f8f-996b-ad240001f420","Type":"ContainerStarted","Data":"a829c02eb06ae2d584fac0250faacaf1d6d45db6f0b86819fe15fb988c55b5e5"} Jul 02 21:41:17.421189 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:17.421167 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zqg5d" Jul 02 21:41:17.424504 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:17.424463 2569 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-zqg5d container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 02 21:41:17.424637 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:17.424534 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zqg5d" podUID="07c52afa-aaed-4f8f-996b-ad240001f420" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 21:41:17.445056 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:17.444994 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zqg5d" podStartSLOduration=2.012130178 podStartE2EDuration="7.444975897s" totalImagesPullingTime="5.432845719s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:41:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:41:17.444941474 +0000 UTC m=+316.783911922" watchObservedRunningTime="2026-07-02 21:41:17.444975897 +0000 UTC m=+316.783946334" Jul 02 21:41:17.974928 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:17.974896 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h7mzx"] Jul 02 21:41:17.978871 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:17.978845 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h7mzx" Jul 02 21:41:17.981376 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:17.981353 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-ctjcv\"" Jul 02 21:41:17.994477 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:17.994446 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h7mzx"] Jul 02 21:41:18.071202 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:18.071168 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h7mzx\" (UID: \"2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h7mzx" Jul 02 21:41:18.071202 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:18.071205 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h7mzx\" (UID: \"2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h7mzx" Jul 02 21:41:18.071452 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:18.071226 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h7mzx\" (UID: \"2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h7mzx" Jul 02 21:41:18.071452 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:18.071253 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h7mzx\" (UID: \"2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h7mzx" Jul 02 21:41:18.071452 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:18.071314 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nt7m\" (UniqueName: \"kubernetes.io/projected/2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7-kube-api-access-8nt7m\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h7mzx\" (UID: \"2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h7mzx" Jul 02 21:41:18.071452 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:18.071376 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h7mzx\" (UID: \"2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h7mzx" Jul 02 21:41:18.071452 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:18.071397 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h7mzx\" (UID: \"2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h7mzx" Jul 02 21:41:18.071452 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:18.071433 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h7mzx\" (UID: \"2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h7mzx" Jul 02 21:41:18.071756 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:18.071472 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h7mzx\" (UID: \"2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h7mzx" Jul 02 21:41:18.172713 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:18.172671 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h7mzx\" (UID: \"2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h7mzx" Jul 02 21:41:18.172934 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:18.172723 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h7mzx\" (UID: \"2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h7mzx" Jul 02 21:41:18.172934 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:18.172758 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h7mzx\" (UID: \"2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h7mzx" Jul 02 21:41:18.172934 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:18.172839 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h7mzx\" (UID: \"2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h7mzx" Jul 02 21:41:18.172934 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:18.172878 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h7mzx\" (UID: \"2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h7mzx" Jul 02 21:41:18.172934 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:18.172903 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h7mzx\" (UID: \"2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h7mzx" Jul 02 21:41:18.172934 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:18.172927 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h7mzx\" (UID: \"2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h7mzx" Jul 02 21:41:18.173243 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:18.173006 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h7mzx\" (UID: \"2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h7mzx" Jul 02 21:41:18.173243 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:18.173047 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nt7m\" (UniqueName: \"kubernetes.io/projected/2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7-kube-api-access-8nt7m\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h7mzx\" (UID: \"2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h7mzx" Jul 02 21:41:18.173243 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:18.173228 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h7mzx\" (UID: \"2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h7mzx" Jul 02 21:41:18.173397 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:18.173266 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h7mzx\" (UID: \"2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h7mzx" Jul 02 21:41:18.173602 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:18.173580 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h7mzx\" (UID: \"2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h7mzx" Jul 02 21:41:18.174031 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:18.173624 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h7mzx\" (UID: \"2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h7mzx" Jul 02 21:41:18.174150 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:18.173777 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h7mzx\" (UID: \"2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h7mzx" Jul 02 21:41:18.175602 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:18.175578 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h7mzx\" (UID: \"2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h7mzx" Jul 02 21:41:18.176093 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:18.176071 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h7mzx\" (UID: \"2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h7mzx" Jul 02 21:41:18.209097 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:18.209054 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h7mzx\" (UID: \"2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h7mzx" Jul 02 21:41:18.218013 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:18.217984 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nt7m\" (UniqueName: \"kubernetes.io/projected/2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7-kube-api-access-8nt7m\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h7mzx\" (UID: \"2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h7mzx" Jul 02 21:41:18.288928 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:18.288894 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h7mzx" Jul 02 21:41:18.428080 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:18.428054 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zqg5d" Jul 02 21:41:18.441775 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:18.441753 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h7mzx"] Jul 02 21:41:18.444323 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:41:18.444297 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c11f1ef_e705_438a_b5d2_0ac53d5aa4e7.slice/crio-a9f217f9280906e966977eb532868752577d3e6f8e92b6a1e118047ae3ca5bc1 WatchSource:0}: Error finding container a9f217f9280906e966977eb532868752577d3e6f8e92b6a1e118047ae3ca5bc1: Status 404 returned error can't find the container with id a9f217f9280906e966977eb532868752577d3e6f8e92b6a1e118047ae3ca5bc1 Jul 02 21:41:19.432850 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:19.432815 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h7mzx" event={"ID":"2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7","Type":"ContainerStarted","Data":"a9f217f9280906e966977eb532868752577d3e6f8e92b6a1e118047ae3ca5bc1"} Jul 02 21:41:19.914769 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:19.914731 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-8hwkh"] Jul 02 21:41:19.918583 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:19.918557 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8hwkh" Jul 02 21:41:19.920925 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:19.920900 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 02 21:41:19.921074 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:19.921049 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-6c6pw\"" Jul 02 21:41:19.926384 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:19.926127 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-8gxdc"] Jul 02 21:41:19.931641 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:19.931616 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-8gxdc" Jul 02 21:41:19.933386 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:19.933331 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-8hwkh"] Jul 02 21:41:19.934481 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:19.934459 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 02 21:41:19.934583 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:19.934480 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-vrzjh\"" Jul 02 21:41:19.940700 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:19.940680 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-8gxdc"] Jul 02 21:41:19.992571 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:19.992534 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lh4l\" (UniqueName: \"kubernetes.io/projected/c970199b-3cae-49ac-a593-9b35bcad6095-kube-api-access-6lh4l\") pod \"kserve-controller-manager-d6f4b65b8-8hwkh\" (UID: \"c970199b-3cae-49ac-a593-9b35bcad6095\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8hwkh" Jul 02 21:41:19.992747 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:19.992600 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpnx5\" (UniqueName: \"kubernetes.io/projected/8e629e4a-cfed-4f2f-aeeb-10ffc9116325-kube-api-access-mpnx5\") pod \"llmisvc-controller-manager-777c6bd7d5-8gxdc\" (UID: \"8e629e4a-cfed-4f2f-aeeb-10ffc9116325\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-8gxdc" Jul 02 21:41:19.992747 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:19.992654 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c970199b-3cae-49ac-a593-9b35bcad6095-cert\") pod \"kserve-controller-manager-d6f4b65b8-8hwkh\" (UID: \"c970199b-3cae-49ac-a593-9b35bcad6095\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8hwkh" Jul 02 21:41:19.992747 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:19.992713 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8e629e4a-cfed-4f2f-aeeb-10ffc9116325-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-8gxdc\" (UID: \"8e629e4a-cfed-4f2f-aeeb-10ffc9116325\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-8gxdc" Jul 02 21:41:20.093739 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:20.093704 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lh4l\" (UniqueName: \"kubernetes.io/projected/c970199b-3cae-49ac-a593-9b35bcad6095-kube-api-access-6lh4l\") pod \"kserve-controller-manager-d6f4b65b8-8hwkh\" (UID: \"c970199b-3cae-49ac-a593-9b35bcad6095\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8hwkh" Jul 02 21:41:20.093988 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:20.093764 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpnx5\" (UniqueName: \"kubernetes.io/projected/8e629e4a-cfed-4f2f-aeeb-10ffc9116325-kube-api-access-mpnx5\") pod \"llmisvc-controller-manager-777c6bd7d5-8gxdc\" (UID: \"8e629e4a-cfed-4f2f-aeeb-10ffc9116325\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-8gxdc" Jul 02 21:41:20.093988 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:20.093815 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c970199b-3cae-49ac-a593-9b35bcad6095-cert\") pod \"kserve-controller-manager-d6f4b65b8-8hwkh\" (UID: \"c970199b-3cae-49ac-a593-9b35bcad6095\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8hwkh" Jul 02 21:41:20.093988 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:20.093863 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8e629e4a-cfed-4f2f-aeeb-10ffc9116325-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-8gxdc\" (UID: \"8e629e4a-cfed-4f2f-aeeb-10ffc9116325\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-8gxdc" Jul 02 21:41:20.093988 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:41:20.093975 2569 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 02 21:41:20.094216 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:41:20.094047 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c970199b-3cae-49ac-a593-9b35bcad6095-cert podName:c970199b-3cae-49ac-a593-9b35bcad6095 nodeName:}" failed. No retries permitted until 2026-07-02 21:41:20.594026626 +0000 UTC m=+319.932997059 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c970199b-3cae-49ac-a593-9b35bcad6095-cert") pod "kserve-controller-manager-d6f4b65b8-8hwkh" (UID: "c970199b-3cae-49ac-a593-9b35bcad6095") : secret "kserve-webhook-server-cert" not found Jul 02 21:41:20.094216 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:41:20.093981 2569 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 02 21:41:20.094216 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:41:20.094133 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e629e4a-cfed-4f2f-aeeb-10ffc9116325-cert podName:8e629e4a-cfed-4f2f-aeeb-10ffc9116325 nodeName:}" failed. No retries permitted until 2026-07-02 21:41:20.594114098 +0000 UTC m=+319.933084517 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/8e629e4a-cfed-4f2f-aeeb-10ffc9116325-cert") pod "llmisvc-controller-manager-777c6bd7d5-8gxdc" (UID: "8e629e4a-cfed-4f2f-aeeb-10ffc9116325") : secret "llmisvc-webhook-server-cert" not found Jul 02 21:41:20.107967 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:20.107937 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpnx5\" (UniqueName: \"kubernetes.io/projected/8e629e4a-cfed-4f2f-aeeb-10ffc9116325-kube-api-access-mpnx5\") pod \"llmisvc-controller-manager-777c6bd7d5-8gxdc\" (UID: \"8e629e4a-cfed-4f2f-aeeb-10ffc9116325\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-8gxdc" Jul 02 21:41:20.108458 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:20.108433 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lh4l\" (UniqueName: \"kubernetes.io/projected/c970199b-3cae-49ac-a593-9b35bcad6095-kube-api-access-6lh4l\") pod \"kserve-controller-manager-d6f4b65b8-8hwkh\" (UID: \"c970199b-3cae-49ac-a593-9b35bcad6095\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8hwkh" Jul 02 21:41:20.599256 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:20.599224 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c970199b-3cae-49ac-a593-9b35bcad6095-cert\") pod \"kserve-controller-manager-d6f4b65b8-8hwkh\" (UID: \"c970199b-3cae-49ac-a593-9b35bcad6095\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8hwkh" Jul 02 21:41:20.599629 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:20.599270 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8e629e4a-cfed-4f2f-aeeb-10ffc9116325-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-8gxdc\" (UID: \"8e629e4a-cfed-4f2f-aeeb-10ffc9116325\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-8gxdc" Jul 02 21:41:20.599629 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:41:20.599373 2569 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 02 21:41:20.599629 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:41:20.599439 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c970199b-3cae-49ac-a593-9b35bcad6095-cert podName:c970199b-3cae-49ac-a593-9b35bcad6095 nodeName:}" failed. No retries permitted until 2026-07-02 21:41:21.59942352 +0000 UTC m=+320.938393934 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c970199b-3cae-49ac-a593-9b35bcad6095-cert") pod "kserve-controller-manager-d6f4b65b8-8hwkh" (UID: "c970199b-3cae-49ac-a593-9b35bcad6095") : secret "kserve-webhook-server-cert" not found Jul 02 21:41:20.601706 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:20.601680 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8e629e4a-cfed-4f2f-aeeb-10ffc9116325-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-8gxdc\" (UID: \"8e629e4a-cfed-4f2f-aeeb-10ffc9116325\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-8gxdc" Jul 02 21:41:20.848455 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:20.848421 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-8gxdc" Jul 02 21:41:20.994626 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:20.994600 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-8gxdc"] Jul 02 21:41:20.997863 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:41:20.997833 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod8e629e4a_cfed_4f2f_aeeb_10ffc9116325.slice/crio-ddf0698f30d8de910abecc781eaae2fd231724153bd6ae00b325d82371705ece WatchSource:0}: Error finding container ddf0698f30d8de910abecc781eaae2fd231724153bd6ae00b325d82371705ece: Status 404 returned error can't find the container with id ddf0698f30d8de910abecc781eaae2fd231724153bd6ae00b325d82371705ece Jul 02 21:41:21.052861 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:21.052821 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-zqg5d"] Jul 02 21:41:21.442367 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:21.442332 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-8gxdc" event={"ID":"8e629e4a-cfed-4f2f-aeeb-10ffc9116325","Type":"ContainerStarted","Data":"ddf0698f30d8de910abecc781eaae2fd231724153bd6ae00b325d82371705ece"} Jul 02 21:41:21.442584 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:21.442467 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zqg5d" podUID="07c52afa-aaed-4f8f-996b-ad240001f420" containerName="discovery" containerID="cri-o://a829c02eb06ae2d584fac0250faacaf1d6d45db6f0b86819fe15fb988c55b5e5" gracePeriod=30 Jul 02 21:41:21.609196 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:21.609128 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c970199b-3cae-49ac-a593-9b35bcad6095-cert\") pod \"kserve-controller-manager-d6f4b65b8-8hwkh\" (UID: \"c970199b-3cae-49ac-a593-9b35bcad6095\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8hwkh" Jul 02 21:41:21.611953 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:21.611885 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c970199b-3cae-49ac-a593-9b35bcad6095-cert\") pod \"kserve-controller-manager-d6f4b65b8-8hwkh\" (UID: \"c970199b-3cae-49ac-a593-9b35bcad6095\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8hwkh" Jul 02 21:41:21.701543 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:21.701483 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zqg5d" Jul 02 21:41:21.733157 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:21.733124 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8hwkh" Jul 02 21:41:21.811523 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:21.811495 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/07c52afa-aaed-4f8f-996b-ad240001f420-istio-csr-ca-configmap\") pod \"07c52afa-aaed-4f8f-996b-ad240001f420\" (UID: \"07c52afa-aaed-4f8f-996b-ad240001f420\") " Jul 02 21:41:21.811692 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:21.811553 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07c52afa-aaed-4f8f-996b-ad240001f420-istio-token\") pod \"07c52afa-aaed-4f8f-996b-ad240001f420\" (UID: \"07c52afa-aaed-4f8f-996b-ad240001f420\") " Jul 02 21:41:21.811692 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:21.811584 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/07c52afa-aaed-4f8f-996b-ad240001f420-local-certs\") pod \"07c52afa-aaed-4f8f-996b-ad240001f420\" (UID: \"07c52afa-aaed-4f8f-996b-ad240001f420\") " Jul 02 21:41:21.811692 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:21.811634 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q7ljm\" (UniqueName: \"kubernetes.io/projected/07c52afa-aaed-4f8f-996b-ad240001f420-kube-api-access-q7ljm\") pod \"07c52afa-aaed-4f8f-996b-ad240001f420\" (UID: \"07c52afa-aaed-4f8f-996b-ad240001f420\") " Jul 02 21:41:21.811692 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:21.811664 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/07c52afa-aaed-4f8f-996b-ad240001f420-istio-csr-dns-cert\") pod \"07c52afa-aaed-4f8f-996b-ad240001f420\" (UID: \"07c52afa-aaed-4f8f-996b-ad240001f420\") " Jul 02 21:41:21.811932 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:21.811762 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/07c52afa-aaed-4f8f-996b-ad240001f420-istio-kubeconfig\") pod \"07c52afa-aaed-4f8f-996b-ad240001f420\" (UID: \"07c52afa-aaed-4f8f-996b-ad240001f420\") " Jul 02 21:41:21.811932 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:21.811818 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/07c52afa-aaed-4f8f-996b-ad240001f420-cacerts\") pod \"07c52afa-aaed-4f8f-996b-ad240001f420\" (UID: \"07c52afa-aaed-4f8f-996b-ad240001f420\") " Jul 02 21:41:21.812030 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:21.811936 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07c52afa-aaed-4f8f-996b-ad240001f420-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "07c52afa-aaed-4f8f-996b-ad240001f420" (UID: "07c52afa-aaed-4f8f-996b-ad240001f420"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 21:41:21.812143 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:21.812123 2569 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/07c52afa-aaed-4f8f-996b-ad240001f420-istio-csr-ca-configmap\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 21:41:21.814096 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:21.814058 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07c52afa-aaed-4f8f-996b-ad240001f420-local-certs" (OuterVolumeSpecName: "local-certs") pod "07c52afa-aaed-4f8f-996b-ad240001f420" (UID: "07c52afa-aaed-4f8f-996b-ad240001f420"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 21:41:21.814216 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:21.814101 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07c52afa-aaed-4f8f-996b-ad240001f420-cacerts" (OuterVolumeSpecName: "cacerts") pod "07c52afa-aaed-4f8f-996b-ad240001f420" (UID: "07c52afa-aaed-4f8f-996b-ad240001f420"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 21:41:21.814283 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:21.814228 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07c52afa-aaed-4f8f-996b-ad240001f420-kube-api-access-q7ljm" (OuterVolumeSpecName: "kube-api-access-q7ljm") pod "07c52afa-aaed-4f8f-996b-ad240001f420" (UID: "07c52afa-aaed-4f8f-996b-ad240001f420"). InnerVolumeSpecName "kube-api-access-q7ljm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 21:41:21.814465 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:21.814417 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07c52afa-aaed-4f8f-996b-ad240001f420-istio-token" (OuterVolumeSpecName: "istio-token") pod "07c52afa-aaed-4f8f-996b-ad240001f420" (UID: "07c52afa-aaed-4f8f-996b-ad240001f420"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 21:41:21.815679 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:21.815659 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07c52afa-aaed-4f8f-996b-ad240001f420-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "07c52afa-aaed-4f8f-996b-ad240001f420" (UID: "07c52afa-aaed-4f8f-996b-ad240001f420"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 21:41:21.815906 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:21.815884 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07c52afa-aaed-4f8f-996b-ad240001f420-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "07c52afa-aaed-4f8f-996b-ad240001f420" (UID: "07c52afa-aaed-4f8f-996b-ad240001f420"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 21:41:21.912953 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:21.912914 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q7ljm\" (UniqueName: \"kubernetes.io/projected/07c52afa-aaed-4f8f-996b-ad240001f420-kube-api-access-q7ljm\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 21:41:21.912953 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:21.912950 2569 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/07c52afa-aaed-4f8f-996b-ad240001f420-istio-csr-dns-cert\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 21:41:21.913165 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:21.912965 2569 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/07c52afa-aaed-4f8f-996b-ad240001f420-istio-kubeconfig\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 21:41:21.913165 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:21.912979 2569 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/07c52afa-aaed-4f8f-996b-ad240001f420-cacerts\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 21:41:21.913165 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:21.912992 2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07c52afa-aaed-4f8f-996b-ad240001f420-istio-token\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 21:41:21.913165 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:21.913003 2569 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/07c52afa-aaed-4f8f-996b-ad240001f420-local-certs\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 21:41:22.448450 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:22.448411 2569 generic.go:356] "Generic (PLEG): container finished" podID="07c52afa-aaed-4f8f-996b-ad240001f420" containerID="a829c02eb06ae2d584fac0250faacaf1d6d45db6f0b86819fe15fb988c55b5e5" exitCode=0 Jul 02 21:41:22.448618 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:22.448462 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zqg5d" Jul 02 21:41:22.448618 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:22.448496 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zqg5d" event={"ID":"07c52afa-aaed-4f8f-996b-ad240001f420","Type":"ContainerDied","Data":"a829c02eb06ae2d584fac0250faacaf1d6d45db6f0b86819fe15fb988c55b5e5"} Jul 02 21:41:22.448618 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:22.448560 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zqg5d" event={"ID":"07c52afa-aaed-4f8f-996b-ad240001f420","Type":"ContainerDied","Data":"9c786ac4a73d99902054650aacb8d723eb579b402521558f5cff0e9274707ee3"} Jul 02 21:41:22.448618 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:22.448586 2569 scope.go:117] "RemoveContainer" containerID="a829c02eb06ae2d584fac0250faacaf1d6d45db6f0b86819fe15fb988c55b5e5" Jul 02 21:41:22.471535 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:22.471505 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-zqg5d"] Jul 02 21:41:22.476091 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:22.476066 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-zqg5d"] Jul 02 21:41:22.961589 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:22.961552 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07c52afa-aaed-4f8f-996b-ad240001f420" path="/var/lib/kubelet/pods/07c52afa-aaed-4f8f-996b-ad240001f420/volumes" Jul 02 21:41:23.274946 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:23.274918 2569 scope.go:117] "RemoveContainer" containerID="a829c02eb06ae2d584fac0250faacaf1d6d45db6f0b86819fe15fb988c55b5e5" Jul 02 21:41:23.275379 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:41:23.275354 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a829c02eb06ae2d584fac0250faacaf1d6d45db6f0b86819fe15fb988c55b5e5\": container with ID starting with a829c02eb06ae2d584fac0250faacaf1d6d45db6f0b86819fe15fb988c55b5e5 not found: ID does not exist" containerID="a829c02eb06ae2d584fac0250faacaf1d6d45db6f0b86819fe15fb988c55b5e5" Jul 02 21:41:23.275496 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:23.275386 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a829c02eb06ae2d584fac0250faacaf1d6d45db6f0b86819fe15fb988c55b5e5"} err="failed to get container status \"a829c02eb06ae2d584fac0250faacaf1d6d45db6f0b86819fe15fb988c55b5e5\": rpc error: code = NotFound desc = could not find container \"a829c02eb06ae2d584fac0250faacaf1d6d45db6f0b86819fe15fb988c55b5e5\": container with ID starting with a829c02eb06ae2d584fac0250faacaf1d6d45db6f0b86819fe15fb988c55b5e5 not found: ID does not exist" Jul 02 21:41:23.428939 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:23.428891 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Jul 02 21:41:23.429085 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:23.428971 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Jul 02 21:41:23.429085 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:23.429004 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Jul 02 21:41:23.445811 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:23.440086 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-8hwkh"] Jul 02 21:41:23.477515 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:41:23.477481 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc970199b_3cae_49ac_a593_9b35bcad6095.slice/crio-161a54d1f5da8bd7d09a742f0bf2c2a23dd664e68e1640b7e360293bbc037462 WatchSource:0}: Error finding container 161a54d1f5da8bd7d09a742f0bf2c2a23dd664e68e1640b7e360293bbc037462: Status 404 returned error can't find the container with id 161a54d1f5da8bd7d09a742f0bf2c2a23dd664e68e1640b7e360293bbc037462 Jul 02 21:41:24.460568 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:24.460528 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h7mzx" event={"ID":"2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7","Type":"ContainerStarted","Data":"002b8cf19b9284104c90bd82e5e1235c99f68cdf2500d504d1f253fbd1e879c2"} Jul 02 21:41:24.461982 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:24.461947 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8hwkh" event={"ID":"c970199b-3cae-49ac-a593-9b35bcad6095","Type":"ContainerStarted","Data":"161a54d1f5da8bd7d09a742f0bf2c2a23dd664e68e1640b7e360293bbc037462"} Jul 02 21:41:24.481841 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:24.481771 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h7mzx" podStartSLOduration=2.499921938 podStartE2EDuration="7.481754565s" totalImagesPullingTime="4.981832627s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:41:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:41:24.480489073 +0000 UTC m=+323.819459545" watchObservedRunningTime="2026-07-02 21:41:24.481754565 +0000 UTC m=+323.820724999" Jul 02 21:41:25.289300 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:25.289265 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h7mzx" Jul 02 21:41:25.295103 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:25.295073 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h7mzx" Jul 02 21:41:25.468646 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:25.468608 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-8gxdc" event={"ID":"8e629e4a-cfed-4f2f-aeeb-10ffc9116325","Type":"ContainerStarted","Data":"0e9ff049642580f5e640cf506788f2510342b47f348ff2704e9a5b2afc3c9ae5"} Jul 02 21:41:25.469101 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:25.468782 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-8gxdc" Jul 02 21:41:25.469101 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:25.468987 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h7mzx" Jul 02 21:41:25.470030 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:25.470012 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h7mzx" Jul 02 21:41:25.500903 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:25.500849 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-8gxdc" podStartSLOduration=3.15272545 podStartE2EDuration="6.500834803s" totalImagesPullingTime="3.348109353s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:41:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:41:25.498199626 +0000 UTC m=+324.837170061" watchObservedRunningTime="2026-07-02 21:41:25.500834803 +0000 UTC m=+324.839805237" Jul 02 21:41:26.414673 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:26.414642 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-qsvm7" Jul 02 21:41:26.417036 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:26.416974 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-94g9b" Jul 02 21:41:27.482979 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:27.482941 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8hwkh" event={"ID":"c970199b-3cae-49ac-a593-9b35bcad6095","Type":"ContainerStarted","Data":"a3f58f417375fe9268801435eb4473f7b3c9f223b148fbe5b2ab5af8a606318b"} Jul 02 21:41:27.483347 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:27.483216 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8hwkh" Jul 02 21:41:27.504195 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:27.504146 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8hwkh" podStartSLOduration=5.336024706 podStartE2EDuration="8.504131419s" totalImagesPullingTime="3.168106713s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:41:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:41:27.501944465 +0000 UTC m=+326.840914900" watchObservedRunningTime="2026-07-02 21:41:27.504131419 +0000 UTC m=+326.843101854" Jul 02 21:41:56.479865 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:56.479769 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-8gxdc" Jul 02 21:41:58.497373 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:41:58.497342 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8hwkh" Jul 02 21:42:10.026987 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:10.026947 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-zm2wf"] Jul 02 21:42:10.027344 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:10.027337 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07c52afa-aaed-4f8f-996b-ad240001f420" containerName="discovery" Jul 02 21:42:10.027384 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:10.027349 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="07c52afa-aaed-4f8f-996b-ad240001f420" containerName="discovery" Jul 02 21:42:10.027444 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:10.027431 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="07c52afa-aaed-4f8f-996b-ad240001f420" containerName="discovery" Jul 02 21:42:10.035708 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:10.035683 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-zm2wf" Jul 02 21:42:10.039656 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:10.039629 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 02 21:42:10.039810 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:10.039629 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-dnqvx\"" Jul 02 21:42:10.039939 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:10.039920 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 02 21:42:10.041361 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:10.041339 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-zm2wf"] Jul 02 21:42:10.127903 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:10.127875 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k645c\" (UniqueName: \"kubernetes.io/projected/11d4930a-0af8-46f1-99f7-e4ca35a9710e-kube-api-access-k645c\") pod \"kuadrant-operator-catalog-zm2wf\" (UID: \"11d4930a-0af8-46f1-99f7-e4ca35a9710e\") " pod="kuadrant-system/kuadrant-operator-catalog-zm2wf" Jul 02 21:42:10.228805 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:10.228758 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k645c\" (UniqueName: \"kubernetes.io/projected/11d4930a-0af8-46f1-99f7-e4ca35a9710e-kube-api-access-k645c\") pod \"kuadrant-operator-catalog-zm2wf\" (UID: \"11d4930a-0af8-46f1-99f7-e4ca35a9710e\") " pod="kuadrant-system/kuadrant-operator-catalog-zm2wf" Jul 02 21:42:10.237455 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:10.237423 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k645c\" (UniqueName: \"kubernetes.io/projected/11d4930a-0af8-46f1-99f7-e4ca35a9710e-kube-api-access-k645c\") pod \"kuadrant-operator-catalog-zm2wf\" (UID: \"11d4930a-0af8-46f1-99f7-e4ca35a9710e\") " pod="kuadrant-system/kuadrant-operator-catalog-zm2wf" Jul 02 21:42:10.347060 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:10.346997 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-zm2wf" Jul 02 21:42:10.385367 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:10.385333 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-zm2wf"] Jul 02 21:42:10.473745 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:10.473716 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-zm2wf"] Jul 02 21:42:10.476800 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:42:10.476761 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11d4930a_0af8_46f1_99f7_e4ca35a9710e.slice/crio-3d6976d2115dd5fede6ec14082826e11d109735fc0de700389033e48d6cc4cf0 WatchSource:0}: Error finding container 3d6976d2115dd5fede6ec14082826e11d109735fc0de700389033e48d6cc4cf0: Status 404 returned error can't find the container with id 3d6976d2115dd5fede6ec14082826e11d109735fc0de700389033e48d6cc4cf0 Jul 02 21:42:10.595961 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:10.595932 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hqzzt"] Jul 02 21:42:10.600200 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:10.600158 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-hqzzt" Jul 02 21:42:10.608240 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:10.608218 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hqzzt"] Jul 02 21:42:10.631923 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:10.631898 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9d6zn\" (UniqueName: \"kubernetes.io/projected/957557fc-7241-4c3d-8c56-88abb6b4139d-kube-api-access-9d6zn\") pod \"kuadrant-operator-catalog-hqzzt\" (UID: \"957557fc-7241-4c3d-8c56-88abb6b4139d\") " pod="kuadrant-system/kuadrant-operator-catalog-hqzzt" Jul 02 21:42:10.678220 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:10.678197 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-zm2wf" event={"ID":"11d4930a-0af8-46f1-99f7-e4ca35a9710e","Type":"ContainerStarted","Data":"3d6976d2115dd5fede6ec14082826e11d109735fc0de700389033e48d6cc4cf0"} Jul 02 21:42:10.733220 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:10.733194 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9d6zn\" (UniqueName: \"kubernetes.io/projected/957557fc-7241-4c3d-8c56-88abb6b4139d-kube-api-access-9d6zn\") pod \"kuadrant-operator-catalog-hqzzt\" (UID: \"957557fc-7241-4c3d-8c56-88abb6b4139d\") " pod="kuadrant-system/kuadrant-operator-catalog-hqzzt" Jul 02 21:42:10.743064 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:10.743042 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9d6zn\" (UniqueName: \"kubernetes.io/projected/957557fc-7241-4c3d-8c56-88abb6b4139d-kube-api-access-9d6zn\") pod \"kuadrant-operator-catalog-hqzzt\" (UID: \"957557fc-7241-4c3d-8c56-88abb6b4139d\") " pod="kuadrant-system/kuadrant-operator-catalog-hqzzt" Jul 02 21:42:10.909700 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:10.909621 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-hqzzt" Jul 02 21:42:11.048100 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:11.048072 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hqzzt"] Jul 02 21:42:11.049616 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:42:11.049590 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod957557fc_7241_4c3d_8c56_88abb6b4139d.slice/crio-908b32664f90734d9b0e0e2c165d296766256cf9edc45507cf49e1aab919174c WatchSource:0}: Error finding container 908b32664f90734d9b0e0e2c165d296766256cf9edc45507cf49e1aab919174c: Status 404 returned error can't find the container with id 908b32664f90734d9b0e0e2c165d296766256cf9edc45507cf49e1aab919174c Jul 02 21:42:11.685864 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:11.685828 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-hqzzt" event={"ID":"957557fc-7241-4c3d-8c56-88abb6b4139d","Type":"ContainerStarted","Data":"908b32664f90734d9b0e0e2c165d296766256cf9edc45507cf49e1aab919174c"} Jul 02 21:42:12.691715 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:12.691676 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-zm2wf" event={"ID":"11d4930a-0af8-46f1-99f7-e4ca35a9710e","Type":"ContainerStarted","Data":"9f3e0983da2b3c924ff5fb3cd9a1a192e86a1dcf810ada97b9a99f5db2e14820"} Jul 02 21:42:12.692144 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:12.691724 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-zm2wf" podUID="11d4930a-0af8-46f1-99f7-e4ca35a9710e" containerName="registry-server" containerID="cri-o://9f3e0983da2b3c924ff5fb3cd9a1a192e86a1dcf810ada97b9a99f5db2e14820" gracePeriod=2 Jul 02 21:42:12.693296 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:12.693271 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-hqzzt" event={"ID":"957557fc-7241-4c3d-8c56-88abb6b4139d","Type":"ContainerStarted","Data":"35e0f78c4c29b8026d538ceb9a9ffb22569caac25f289154d708c7febc87cb48"} Jul 02 21:42:12.712750 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:12.712708 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-zm2wf" podStartSLOduration=0.612182176 podStartE2EDuration="2.712693893s" totalImagesPullingTime="2.100511717s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:42:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:42:12.710316146 +0000 UTC m=+372.049286584" watchObservedRunningTime="2026-07-02 21:42:12.712693893 +0000 UTC m=+372.051664336" Jul 02 21:42:12.727634 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:12.727594 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-hqzzt" podStartSLOduration=1.199616301 podStartE2EDuration="2.727580221s" totalImagesPullingTime="1.52796392s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:42:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:42:12.726858898 +0000 UTC m=+372.065829333" watchObservedRunningTime="2026-07-02 21:42:12.727580221 +0000 UTC m=+372.066550660" Jul 02 21:42:12.933927 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:12.933896 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-zm2wf_11d4930a-0af8-46f1-99f7-e4ca35a9710e/registry-server/0.log" Jul 02 21:42:12.936166 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:12.936147 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-zm2wf" Jul 02 21:42:13.056900 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:13.056863 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k645c\" (UniqueName: \"kubernetes.io/projected/11d4930a-0af8-46f1-99f7-e4ca35a9710e-kube-api-access-k645c\") pod \"11d4930a-0af8-46f1-99f7-e4ca35a9710e\" (UID: \"11d4930a-0af8-46f1-99f7-e4ca35a9710e\") " Jul 02 21:42:13.059291 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:13.059258 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11d4930a-0af8-46f1-99f7-e4ca35a9710e-kube-api-access-k645c" (OuterVolumeSpecName: "kube-api-access-k645c") pod "11d4930a-0af8-46f1-99f7-e4ca35a9710e" (UID: "11d4930a-0af8-46f1-99f7-e4ca35a9710e"). InnerVolumeSpecName "kube-api-access-k645c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 21:42:13.157947 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:13.157851 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k645c\" (UniqueName: \"kubernetes.io/projected/11d4930a-0af8-46f1-99f7-e4ca35a9710e-kube-api-access-k645c\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 21:42:13.699582 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:13.699554 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-zm2wf_11d4930a-0af8-46f1-99f7-e4ca35a9710e/registry-server/0.log" Jul 02 21:42:13.700045 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:13.699594 2569 generic.go:356] "Generic (PLEG): container finished" podID="11d4930a-0af8-46f1-99f7-e4ca35a9710e" containerID="9f3e0983da2b3c924ff5fb3cd9a1a192e86a1dcf810ada97b9a99f5db2e14820" exitCode=1 Jul 02 21:42:13.700045 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:13.699657 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-zm2wf" Jul 02 21:42:13.700045 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:13.699695 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-zm2wf" event={"ID":"11d4930a-0af8-46f1-99f7-e4ca35a9710e","Type":"ContainerDied","Data":"9f3e0983da2b3c924ff5fb3cd9a1a192e86a1dcf810ada97b9a99f5db2e14820"} Jul 02 21:42:13.700045 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:13.699753 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-zm2wf" event={"ID":"11d4930a-0af8-46f1-99f7-e4ca35a9710e","Type":"ContainerDied","Data":"3d6976d2115dd5fede6ec14082826e11d109735fc0de700389033e48d6cc4cf0"} Jul 02 21:42:13.700045 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:13.699779 2569 scope.go:117] "RemoveContainer" containerID="9f3e0983da2b3c924ff5fb3cd9a1a192e86a1dcf810ada97b9a99f5db2e14820" Jul 02 21:42:13.709627 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:13.709608 2569 scope.go:117] "RemoveContainer" containerID="9f3e0983da2b3c924ff5fb3cd9a1a192e86a1dcf810ada97b9a99f5db2e14820" Jul 02 21:42:13.709961 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:42:13.709938 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f3e0983da2b3c924ff5fb3cd9a1a192e86a1dcf810ada97b9a99f5db2e14820\": container with ID starting with 9f3e0983da2b3c924ff5fb3cd9a1a192e86a1dcf810ada97b9a99f5db2e14820 not found: ID does not exist" containerID="9f3e0983da2b3c924ff5fb3cd9a1a192e86a1dcf810ada97b9a99f5db2e14820" Jul 02 21:42:13.710020 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:13.709975 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f3e0983da2b3c924ff5fb3cd9a1a192e86a1dcf810ada97b9a99f5db2e14820"} err="failed to get container status \"9f3e0983da2b3c924ff5fb3cd9a1a192e86a1dcf810ada97b9a99f5db2e14820\": rpc error: code = NotFound desc = could not find container \"9f3e0983da2b3c924ff5fb3cd9a1a192e86a1dcf810ada97b9a99f5db2e14820\": container with ID starting with 9f3e0983da2b3c924ff5fb3cd9a1a192e86a1dcf810ada97b9a99f5db2e14820 not found: ID does not exist" Jul 02 21:42:13.720597 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:13.720576 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-zm2wf"] Jul 02 21:42:13.722837 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:13.722812 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-zm2wf"] Jul 02 21:42:14.955969 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:14.955938 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11d4930a-0af8-46f1-99f7-e4ca35a9710e" path="/var/lib/kubelet/pods/11d4930a-0af8-46f1-99f7-e4ca35a9710e/volumes" Jul 02 21:42:20.910340 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:20.910308 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-hqzzt" Jul 02 21:42:20.910726 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:20.910363 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-hqzzt" Jul 02 21:42:20.932817 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:20.932781 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-hqzzt" Jul 02 21:42:21.757328 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:21.757301 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-hqzzt" Jul 02 21:42:43.579853 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:43.579819 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-2dl5k"] Jul 02 21:42:43.580301 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:43.580203 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11d4930a-0af8-46f1-99f7-e4ca35a9710e" containerName="registry-server" Jul 02 21:42:43.580301 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:43.580214 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="11d4930a-0af8-46f1-99f7-e4ca35a9710e" containerName="registry-server" Jul 02 21:42:43.580301 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:43.580272 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="11d4930a-0af8-46f1-99f7-e4ca35a9710e" containerName="registry-server" Jul 02 21:42:43.587669 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:43.587651 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-2dl5k" Jul 02 21:42:43.591392 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:43.591365 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-nznmf\"" Jul 02 21:42:43.597052 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:43.597027 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-2dl5k"] Jul 02 21:42:43.622686 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:43.622656 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkzwh\" (UniqueName: \"kubernetes.io/projected/d9dd6b60-ceea-4092-93b6-98845a1d3974-kube-api-access-mkzwh\") pod \"authorino-operator-75cc8d5cb7-2dl5k\" (UID: \"d9dd6b60-ceea-4092-93b6-98845a1d3974\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-2dl5k" Jul 02 21:42:43.723402 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:43.723375 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkzwh\" (UniqueName: \"kubernetes.io/projected/d9dd6b60-ceea-4092-93b6-98845a1d3974-kube-api-access-mkzwh\") pod \"authorino-operator-75cc8d5cb7-2dl5k\" (UID: \"d9dd6b60-ceea-4092-93b6-98845a1d3974\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-2dl5k" Jul 02 21:42:43.736186 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:43.736167 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkzwh\" (UniqueName: \"kubernetes.io/projected/d9dd6b60-ceea-4092-93b6-98845a1d3974-kube-api-access-mkzwh\") pod \"authorino-operator-75cc8d5cb7-2dl5k\" (UID: \"d9dd6b60-ceea-4092-93b6-98845a1d3974\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-2dl5k" Jul 02 21:42:43.898088 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:43.898024 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-2dl5k" Jul 02 21:42:44.027178 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:44.027151 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-2dl5k"] Jul 02 21:42:44.030972 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:42:44.030945 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9dd6b60_ceea_4092_93b6_98845a1d3974.slice/crio-3068967c39742bff604e20ea320a72ad3b406d5644722c0e67fd4a4f50e98867 WatchSource:0}: Error finding container 3068967c39742bff604e20ea320a72ad3b406d5644722c0e67fd4a4f50e98867: Status 404 returned error can't find the container with id 3068967c39742bff604e20ea320a72ad3b406d5644722c0e67fd4a4f50e98867 Jul 02 21:42:44.842630 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:44.842595 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-2dl5k" event={"ID":"d9dd6b60-ceea-4092-93b6-98845a1d3974","Type":"ContainerStarted","Data":"3068967c39742bff604e20ea320a72ad3b406d5644722c0e67fd4a4f50e98867"} Jul 02 21:42:46.810167 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:46.810130 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-fj4xx"] Jul 02 21:42:46.813644 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:46.813621 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-fj4xx" Jul 02 21:42:46.816513 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:46.816494 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 02 21:42:46.816620 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:46.816509 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-7v94d\"" Jul 02 21:42:46.824015 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:46.823993 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-fj4xx"] Jul 02 21:42:46.843965 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:46.843936 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7qv9\" (UniqueName: \"kubernetes.io/projected/8281e5ad-c64f-4203-826c-4f081a7a2267-kube-api-access-q7qv9\") pod \"dns-operator-controller-manager-864dd78c4c-fj4xx\" (UID: \"8281e5ad-c64f-4203-826c-4f081a7a2267\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-fj4xx" Jul 02 21:42:46.859374 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:46.859344 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-2dl5k" event={"ID":"d9dd6b60-ceea-4092-93b6-98845a1d3974","Type":"ContainerStarted","Data":"5619323605ce4ee1d0e2289f8271731090b3d1ef4bb10d4b6cc9b75bb7e378d5"} Jul 02 21:42:46.859497 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:46.859479 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-2dl5k" Jul 02 21:42:46.881149 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:46.881097 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-2dl5k" podStartSLOduration=1.595557742 podStartE2EDuration="3.881057584s" totalImagesPullingTime="2.285499842s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:42:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:42:46.88063161 +0000 UTC m=+406.219602046" watchObservedRunningTime="2026-07-02 21:42:46.881057584 +0000 UTC m=+406.220028018" Jul 02 21:42:46.945230 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:46.945196 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7qv9\" (UniqueName: \"kubernetes.io/projected/8281e5ad-c64f-4203-826c-4f081a7a2267-kube-api-access-q7qv9\") pod \"dns-operator-controller-manager-864dd78c4c-fj4xx\" (UID: \"8281e5ad-c64f-4203-826c-4f081a7a2267\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-fj4xx" Jul 02 21:42:46.954966 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:46.954942 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7qv9\" (UniqueName: \"kubernetes.io/projected/8281e5ad-c64f-4203-826c-4f081a7a2267-kube-api-access-q7qv9\") pod \"dns-operator-controller-manager-864dd78c4c-fj4xx\" (UID: \"8281e5ad-c64f-4203-826c-4f081a7a2267\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-fj4xx" Jul 02 21:42:47.125006 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:47.124925 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-fj4xx" Jul 02 21:42:47.263211 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:47.263176 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-fj4xx"] Jul 02 21:42:47.266548 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:42:47.266518 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8281e5ad_c64f_4203_826c_4f081a7a2267.slice/crio-3328d2390359366a643de9c953c8abd4c2077d91dc7b497dec1e311d6c3c7eec WatchSource:0}: Error finding container 3328d2390359366a643de9c953c8abd4c2077d91dc7b497dec1e311d6c3c7eec: Status 404 returned error can't find the container with id 3328d2390359366a643de9c953c8abd4c2077d91dc7b497dec1e311d6c3c7eec Jul 02 21:42:47.866137 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:47.866100 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-fj4xx" event={"ID":"8281e5ad-c64f-4203-826c-4f081a7a2267","Type":"ContainerStarted","Data":"3328d2390359366a643de9c953c8abd4c2077d91dc7b497dec1e311d6c3c7eec"} Jul 02 21:42:49.877855 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:49.877808 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-fj4xx" event={"ID":"8281e5ad-c64f-4203-826c-4f081a7a2267","Type":"ContainerStarted","Data":"3e4fb953dbeb62c15305ca6dac5372c48e22dd9f132b4086926828c4936eaf40"} Jul 02 21:42:49.878302 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:49.877885 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-fj4xx" Jul 02 21:42:49.893950 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:49.893903 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-fj4xx" podStartSLOduration=2.093732692 podStartE2EDuration="3.893881048s" totalImagesPullingTime="1.800148356s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:42:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:42:49.892865199 +0000 UTC m=+409.231835635" watchObservedRunningTime="2026-07-02 21:42:49.893881048 +0000 UTC m=+409.232851487" Jul 02 21:42:50.729987 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:50.729952 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-crplr"] Jul 02 21:42:50.733516 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:50.733497 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-crplr" Jul 02 21:42:50.736192 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:50.736178 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-q9lv2\"" Jul 02 21:42:50.748279 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:50.748259 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-crplr"] Jul 02 21:42:50.773390 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:50.773364 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55r5q\" (UniqueName: \"kubernetes.io/projected/f702232d-6b31-409c-8ef5-c2473198f0a5-kube-api-access-55r5q\") pod \"kuadrant-operator-controller-manager-8684c9ff49-crplr\" (UID: \"f702232d-6b31-409c-8ef5-c2473198f0a5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-crplr" Jul 02 21:42:50.773496 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:50.773411 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f702232d-6b31-409c-8ef5-c2473198f0a5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-crplr\" (UID: \"f702232d-6b31-409c-8ef5-c2473198f0a5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-crplr" Jul 02 21:42:50.874431 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:50.874400 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55r5q\" (UniqueName: \"kubernetes.io/projected/f702232d-6b31-409c-8ef5-c2473198f0a5-kube-api-access-55r5q\") pod \"kuadrant-operator-controller-manager-8684c9ff49-crplr\" (UID: \"f702232d-6b31-409c-8ef5-c2473198f0a5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-crplr" Jul 02 21:42:50.874578 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:50.874456 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f702232d-6b31-409c-8ef5-c2473198f0a5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-crplr\" (UID: \"f702232d-6b31-409c-8ef5-c2473198f0a5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-crplr" Jul 02 21:42:50.874867 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:50.874847 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f702232d-6b31-409c-8ef5-c2473198f0a5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-crplr\" (UID: \"f702232d-6b31-409c-8ef5-c2473198f0a5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-crplr" Jul 02 21:42:50.883178 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:50.883156 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-55r5q\" (UniqueName: \"kubernetes.io/projected/f702232d-6b31-409c-8ef5-c2473198f0a5-kube-api-access-55r5q\") pod \"kuadrant-operator-controller-manager-8684c9ff49-crplr\" (UID: \"f702232d-6b31-409c-8ef5-c2473198f0a5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-crplr" Jul 02 21:42:51.043672 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:51.043644 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-crplr" Jul 02 21:42:51.178157 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:51.178128 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-crplr"] Jul 02 21:42:51.181774 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:42:51.181745 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf702232d_6b31_409c_8ef5_c2473198f0a5.slice/crio-191f7a5426f77531445cb5d866ef0dfdfd0e72cb62baa514db72f7290eb1be7e WatchSource:0}: Error finding container 191f7a5426f77531445cb5d866ef0dfdfd0e72cb62baa514db72f7290eb1be7e: Status 404 returned error can't find the container with id 191f7a5426f77531445cb5d866ef0dfdfd0e72cb62baa514db72f7290eb1be7e Jul 02 21:42:51.888857 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:51.888826 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-crplr" event={"ID":"f702232d-6b31-409c-8ef5-c2473198f0a5","Type":"ContainerStarted","Data":"191f7a5426f77531445cb5d866ef0dfdfd0e72cb62baa514db72f7290eb1be7e"} Jul 02 21:42:53.051380 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:53.051345 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s5fsn"] Jul 02 21:42:53.055138 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:53.055112 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s5fsn" Jul 02 21:42:53.060422 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:53.060396 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-td8cg\"" Jul 02 21:42:53.084447 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:53.084424 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s5fsn"] Jul 02 21:42:53.092010 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:53.091983 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgjdr\" (UniqueName: \"kubernetes.io/projected/c5be070d-0a8d-4a83-ae25-c5b4b3f71ad4-kube-api-access-cgjdr\") pod \"limitador-operator-controller-manager-859d7b78bd-s5fsn\" (UID: \"c5be070d-0a8d-4a83-ae25-c5b4b3f71ad4\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s5fsn" Jul 02 21:42:53.193166 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:53.193128 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgjdr\" (UniqueName: \"kubernetes.io/projected/c5be070d-0a8d-4a83-ae25-c5b4b3f71ad4-kube-api-access-cgjdr\") pod \"limitador-operator-controller-manager-859d7b78bd-s5fsn\" (UID: \"c5be070d-0a8d-4a83-ae25-c5b4b3f71ad4\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s5fsn" Jul 02 21:42:53.206608 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:53.206581 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgjdr\" (UniqueName: \"kubernetes.io/projected/c5be070d-0a8d-4a83-ae25-c5b4b3f71ad4-kube-api-access-cgjdr\") pod \"limitador-operator-controller-manager-859d7b78bd-s5fsn\" (UID: \"c5be070d-0a8d-4a83-ae25-c5b4b3f71ad4\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s5fsn" Jul 02 21:42:53.367198 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:53.367121 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s5fsn" Jul 02 21:42:53.503227 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:53.503206 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s5fsn"] Jul 02 21:42:53.506131 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:42:53.506100 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5be070d_0a8d_4a83_ae25_c5b4b3f71ad4.slice/crio-47d9315e8336840ca364cbddc80f92902a8cd6d5d63dbb9db2aefac018320f86 WatchSource:0}: Error finding container 47d9315e8336840ca364cbddc80f92902a8cd6d5d63dbb9db2aefac018320f86: Status 404 returned error can't find the container with id 47d9315e8336840ca364cbddc80f92902a8cd6d5d63dbb9db2aefac018320f86 Jul 02 21:42:53.901186 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:53.901139 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s5fsn" event={"ID":"c5be070d-0a8d-4a83-ae25-c5b4b3f71ad4","Type":"ContainerStarted","Data":"47d9315e8336840ca364cbddc80f92902a8cd6d5d63dbb9db2aefac018320f86"} Jul 02 21:42:55.915726 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:55.915680 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s5fsn" event={"ID":"c5be070d-0a8d-4a83-ae25-c5b4b3f71ad4","Type":"ContainerStarted","Data":"841f5e3c7699874ab5a07c1b942d4f853d66d0b58457383b84f0db94d53f0203"} Jul 02 21:42:55.916229 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:55.915811 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s5fsn" Jul 02 21:42:55.940877 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:55.940822 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s5fsn" podStartSLOduration=1.310765472 podStartE2EDuration="2.940807842s" totalImagesPullingTime="1.63004237s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:42:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:42:55.937648488 +0000 UTC m=+415.276618937" watchObservedRunningTime="2026-07-02 21:42:55.940807842 +0000 UTC m=+415.279778268" Jul 02 21:42:56.922822 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:56.922724 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-crplr" event={"ID":"f702232d-6b31-409c-8ef5-c2473198f0a5","Type":"ContainerStarted","Data":"aa4cfc29c7b934362c952c14a33ead8d9385ba4c208437fec9bd394bf12c370c"} Jul 02 21:42:56.923172 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:56.922838 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-crplr" Jul 02 21:42:56.945087 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:56.945033 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-crplr" podStartSLOduration=1.545820991 podStartE2EDuration="6.945018041s" totalImagesPullingTime="5.39919705s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:42:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:42:56.944011969 +0000 UTC m=+416.282982405" watchObservedRunningTime="2026-07-02 21:42:56.945018041 +0000 UTC m=+416.283988476" Jul 02 21:42:57.868757 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:42:57.868732 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-2dl5k" Jul 02 21:43:00.885168 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:00.885136 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-fj4xx" Jul 02 21:43:06.925082 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:06.925050 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s5fsn" Jul 02 21:43:07.930371 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:07.930338 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-crplr" Jul 02 21:43:09.207180 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:09.207149 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2glnb"] Jul 02 21:43:09.211953 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:09.211935 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2glnb" Jul 02 21:43:09.227466 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:09.227441 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2glnb"] Jul 02 21:43:09.338878 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:09.338845 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7ee26cd8-e7c6-4299-89ff-9c9ba3d3c9e2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-2glnb\" (UID: \"7ee26cd8-e7c6-4299-89ff-9c9ba3d3c9e2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2glnb" Jul 02 21:43:09.339037 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:09.338915 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvhwt\" (UniqueName: \"kubernetes.io/projected/7ee26cd8-e7c6-4299-89ff-9c9ba3d3c9e2-kube-api-access-rvhwt\") pod \"kuadrant-operator-controller-manager-55467d9f7-2glnb\" (UID: \"7ee26cd8-e7c6-4299-89ff-9c9ba3d3c9e2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2glnb" Jul 02 21:43:09.440196 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:09.440167 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7ee26cd8-e7c6-4299-89ff-9c9ba3d3c9e2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-2glnb\" (UID: \"7ee26cd8-e7c6-4299-89ff-9c9ba3d3c9e2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2glnb" Jul 02 21:43:09.440334 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:09.440247 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvhwt\" (UniqueName: \"kubernetes.io/projected/7ee26cd8-e7c6-4299-89ff-9c9ba3d3c9e2-kube-api-access-rvhwt\") pod \"kuadrant-operator-controller-manager-55467d9f7-2glnb\" (UID: \"7ee26cd8-e7c6-4299-89ff-9c9ba3d3c9e2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2glnb" Jul 02 21:43:09.440517 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:09.440496 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7ee26cd8-e7c6-4299-89ff-9c9ba3d3c9e2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-2glnb\" (UID: \"7ee26cd8-e7c6-4299-89ff-9c9ba3d3c9e2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2glnb" Jul 02 21:43:09.455526 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:09.455503 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvhwt\" (UniqueName: \"kubernetes.io/projected/7ee26cd8-e7c6-4299-89ff-9c9ba3d3c9e2-kube-api-access-rvhwt\") pod \"kuadrant-operator-controller-manager-55467d9f7-2glnb\" (UID: \"7ee26cd8-e7c6-4299-89ff-9c9ba3d3c9e2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2glnb" Jul 02 21:43:09.521897 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:09.521872 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2glnb" Jul 02 21:43:09.659946 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:09.659920 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2glnb"] Jul 02 21:43:09.662045 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:43:09.662017 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ee26cd8_e7c6_4299_89ff_9c9ba3d3c9e2.slice/crio-f5c1b547571d83fa32193bce9491454756f745c5d29ee6f0effae96ec26c4053 WatchSource:0}: Error finding container f5c1b547571d83fa32193bce9491454756f745c5d29ee6f0effae96ec26c4053: Status 404 returned error can't find the container with id f5c1b547571d83fa32193bce9491454756f745c5d29ee6f0effae96ec26c4053 Jul 02 21:43:09.796254 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:09.796178 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2glnb"] Jul 02 21:43:09.808979 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:09.808950 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2glnb"] Jul 02 21:43:09.823456 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:09.823433 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-crplr"] Jul 02 21:43:09.823628 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:09.823591 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-crplr" podUID="f702232d-6b31-409c-8ef5-c2473198f0a5" containerName="manager" containerID="cri-o://aa4cfc29c7b934362c952c14a33ead8d9385ba4c208437fec9bd394bf12c370c" gracePeriod=2 Jul 02 21:43:09.829213 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:09.829188 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-crplr"] Jul 02 21:43:09.842301 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:09.842277 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-v2j2d"] Jul 02 21:43:09.842692 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:09.842676 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f702232d-6b31-409c-8ef5-c2473198f0a5" containerName="manager" Jul 02 21:43:09.842766 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:09.842695 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f702232d-6b31-409c-8ef5-c2473198f0a5" containerName="manager" Jul 02 21:43:09.842766 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:09.842745 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ee26cd8-e7c6-4299-89ff-9c9ba3d3c9e2" containerName="manager" Jul 02 21:43:09.842766 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:09.842754 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ee26cd8-e7c6-4299-89ff-9c9ba3d3c9e2" containerName="manager" Jul 02 21:43:09.842952 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:09.842855 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ee26cd8-e7c6-4299-89ff-9c9ba3d3c9e2" containerName="manager" Jul 02 21:43:09.842952 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:09.842870 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="f702232d-6b31-409c-8ef5-c2473198f0a5" containerName="manager" Jul 02 21:43:09.845884 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:09.845866 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-v2j2d" Jul 02 21:43:09.848324 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:43:09.848295 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-crplr\" is forbidden: User \"system:node:ip-10-0-139-75.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-75.ec2.internal' and this object" podUID="f702232d-6b31-409c-8ef5-c2473198f0a5" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-crplr" Jul 02 21:43:09.856430 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:09.856401 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s5fsn"] Jul 02 21:43:09.856581 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:09.856557 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s5fsn" podUID="c5be070d-0a8d-4a83-ae25-c5b4b3f71ad4" containerName="manager" containerID="cri-o://841f5e3c7699874ab5a07c1b942d4f853d66d0b58457383b84f0db94d53f0203" gracePeriod=2 Jul 02 21:43:09.862661 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:09.862644 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-v2j2d"] Jul 02 21:43:09.872471 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:09.872447 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s5fsn"] Jul 02 21:43:09.874152 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:09.874131 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c9jbj"] Jul 02 21:43:09.874553 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:09.874537 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5be070d-0a8d-4a83-ae25-c5b4b3f71ad4" containerName="manager" Jul 02 21:43:09.874628 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:09.874557 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5be070d-0a8d-4a83-ae25-c5b4b3f71ad4" containerName="manager" Jul 02 21:43:09.874722 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:09.874710 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5be070d-0a8d-4a83-ae25-c5b4b3f71ad4" containerName="manager" Jul 02 21:43:09.878014 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:09.877996 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c9jbj" Jul 02 21:43:09.892317 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:09.892294 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c9jbj"] Jul 02 21:43:09.894663 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:09.894638 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vsgpl"] Jul 02 21:43:09.903484 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:09.902855 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vsgpl" Jul 02 21:43:09.918304 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:09.918274 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vsgpl"] Jul 02 21:43:09.931249 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:43:09.931213 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-s5fsn\" is forbidden: User \"system:node:ip-10-0-139-75.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-75.ec2.internal' and this object" podUID="c5be070d-0a8d-4a83-ae25-c5b4b3f71ad4" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s5fsn" Jul 02 21:43:09.933587 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:43:09.933553 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-crplr\" is forbidden: User \"system:node:ip-10-0-139-75.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-75.ec2.internal' and this object" podUID="f702232d-6b31-409c-8ef5-c2473198f0a5" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-crplr" Jul 02 21:43:09.944514 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:09.944459 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prllz\" (UniqueName: \"kubernetes.io/projected/5315e1f1-17e9-4f07-a961-f04ef2d00e64-kube-api-access-prllz\") pod \"limitador-operator-controller-manager-859d7b78bd-vsgpl\" (UID: \"5315e1f1-17e9-4f07-a961-f04ef2d00e64\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vsgpl" Jul 02 21:43:09.944514 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:09.944507 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gwlb\" (UniqueName: \"kubernetes.io/projected/d2cef987-d128-4b1d-b9cf-272159207238-kube-api-access-2gwlb\") pod \"kuadrant-operator-controller-manager-8684c9ff49-c9jbj\" (UID: \"d2cef987-d128-4b1d-b9cf-272159207238\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c9jbj" Jul 02 21:43:09.944687 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:09.944582 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbf5q\" (UniqueName: \"kubernetes.io/projected/efd356c4-9cae-4f32-86b5-d8e7d98b7614-kube-api-access-gbf5q\") pod \"kuadrant-operator-controller-manager-55467d9f7-v2j2d\" (UID: \"efd356c4-9cae-4f32-86b5-d8e7d98b7614\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-v2j2d" Jul 02 21:43:09.944687 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:09.944652 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/efd356c4-9cae-4f32-86b5-d8e7d98b7614-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-v2j2d\" (UID: \"efd356c4-9cae-4f32-86b5-d8e7d98b7614\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-v2j2d" Jul 02 21:43:09.944773 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:09.944696 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d2cef987-d128-4b1d-b9cf-272159207238-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-c9jbj\" (UID: \"d2cef987-d128-4b1d-b9cf-272159207238\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c9jbj" Jul 02 21:43:09.988422 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:09.988391 2569 generic.go:356] "Generic (PLEG): container finished" podID="c5be070d-0a8d-4a83-ae25-c5b4b3f71ad4" containerID="841f5e3c7699874ab5a07c1b942d4f853d66d0b58457383b84f0db94d53f0203" exitCode=0 Jul 02 21:43:09.990362 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:09.990335 2569 generic.go:356] "Generic (PLEG): container finished" podID="f702232d-6b31-409c-8ef5-c2473198f0a5" containerID="aa4cfc29c7b934362c952c14a33ead8d9385ba4c208437fec9bd394bf12c370c" exitCode=0 Jul 02 21:43:09.992728 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:43:09.992704 2569 kuberuntime_manager.go:649] "Missing actuated resource record" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2glnb" container="manager" Jul 02 21:43:09.995754 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:43:09.995687 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-2glnb\" is forbidden: User \"system:node:ip-10-0-139-75.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-75.ec2.internal' and this object" podUID="7ee26cd8-e7c6-4299-89ff-9c9ba3d3c9e2" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2glnb" Jul 02 21:43:09.997839 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:43:09.997807 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-s5fsn\" is forbidden: User \"system:node:ip-10-0-139-75.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-75.ec2.internal' and this object" podUID="c5be070d-0a8d-4a83-ae25-c5b4b3f71ad4" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s5fsn" Jul 02 21:43:09.999794 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:43:09.999763 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-crplr\" is forbidden: User \"system:node:ip-10-0-139-75.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-75.ec2.internal' and this object" podUID="f702232d-6b31-409c-8ef5-c2473198f0a5" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-crplr" Jul 02 21:43:10.045402 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:10.045369 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prllz\" (UniqueName: \"kubernetes.io/projected/5315e1f1-17e9-4f07-a961-f04ef2d00e64-kube-api-access-prllz\") pod \"limitador-operator-controller-manager-859d7b78bd-vsgpl\" (UID: \"5315e1f1-17e9-4f07-a961-f04ef2d00e64\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vsgpl" Jul 02 21:43:10.045522 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:10.045427 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gwlb\" (UniqueName: \"kubernetes.io/projected/d2cef987-d128-4b1d-b9cf-272159207238-kube-api-access-2gwlb\") pod \"kuadrant-operator-controller-manager-8684c9ff49-c9jbj\" (UID: \"d2cef987-d128-4b1d-b9cf-272159207238\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c9jbj" Jul 02 21:43:10.045522 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:10.045458 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbf5q\" (UniqueName: \"kubernetes.io/projected/efd356c4-9cae-4f32-86b5-d8e7d98b7614-kube-api-access-gbf5q\") pod \"kuadrant-operator-controller-manager-55467d9f7-v2j2d\" (UID: \"efd356c4-9cae-4f32-86b5-d8e7d98b7614\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-v2j2d" Jul 02 21:43:10.045522 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:10.045510 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/efd356c4-9cae-4f32-86b5-d8e7d98b7614-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-v2j2d\" (UID: \"efd356c4-9cae-4f32-86b5-d8e7d98b7614\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-v2j2d" Jul 02 21:43:10.045696 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:10.045555 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d2cef987-d128-4b1d-b9cf-272159207238-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-c9jbj\" (UID: \"d2cef987-d128-4b1d-b9cf-272159207238\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c9jbj" Jul 02 21:43:10.046074 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:10.046049 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d2cef987-d128-4b1d-b9cf-272159207238-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-c9jbj\" (UID: \"d2cef987-d128-4b1d-b9cf-272159207238\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c9jbj" Jul 02 21:43:10.046197 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:10.046173 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/efd356c4-9cae-4f32-86b5-d8e7d98b7614-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-v2j2d\" (UID: \"efd356c4-9cae-4f32-86b5-d8e7d98b7614\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-v2j2d" Jul 02 21:43:10.054622 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:10.054562 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbf5q\" (UniqueName: \"kubernetes.io/projected/efd356c4-9cae-4f32-86b5-d8e7d98b7614-kube-api-access-gbf5q\") pod \"kuadrant-operator-controller-manager-55467d9f7-v2j2d\" (UID: \"efd356c4-9cae-4f32-86b5-d8e7d98b7614\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-v2j2d" Jul 02 21:43:10.055016 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:10.054997 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-prllz\" (UniqueName: \"kubernetes.io/projected/5315e1f1-17e9-4f07-a961-f04ef2d00e64-kube-api-access-prllz\") pod \"limitador-operator-controller-manager-859d7b78bd-vsgpl\" (UID: \"5315e1f1-17e9-4f07-a961-f04ef2d00e64\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vsgpl" Jul 02 21:43:10.055577 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:10.055560 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gwlb\" (UniqueName: \"kubernetes.io/projected/d2cef987-d128-4b1d-b9cf-272159207238-kube-api-access-2gwlb\") pod \"kuadrant-operator-controller-manager-8684c9ff49-c9jbj\" (UID: \"d2cef987-d128-4b1d-b9cf-272159207238\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c9jbj" Jul 02 21:43:10.095761 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:10.095742 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-crplr" Jul 02 21:43:10.098204 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:43:10.098173 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-2glnb\" is forbidden: User \"system:node:ip-10-0-139-75.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-75.ec2.internal' and this object" podUID="7ee26cd8-e7c6-4299-89ff-9c9ba3d3c9e2" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2glnb" Jul 02 21:43:10.100123 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:43:10.100095 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-s5fsn\" is forbidden: User \"system:node:ip-10-0-139-75.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-75.ec2.internal' and this object" podUID="c5be070d-0a8d-4a83-ae25-c5b4b3f71ad4" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s5fsn" Jul 02 21:43:10.102063 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:43:10.102043 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-crplr\" is forbidden: User \"system:node:ip-10-0-139-75.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-75.ec2.internal' and this object" podUID="f702232d-6b31-409c-8ef5-c2473198f0a5" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-crplr" Jul 02 21:43:10.102335 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:10.102315 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s5fsn" Jul 02 21:43:10.104324 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:43:10.104306 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-2glnb\" is forbidden: User \"system:node:ip-10-0-139-75.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-75.ec2.internal' and this object" podUID="7ee26cd8-e7c6-4299-89ff-9c9ba3d3c9e2" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2glnb" Jul 02 21:43:10.106062 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:43:10.106043 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-s5fsn\" is forbidden: User \"system:node:ip-10-0-139-75.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-75.ec2.internal' and this object" podUID="c5be070d-0a8d-4a83-ae25-c5b4b3f71ad4" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s5fsn" Jul 02 21:43:10.107966 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:43:10.107948 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-crplr\" is forbidden: User \"system:node:ip-10-0-139-75.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-75.ec2.internal' and this object" podUID="f702232d-6b31-409c-8ef5-c2473198f0a5" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-crplr" Jul 02 21:43:10.146028 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:10.146001 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-55r5q\" (UniqueName: \"kubernetes.io/projected/f702232d-6b31-409c-8ef5-c2473198f0a5-kube-api-access-55r5q\") pod \"f702232d-6b31-409c-8ef5-c2473198f0a5\" (UID: \"f702232d-6b31-409c-8ef5-c2473198f0a5\") " Jul 02 21:43:10.146132 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:10.146060 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f702232d-6b31-409c-8ef5-c2473198f0a5-extensions-socket-volume\") pod \"f702232d-6b31-409c-8ef5-c2473198f0a5\" (UID: \"f702232d-6b31-409c-8ef5-c2473198f0a5\") " Jul 02 21:43:10.146132 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:10.146085 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cgjdr\" (UniqueName: \"kubernetes.io/projected/c5be070d-0a8d-4a83-ae25-c5b4b3f71ad4-kube-api-access-cgjdr\") pod \"c5be070d-0a8d-4a83-ae25-c5b4b3f71ad4\" (UID: \"c5be070d-0a8d-4a83-ae25-c5b4b3f71ad4\") " Jul 02 21:43:10.146440 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:10.146421 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f702232d-6b31-409c-8ef5-c2473198f0a5-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f702232d-6b31-409c-8ef5-c2473198f0a5" (UID: "f702232d-6b31-409c-8ef5-c2473198f0a5"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 21:43:10.148077 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:10.148054 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f702232d-6b31-409c-8ef5-c2473198f0a5-kube-api-access-55r5q" (OuterVolumeSpecName: "kube-api-access-55r5q") pod "f702232d-6b31-409c-8ef5-c2473198f0a5" (UID: "f702232d-6b31-409c-8ef5-c2473198f0a5"). InnerVolumeSpecName "kube-api-access-55r5q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 21:43:10.148077 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:10.148066 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5be070d-0a8d-4a83-ae25-c5b4b3f71ad4-kube-api-access-cgjdr" (OuterVolumeSpecName: "kube-api-access-cgjdr") pod "c5be070d-0a8d-4a83-ae25-c5b4b3f71ad4" (UID: "c5be070d-0a8d-4a83-ae25-c5b4b3f71ad4"). InnerVolumeSpecName "kube-api-access-cgjdr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 21:43:10.246751 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:10.246714 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-55r5q\" (UniqueName: \"kubernetes.io/projected/f702232d-6b31-409c-8ef5-c2473198f0a5-kube-api-access-55r5q\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 21:43:10.246751 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:10.246743 2569 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f702232d-6b31-409c-8ef5-c2473198f0a5-extensions-socket-volume\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 21:43:10.246751 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:10.246752 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cgjdr\" (UniqueName: \"kubernetes.io/projected/c5be070d-0a8d-4a83-ae25-c5b4b3f71ad4-kube-api-access-cgjdr\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 21:43:10.248578 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:10.248557 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-v2j2d" Jul 02 21:43:10.255538 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:10.255517 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c9jbj" Jul 02 21:43:10.261521 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:10.261493 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vsgpl" Jul 02 21:43:10.409279 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:10.409249 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-v2j2d"] Jul 02 21:43:10.412842 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:43:10.412815 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefd356c4_9cae_4f32_86b5_d8e7d98b7614.slice/crio-b007e9790927ddf21279da64e8c0ea1c7bc781d31c0f5ae56ace55a884e3dee8 WatchSource:0}: Error finding container b007e9790927ddf21279da64e8c0ea1c7bc781d31c0f5ae56ace55a884e3dee8: Status 404 returned error can't find the container with id b007e9790927ddf21279da64e8c0ea1c7bc781d31c0f5ae56ace55a884e3dee8 Jul 02 21:43:10.647344 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:10.647318 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vsgpl"] Jul 02 21:43:10.649558 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:43:10.649521 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5315e1f1_17e9_4f07_a961_f04ef2d00e64.slice/crio-1a20ea1c73acfa02e6335f4b94dfbc90b05b83b90348dd7a2cc9795846ba8ce3 WatchSource:0}: Error finding container 1a20ea1c73acfa02e6335f4b94dfbc90b05b83b90348dd7a2cc9795846ba8ce3: Status 404 returned error can't find the container with id 1a20ea1c73acfa02e6335f4b94dfbc90b05b83b90348dd7a2cc9795846ba8ce3 Jul 02 21:43:10.654037 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:10.654013 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c9jbj"] Jul 02 21:43:10.654662 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:43:10.654639 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2cef987_d128_4b1d_b9cf_272159207238.slice/crio-2caab3c0487e349a875ec4a0362d67847296ce372f0f253fd46801d74bce9e1d WatchSource:0}: Error finding container 2caab3c0487e349a875ec4a0362d67847296ce372f0f253fd46801d74bce9e1d: Status 404 returned error can't find the container with id 2caab3c0487e349a875ec4a0362d67847296ce372f0f253fd46801d74bce9e1d Jul 02 21:43:10.957488 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:10.957416 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5be070d-0a8d-4a83-ae25-c5b4b3f71ad4" path="/var/lib/kubelet/pods/c5be070d-0a8d-4a83-ae25-c5b4b3f71ad4/volumes" Jul 02 21:43:10.957778 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:10.957765 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f702232d-6b31-409c-8ef5-c2473198f0a5" path="/var/lib/kubelet/pods/f702232d-6b31-409c-8ef5-c2473198f0a5/volumes" Jul 02 21:43:10.999497 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:10.999467 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s5fsn" Jul 02 21:43:10.999497 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:10.999492 2569 scope.go:117] "RemoveContainer" containerID="841f5e3c7699874ab5a07c1b942d4f853d66d0b58457383b84f0db94d53f0203" Jul 02 21:43:11.004130 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:11.004100 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-crplr" Jul 02 21:43:11.006485 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:11.006433 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2glnb" podUID="7ee26cd8-e7c6-4299-89ff-9c9ba3d3c9e2" containerName="manager" containerID="cri-o://faa5c7e26f9378469e71059adc61a2860e495af13dfefa7601b26d556b11b36e" gracePeriod=2 Jul 02 21:43:11.008298 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:11.008271 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vsgpl" event={"ID":"5315e1f1-17e9-4f07-a961-f04ef2d00e64","Type":"ContainerStarted","Data":"d542922ff3a05d151e32622fba4f404fdc4917042994c5c1527e334ab6465b34"} Jul 02 21:43:11.008456 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:11.008431 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vsgpl" Jul 02 21:43:11.008545 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:11.008464 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vsgpl" event={"ID":"5315e1f1-17e9-4f07-a961-f04ef2d00e64","Type":"ContainerStarted","Data":"1a20ea1c73acfa02e6335f4b94dfbc90b05b83b90348dd7a2cc9795846ba8ce3"} Jul 02 21:43:11.010126 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:11.010098 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c9jbj" event={"ID":"d2cef987-d128-4b1d-b9cf-272159207238","Type":"ContainerStarted","Data":"7253483fdfeb2307ce69a43253290e0f276953e7cf32e680d2e4fef6b073c112"} Jul 02 21:43:11.010239 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:11.010149 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c9jbj" event={"ID":"d2cef987-d128-4b1d-b9cf-272159207238","Type":"ContainerStarted","Data":"2caab3c0487e349a875ec4a0362d67847296ce372f0f253fd46801d74bce9e1d"} Jul 02 21:43:11.010306 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:11.010286 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c9jbj" Jul 02 21:43:11.011080 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:11.010933 2569 scope.go:117] "RemoveContainer" containerID="aa4cfc29c7b934362c952c14a33ead8d9385ba4c208437fec9bd394bf12c370c" Jul 02 21:43:11.012179 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:11.012155 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-v2j2d" event={"ID":"efd356c4-9cae-4f32-86b5-d8e7d98b7614","Type":"ContainerStarted","Data":"57aebef8d6801936ce4f2be5461ebb7ebce04f9dfdde1841963e3e5d2bd6aa39"} Jul 02 21:43:11.012179 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:11.012201 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-v2j2d" event={"ID":"efd356c4-9cae-4f32-86b5-d8e7d98b7614","Type":"ContainerStarted","Data":"b007e9790927ddf21279da64e8c0ea1c7bc781d31c0f5ae56ace55a884e3dee8"} Jul 02 21:43:11.012341 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:11.012256 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-v2j2d" Jul 02 21:43:11.027475 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:11.027438 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vsgpl" podStartSLOduration=2.027426193 podStartE2EDuration="2.027426193s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:43:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:43:11.026993452 +0000 UTC m=+430.365963900" watchObservedRunningTime="2026-07-02 21:43:11.027426193 +0000 UTC m=+430.366396627" Jul 02 21:43:11.046768 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:11.046728 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c9jbj" podStartSLOduration=2.046716054 podStartE2EDuration="2.046716054s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:43:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:43:11.045306149 +0000 UTC m=+430.384276583" watchObservedRunningTime="2026-07-02 21:43:11.046716054 +0000 UTC m=+430.385686489" Jul 02 21:43:11.065035 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:11.064994 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-v2j2d" podStartSLOduration=2.064982551 podStartE2EDuration="2.064982551s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:43:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:43:11.062740259 +0000 UTC m=+430.401710706" watchObservedRunningTime="2026-07-02 21:43:11.064982551 +0000 UTC m=+430.403952986" Jul 02 21:43:11.253575 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:11.253550 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2glnb" Jul 02 21:43:11.256233 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:43:11.256208 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-2glnb\" is forbidden: User \"system:node:ip-10-0-139-75.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-75.ec2.internal' and this object" podUID="7ee26cd8-e7c6-4299-89ff-9c9ba3d3c9e2" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2glnb" Jul 02 21:43:11.355256 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:11.355222 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rvhwt\" (UniqueName: \"kubernetes.io/projected/7ee26cd8-e7c6-4299-89ff-9c9ba3d3c9e2-kube-api-access-rvhwt\") pod \"7ee26cd8-e7c6-4299-89ff-9c9ba3d3c9e2\" (UID: \"7ee26cd8-e7c6-4299-89ff-9c9ba3d3c9e2\") " Jul 02 21:43:11.355424 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:11.355315 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7ee26cd8-e7c6-4299-89ff-9c9ba3d3c9e2-extensions-socket-volume\") pod \"7ee26cd8-e7c6-4299-89ff-9c9ba3d3c9e2\" (UID: \"7ee26cd8-e7c6-4299-89ff-9c9ba3d3c9e2\") " Jul 02 21:43:11.355592 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:11.355571 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ee26cd8-e7c6-4299-89ff-9c9ba3d3c9e2-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "7ee26cd8-e7c6-4299-89ff-9c9ba3d3c9e2" (UID: "7ee26cd8-e7c6-4299-89ff-9c9ba3d3c9e2"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 21:43:11.357312 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:11.357292 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ee26cd8-e7c6-4299-89ff-9c9ba3d3c9e2-kube-api-access-rvhwt" (OuterVolumeSpecName: "kube-api-access-rvhwt") pod "7ee26cd8-e7c6-4299-89ff-9c9ba3d3c9e2" (UID: "7ee26cd8-e7c6-4299-89ff-9c9ba3d3c9e2"). InnerVolumeSpecName "kube-api-access-rvhwt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 21:43:11.456596 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:11.456563 2569 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7ee26cd8-e7c6-4299-89ff-9c9ba3d3c9e2-extensions-socket-volume\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 21:43:11.456596 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:11.456591 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rvhwt\" (UniqueName: \"kubernetes.io/projected/7ee26cd8-e7c6-4299-89ff-9c9ba3d3c9e2-kube-api-access-rvhwt\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 21:43:12.021942 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:12.021910 2569 generic.go:356] "Generic (PLEG): container finished" podID="7ee26cd8-e7c6-4299-89ff-9c9ba3d3c9e2" containerID="faa5c7e26f9378469e71059adc61a2860e495af13dfefa7601b26d556b11b36e" exitCode=2 Jul 02 21:43:12.022128 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:12.021952 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2glnb" Jul 02 21:43:12.022128 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:12.022015 2569 scope.go:117] "RemoveContainer" containerID="faa5c7e26f9378469e71059adc61a2860e495af13dfefa7601b26d556b11b36e" Jul 02 21:43:12.024383 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:43:12.024360 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-2glnb\" is forbidden: User \"system:node:ip-10-0-139-75.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-75.ec2.internal' and this object" podUID="7ee26cd8-e7c6-4299-89ff-9c9ba3d3c9e2" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2glnb" Jul 02 21:43:12.030976 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:12.030959 2569 scope.go:117] "RemoveContainer" containerID="faa5c7e26f9378469e71059adc61a2860e495af13dfefa7601b26d556b11b36e" Jul 02 21:43:12.031224 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:43:12.031208 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"faa5c7e26f9378469e71059adc61a2860e495af13dfefa7601b26d556b11b36e\": container with ID starting with faa5c7e26f9378469e71059adc61a2860e495af13dfefa7601b26d556b11b36e not found: ID does not exist" containerID="faa5c7e26f9378469e71059adc61a2860e495af13dfefa7601b26d556b11b36e" Jul 02 21:43:12.031262 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:12.031234 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"faa5c7e26f9378469e71059adc61a2860e495af13dfefa7601b26d556b11b36e"} err="failed to get container status \"faa5c7e26f9378469e71059adc61a2860e495af13dfefa7601b26d556b11b36e\": rpc error: code = NotFound desc = could not find container \"faa5c7e26f9378469e71059adc61a2860e495af13dfefa7601b26d556b11b36e\": container with ID starting with faa5c7e26f9378469e71059adc61a2860e495af13dfefa7601b26d556b11b36e not found: ID does not exist" Jul 02 21:43:12.034011 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:43:12.033990 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-2glnb\" is forbidden: User \"system:node:ip-10-0-139-75.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-75.ec2.internal' and this object" podUID="7ee26cd8-e7c6-4299-89ff-9c9ba3d3c9e2" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2glnb" Jul 02 21:43:12.956102 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:43:12.956062 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-2glnb\" is forbidden: User \"system:node:ip-10-0-139-75.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-75.ec2.internal' and this object" podUID="7ee26cd8-e7c6-4299-89ff-9c9ba3d3c9e2" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2glnb" Jul 02 21:43:12.958477 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:12.958452 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ee26cd8-e7c6-4299-89ff-9c9ba3d3c9e2" path="/var/lib/kubelet/pods/7ee26cd8-e7c6-4299-89ff-9c9ba3d3c9e2/volumes" Jul 02 21:43:22.024878 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:22.024847 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c9jbj" Jul 02 21:43:22.025380 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:22.024920 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vsgpl" Jul 02 21:43:22.025380 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:22.024959 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-v2j2d" Jul 02 21:43:22.109316 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:22.109283 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c9jbj"] Jul 02 21:43:22.109543 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:22.109514 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c9jbj" podUID="d2cef987-d128-4b1d-b9cf-272159207238" containerName="manager" containerID="cri-o://7253483fdfeb2307ce69a43253290e0f276953e7cf32e680d2e4fef6b073c112" gracePeriod=10 Jul 02 21:43:22.363419 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:22.363396 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c9jbj" Jul 02 21:43:22.454960 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:22.454925 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d2cef987-d128-4b1d-b9cf-272159207238-extensions-socket-volume\") pod \"d2cef987-d128-4b1d-b9cf-272159207238\" (UID: \"d2cef987-d128-4b1d-b9cf-272159207238\") " Jul 02 21:43:22.455123 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:22.454991 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2gwlb\" (UniqueName: \"kubernetes.io/projected/d2cef987-d128-4b1d-b9cf-272159207238-kube-api-access-2gwlb\") pod \"d2cef987-d128-4b1d-b9cf-272159207238\" (UID: \"d2cef987-d128-4b1d-b9cf-272159207238\") " Jul 02 21:43:22.455356 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:22.455333 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2cef987-d128-4b1d-b9cf-272159207238-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d2cef987-d128-4b1d-b9cf-272159207238" (UID: "d2cef987-d128-4b1d-b9cf-272159207238"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 21:43:22.457232 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:22.457202 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2cef987-d128-4b1d-b9cf-272159207238-kube-api-access-2gwlb" (OuterVolumeSpecName: "kube-api-access-2gwlb") pod "d2cef987-d128-4b1d-b9cf-272159207238" (UID: "d2cef987-d128-4b1d-b9cf-272159207238"). InnerVolumeSpecName "kube-api-access-2gwlb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 21:43:22.458144 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:22.458123 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rppgc"] Jul 02 21:43:22.458508 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:22.458497 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2cef987-d128-4b1d-b9cf-272159207238" containerName="manager" Jul 02 21:43:22.458545 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:22.458510 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2cef987-d128-4b1d-b9cf-272159207238" containerName="manager" Jul 02 21:43:22.458580 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:22.458575 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2cef987-d128-4b1d-b9cf-272159207238" containerName="manager" Jul 02 21:43:22.460641 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:22.460625 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rppgc" Jul 02 21:43:22.474060 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:22.474036 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rppgc"] Jul 02 21:43:22.555915 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:22.555879 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/37f15a25-f233-47ad-acde-94dcb577b1d2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-rppgc\" (UID: \"37f15a25-f233-47ad-acde-94dcb577b1d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rppgc" Jul 02 21:43:22.555915 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:22.555920 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mzsw\" (UniqueName: \"kubernetes.io/projected/37f15a25-f233-47ad-acde-94dcb577b1d2-kube-api-access-5mzsw\") pod \"kuadrant-operator-controller-manager-86878c769f-rppgc\" (UID: \"37f15a25-f233-47ad-acde-94dcb577b1d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rppgc" Jul 02 21:43:22.556127 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:22.556024 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2gwlb\" (UniqueName: \"kubernetes.io/projected/d2cef987-d128-4b1d-b9cf-272159207238-kube-api-access-2gwlb\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 21:43:22.556127 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:22.556044 2569 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d2cef987-d128-4b1d-b9cf-272159207238-extensions-socket-volume\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 21:43:22.657522 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:22.657418 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/37f15a25-f233-47ad-acde-94dcb577b1d2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-rppgc\" (UID: \"37f15a25-f233-47ad-acde-94dcb577b1d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rppgc" Jul 02 21:43:22.657522 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:22.657453 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mzsw\" (UniqueName: \"kubernetes.io/projected/37f15a25-f233-47ad-acde-94dcb577b1d2-kube-api-access-5mzsw\") pod \"kuadrant-operator-controller-manager-86878c769f-rppgc\" (UID: \"37f15a25-f233-47ad-acde-94dcb577b1d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rppgc" Jul 02 21:43:22.657844 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:22.657825 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/37f15a25-f233-47ad-acde-94dcb577b1d2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-rppgc\" (UID: \"37f15a25-f233-47ad-acde-94dcb577b1d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rppgc" Jul 02 21:43:22.676672 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:22.676633 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mzsw\" (UniqueName: \"kubernetes.io/projected/37f15a25-f233-47ad-acde-94dcb577b1d2-kube-api-access-5mzsw\") pod \"kuadrant-operator-controller-manager-86878c769f-rppgc\" (UID: \"37f15a25-f233-47ad-acde-94dcb577b1d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rppgc" Jul 02 21:43:22.774684 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:22.774648 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rppgc" Jul 02 21:43:22.909698 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:22.909661 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rppgc"] Jul 02 21:43:22.912083 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:43:22.912052 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37f15a25_f233_47ad_acde_94dcb577b1d2.slice/crio-ee0400b3e8141e50856d9d69033be34150c7c40240162d8fdad3ea06934dcdca WatchSource:0}: Error finding container ee0400b3e8141e50856d9d69033be34150c7c40240162d8fdad3ea06934dcdca: Status 404 returned error can't find the container with id ee0400b3e8141e50856d9d69033be34150c7c40240162d8fdad3ea06934dcdca Jul 02 21:43:23.080170 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:23.080132 2569 generic.go:356] "Generic (PLEG): container finished" podID="d2cef987-d128-4b1d-b9cf-272159207238" containerID="7253483fdfeb2307ce69a43253290e0f276953e7cf32e680d2e4fef6b073c112" exitCode=0 Jul 02 21:43:23.080170 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:23.080159 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c9jbj" event={"ID":"d2cef987-d128-4b1d-b9cf-272159207238","Type":"ContainerDied","Data":"7253483fdfeb2307ce69a43253290e0f276953e7cf32e680d2e4fef6b073c112"} Jul 02 21:43:23.080716 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:23.080188 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c9jbj" Jul 02 21:43:23.080716 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:23.080215 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c9jbj" event={"ID":"d2cef987-d128-4b1d-b9cf-272159207238","Type":"ContainerDied","Data":"2caab3c0487e349a875ec4a0362d67847296ce372f0f253fd46801d74bce9e1d"} Jul 02 21:43:23.080716 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:23.080239 2569 scope.go:117] "RemoveContainer" containerID="7253483fdfeb2307ce69a43253290e0f276953e7cf32e680d2e4fef6b073c112" Jul 02 21:43:23.082319 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:23.082290 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rppgc" event={"ID":"37f15a25-f233-47ad-acde-94dcb577b1d2","Type":"ContainerStarted","Data":"be1642754aa06cd086dc70791c9c94865ffc6f3a715766932b08aeaa6a894df5"} Jul 02 21:43:23.082477 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:23.082346 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rppgc" event={"ID":"37f15a25-f233-47ad-acde-94dcb577b1d2","Type":"ContainerStarted","Data":"ee0400b3e8141e50856d9d69033be34150c7c40240162d8fdad3ea06934dcdca"} Jul 02 21:43:23.082834 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:23.082814 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rppgc" Jul 02 21:43:23.089481 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:23.089464 2569 scope.go:117] "RemoveContainer" containerID="7253483fdfeb2307ce69a43253290e0f276953e7cf32e680d2e4fef6b073c112" Jul 02 21:43:23.089725 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:43:23.089705 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7253483fdfeb2307ce69a43253290e0f276953e7cf32e680d2e4fef6b073c112\": container with ID starting with 7253483fdfeb2307ce69a43253290e0f276953e7cf32e680d2e4fef6b073c112 not found: ID does not exist" containerID="7253483fdfeb2307ce69a43253290e0f276953e7cf32e680d2e4fef6b073c112" Jul 02 21:43:23.089798 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:23.089738 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7253483fdfeb2307ce69a43253290e0f276953e7cf32e680d2e4fef6b073c112"} err="failed to get container status \"7253483fdfeb2307ce69a43253290e0f276953e7cf32e680d2e4fef6b073c112\": rpc error: code = NotFound desc = could not find container \"7253483fdfeb2307ce69a43253290e0f276953e7cf32e680d2e4fef6b073c112\": container with ID starting with 7253483fdfeb2307ce69a43253290e0f276953e7cf32e680d2e4fef6b073c112 not found: ID does not exist" Jul 02 21:43:23.118293 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:23.118252 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rppgc" podStartSLOduration=1.118239513 podStartE2EDuration="1.118239513s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:43:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:43:23.116034148 +0000 UTC m=+442.455004583" watchObservedRunningTime="2026-07-02 21:43:23.118239513 +0000 UTC m=+442.457209995" Jul 02 21:43:23.144492 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:23.144469 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c9jbj"] Jul 02 21:43:23.156016 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:23.155991 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c9jbj"] Jul 02 21:43:24.956038 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:24.956009 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2cef987-d128-4b1d-b9cf-272159207238" path="/var/lib/kubelet/pods/d2cef987-d128-4b1d-b9cf-272159207238/volumes" Jul 02 21:43:34.091737 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:34.091708 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rppgc" Jul 02 21:43:34.150734 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:34.150695 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-v2j2d"] Jul 02 21:43:34.151023 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:34.150952 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-v2j2d" podUID="efd356c4-9cae-4f32-86b5-d8e7d98b7614" containerName="manager" containerID="cri-o://57aebef8d6801936ce4f2be5461ebb7ebce04f9dfdde1841963e3e5d2bd6aa39" gracePeriod=10 Jul 02 21:43:34.402051 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:34.402028 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-v2j2d" Jul 02 21:43:34.568143 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:34.568114 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/efd356c4-9cae-4f32-86b5-d8e7d98b7614-extensions-socket-volume\") pod \"efd356c4-9cae-4f32-86b5-d8e7d98b7614\" (UID: \"efd356c4-9cae-4f32-86b5-d8e7d98b7614\") " Jul 02 21:43:34.568302 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:34.568158 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gbf5q\" (UniqueName: \"kubernetes.io/projected/efd356c4-9cae-4f32-86b5-d8e7d98b7614-kube-api-access-gbf5q\") pod \"efd356c4-9cae-4f32-86b5-d8e7d98b7614\" (UID: \"efd356c4-9cae-4f32-86b5-d8e7d98b7614\") " Jul 02 21:43:34.568503 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:34.568480 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efd356c4-9cae-4f32-86b5-d8e7d98b7614-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "efd356c4-9cae-4f32-86b5-d8e7d98b7614" (UID: "efd356c4-9cae-4f32-86b5-d8e7d98b7614"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 21:43:34.570133 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:34.570113 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efd356c4-9cae-4f32-86b5-d8e7d98b7614-kube-api-access-gbf5q" (OuterVolumeSpecName: "kube-api-access-gbf5q") pod "efd356c4-9cae-4f32-86b5-d8e7d98b7614" (UID: "efd356c4-9cae-4f32-86b5-d8e7d98b7614"). InnerVolumeSpecName "kube-api-access-gbf5q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 21:43:34.669260 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:34.669180 2569 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/efd356c4-9cae-4f32-86b5-d8e7d98b7614-extensions-socket-volume\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 21:43:34.669260 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:34.669217 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gbf5q\" (UniqueName: \"kubernetes.io/projected/efd356c4-9cae-4f32-86b5-d8e7d98b7614-kube-api-access-gbf5q\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 21:43:35.148337 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:35.148300 2569 generic.go:356] "Generic (PLEG): container finished" podID="efd356c4-9cae-4f32-86b5-d8e7d98b7614" containerID="57aebef8d6801936ce4f2be5461ebb7ebce04f9dfdde1841963e3e5d2bd6aa39" exitCode=0 Jul 02 21:43:35.148700 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:35.148350 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-v2j2d" Jul 02 21:43:35.148700 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:35.148384 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-v2j2d" event={"ID":"efd356c4-9cae-4f32-86b5-d8e7d98b7614","Type":"ContainerDied","Data":"57aebef8d6801936ce4f2be5461ebb7ebce04f9dfdde1841963e3e5d2bd6aa39"} Jul 02 21:43:35.148700 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:35.148447 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-v2j2d" event={"ID":"efd356c4-9cae-4f32-86b5-d8e7d98b7614","Type":"ContainerDied","Data":"b007e9790927ddf21279da64e8c0ea1c7bc781d31c0f5ae56ace55a884e3dee8"} Jul 02 21:43:35.148700 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:35.148469 2569 scope.go:117] "RemoveContainer" containerID="57aebef8d6801936ce4f2be5461ebb7ebce04f9dfdde1841963e3e5d2bd6aa39" Jul 02 21:43:35.156949 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:35.156806 2569 scope.go:117] "RemoveContainer" containerID="57aebef8d6801936ce4f2be5461ebb7ebce04f9dfdde1841963e3e5d2bd6aa39" Jul 02 21:43:35.157065 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:43:35.157046 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57aebef8d6801936ce4f2be5461ebb7ebce04f9dfdde1841963e3e5d2bd6aa39\": container with ID starting with 57aebef8d6801936ce4f2be5461ebb7ebce04f9dfdde1841963e3e5d2bd6aa39 not found: ID does not exist" containerID="57aebef8d6801936ce4f2be5461ebb7ebce04f9dfdde1841963e3e5d2bd6aa39" Jul 02 21:43:35.157108 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:35.157073 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57aebef8d6801936ce4f2be5461ebb7ebce04f9dfdde1841963e3e5d2bd6aa39"} err="failed to get container status \"57aebef8d6801936ce4f2be5461ebb7ebce04f9dfdde1841963e3e5d2bd6aa39\": rpc error: code = NotFound desc = could not find container \"57aebef8d6801936ce4f2be5461ebb7ebce04f9dfdde1841963e3e5d2bd6aa39\": container with ID starting with 57aebef8d6801936ce4f2be5461ebb7ebce04f9dfdde1841963e3e5d2bd6aa39 not found: ID does not exist" Jul 02 21:43:35.166669 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:35.166650 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-v2j2d"] Jul 02 21:43:35.172442 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:35.172423 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-v2j2d"] Jul 02 21:43:36.956390 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:36.956361 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efd356c4-9cae-4f32-86b5-d8e7d98b7614" path="/var/lib/kubelet/pods/efd356c4-9cae-4f32-86b5-d8e7d98b7614/volumes" Jul 02 21:43:38.495853 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:38.495818 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5tjmq"] Jul 02 21:43:38.496270 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:38.496255 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efd356c4-9cae-4f32-86b5-d8e7d98b7614" containerName="manager" Jul 02 21:43:38.496319 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:38.496272 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="efd356c4-9cae-4f32-86b5-d8e7d98b7614" containerName="manager" Jul 02 21:43:38.496361 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:38.496332 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="efd356c4-9cae-4f32-86b5-d8e7d98b7614" containerName="manager" Jul 02 21:43:38.499050 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:38.499028 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5tjmq" Jul 02 21:43:38.501279 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:38.501255 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-8nlnb\"" Jul 02 21:43:38.510423 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:38.510403 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5tjmq"] Jul 02 21:43:38.598362 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:38.598327 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2749fe4b-86bd-4342-863c-ab4e0bd8bd24-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-5tjmq\" (UID: \"2749fe4b-86bd-4342-863c-ab4e0bd8bd24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5tjmq" Jul 02 21:43:38.598362 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:38.598361 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2749fe4b-86bd-4342-863c-ab4e0bd8bd24-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-5tjmq\" (UID: \"2749fe4b-86bd-4342-863c-ab4e0bd8bd24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5tjmq" Jul 02 21:43:38.598583 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:38.598441 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2749fe4b-86bd-4342-863c-ab4e0bd8bd24-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-5tjmq\" (UID: \"2749fe4b-86bd-4342-863c-ab4e0bd8bd24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5tjmq" Jul 02 21:43:38.598583 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:38.598497 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2749fe4b-86bd-4342-863c-ab4e0bd8bd24-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-5tjmq\" (UID: \"2749fe4b-86bd-4342-863c-ab4e0bd8bd24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5tjmq" Jul 02 21:43:38.598583 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:38.598519 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2749fe4b-86bd-4342-863c-ab4e0bd8bd24-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-5tjmq\" (UID: \"2749fe4b-86bd-4342-863c-ab4e0bd8bd24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5tjmq" Jul 02 21:43:38.598583 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:38.598536 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2749fe4b-86bd-4342-863c-ab4e0bd8bd24-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-5tjmq\" (UID: \"2749fe4b-86bd-4342-863c-ab4e0bd8bd24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5tjmq" Jul 02 21:43:38.598583 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:38.598557 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6zdz\" (UniqueName: \"kubernetes.io/projected/2749fe4b-86bd-4342-863c-ab4e0bd8bd24-kube-api-access-v6zdz\") pod \"maas-default-gateway-openshift-default-56654dcf7-5tjmq\" (UID: \"2749fe4b-86bd-4342-863c-ab4e0bd8bd24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5tjmq" Jul 02 21:43:38.598583 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:38.598575 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2749fe4b-86bd-4342-863c-ab4e0bd8bd24-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-5tjmq\" (UID: \"2749fe4b-86bd-4342-863c-ab4e0bd8bd24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5tjmq" Jul 02 21:43:38.598817 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:38.598599 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2749fe4b-86bd-4342-863c-ab4e0bd8bd24-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-5tjmq\" (UID: \"2749fe4b-86bd-4342-863c-ab4e0bd8bd24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5tjmq" Jul 02 21:43:38.699856 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:38.699821 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2749fe4b-86bd-4342-863c-ab4e0bd8bd24-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-5tjmq\" (UID: \"2749fe4b-86bd-4342-863c-ab4e0bd8bd24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5tjmq" Jul 02 21:43:38.700042 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:38.699885 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2749fe4b-86bd-4342-863c-ab4e0bd8bd24-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-5tjmq\" (UID: \"2749fe4b-86bd-4342-863c-ab4e0bd8bd24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5tjmq" Jul 02 21:43:38.700042 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:38.699909 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2749fe4b-86bd-4342-863c-ab4e0bd8bd24-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-5tjmq\" (UID: \"2749fe4b-86bd-4342-863c-ab4e0bd8bd24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5tjmq" Jul 02 21:43:38.700042 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:38.699926 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2749fe4b-86bd-4342-863c-ab4e0bd8bd24-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-5tjmq\" (UID: \"2749fe4b-86bd-4342-863c-ab4e0bd8bd24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5tjmq" Jul 02 21:43:38.700042 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:38.699949 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6zdz\" (UniqueName: \"kubernetes.io/projected/2749fe4b-86bd-4342-863c-ab4e0bd8bd24-kube-api-access-v6zdz\") pod \"maas-default-gateway-openshift-default-56654dcf7-5tjmq\" (UID: \"2749fe4b-86bd-4342-863c-ab4e0bd8bd24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5tjmq" Jul 02 21:43:38.700042 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:38.699970 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2749fe4b-86bd-4342-863c-ab4e0bd8bd24-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-5tjmq\" (UID: \"2749fe4b-86bd-4342-863c-ab4e0bd8bd24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5tjmq" Jul 02 21:43:38.700042 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:38.699995 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2749fe4b-86bd-4342-863c-ab4e0bd8bd24-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-5tjmq\" (UID: \"2749fe4b-86bd-4342-863c-ab4e0bd8bd24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5tjmq" Jul 02 21:43:38.700042 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:38.700025 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2749fe4b-86bd-4342-863c-ab4e0bd8bd24-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-5tjmq\" (UID: \"2749fe4b-86bd-4342-863c-ab4e0bd8bd24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5tjmq" Jul 02 21:43:38.700425 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:38.700052 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2749fe4b-86bd-4342-863c-ab4e0bd8bd24-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-5tjmq\" (UID: \"2749fe4b-86bd-4342-863c-ab4e0bd8bd24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5tjmq" Jul 02 21:43:38.700505 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:38.700472 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2749fe4b-86bd-4342-863c-ab4e0bd8bd24-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-5tjmq\" (UID: \"2749fe4b-86bd-4342-863c-ab4e0bd8bd24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5tjmq" Jul 02 21:43:38.700632 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:38.700509 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2749fe4b-86bd-4342-863c-ab4e0bd8bd24-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-5tjmq\" (UID: \"2749fe4b-86bd-4342-863c-ab4e0bd8bd24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5tjmq" Jul 02 21:43:38.700632 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:38.700574 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2749fe4b-86bd-4342-863c-ab4e0bd8bd24-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-5tjmq\" (UID: \"2749fe4b-86bd-4342-863c-ab4e0bd8bd24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5tjmq" Jul 02 21:43:38.700742 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:38.700633 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2749fe4b-86bd-4342-863c-ab4e0bd8bd24-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-5tjmq\" (UID: \"2749fe4b-86bd-4342-863c-ab4e0bd8bd24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5tjmq" Jul 02 21:43:38.700825 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:38.700750 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2749fe4b-86bd-4342-863c-ab4e0bd8bd24-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-5tjmq\" (UID: \"2749fe4b-86bd-4342-863c-ab4e0bd8bd24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5tjmq" Jul 02 21:43:38.702528 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:38.702506 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2749fe4b-86bd-4342-863c-ab4e0bd8bd24-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-5tjmq\" (UID: \"2749fe4b-86bd-4342-863c-ab4e0bd8bd24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5tjmq" Jul 02 21:43:38.702780 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:38.702764 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2749fe4b-86bd-4342-863c-ab4e0bd8bd24-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-5tjmq\" (UID: \"2749fe4b-86bd-4342-863c-ab4e0bd8bd24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5tjmq" Jul 02 21:43:38.711444 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:38.711420 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2749fe4b-86bd-4342-863c-ab4e0bd8bd24-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-5tjmq\" (UID: \"2749fe4b-86bd-4342-863c-ab4e0bd8bd24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5tjmq" Jul 02 21:43:38.711959 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:38.711941 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6zdz\" (UniqueName: \"kubernetes.io/projected/2749fe4b-86bd-4342-863c-ab4e0bd8bd24-kube-api-access-v6zdz\") pod \"maas-default-gateway-openshift-default-56654dcf7-5tjmq\" (UID: \"2749fe4b-86bd-4342-863c-ab4e0bd8bd24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5tjmq" Jul 02 21:43:38.809849 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:38.809814 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5tjmq" Jul 02 21:43:38.953254 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:43:38.953223 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2749fe4b_86bd_4342_863c_ab4e0bd8bd24.slice/crio-9074fff205a2b7534b83d0153c5f0cf9b4593abf17b75b0ccd8bd4ad1fd6d446 WatchSource:0}: Error finding container 9074fff205a2b7534b83d0153c5f0cf9b4593abf17b75b0ccd8bd4ad1fd6d446: Status 404 returned error can't find the container with id 9074fff205a2b7534b83d0153c5f0cf9b4593abf17b75b0ccd8bd4ad1fd6d446 Jul 02 21:43:38.955580 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:38.955538 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Jul 02 21:43:38.955646 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:38.955614 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Jul 02 21:43:38.955684 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:38.955649 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Jul 02 21:43:38.957608 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:38.957588 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5tjmq"] Jul 02 21:43:39.173300 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:39.173217 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5tjmq" event={"ID":"2749fe4b-86bd-4342-863c-ab4e0bd8bd24","Type":"ContainerStarted","Data":"6aa8ac15281040fcd64d3632832db524796b2708800739cd91effce9abc31e48"} Jul 02 21:43:39.173300 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:39.173278 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5tjmq" event={"ID":"2749fe4b-86bd-4342-863c-ab4e0bd8bd24","Type":"ContainerStarted","Data":"9074fff205a2b7534b83d0153c5f0cf9b4593abf17b75b0ccd8bd4ad1fd6d446"} Jul 02 21:43:39.194129 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:39.194083 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5tjmq" podStartSLOduration=1.194068526 podStartE2EDuration="1.194068526s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:43:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:43:39.192472787 +0000 UTC m=+458.531443223" watchObservedRunningTime="2026-07-02 21:43:39.194068526 +0000 UTC m=+458.533038960" Jul 02 21:43:39.810423 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:39.810391 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5tjmq" Jul 02 21:43:40.814697 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:40.814668 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5tjmq" Jul 02 21:43:41.182898 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:41.182827 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5tjmq" Jul 02 21:43:41.183870 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:41.183852 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5tjmq" Jul 02 21:43:51.691203 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:51.691172 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-mnkdh"] Jul 02 21:43:51.694599 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:51.694577 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-mnkdh" Jul 02 21:43:51.696960 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:51.696942 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-gbfcb\"" Jul 02 21:43:51.701232 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:51.701207 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-mnkdh"] Jul 02 21:43:51.816075 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:51.816038 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bgmc\" (UniqueName: \"kubernetes.io/projected/20394ae0-55e3-42d9-b7af-645119782542-kube-api-access-5bgmc\") pod \"authorino-58cd955675-mnkdh\" (UID: \"20394ae0-55e3-42d9-b7af-645119782542\") " pod="kuadrant-system/authorino-58cd955675-mnkdh" Jul 02 21:43:51.917346 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:51.917313 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bgmc\" (UniqueName: \"kubernetes.io/projected/20394ae0-55e3-42d9-b7af-645119782542-kube-api-access-5bgmc\") pod \"authorino-58cd955675-mnkdh\" (UID: \"20394ae0-55e3-42d9-b7af-645119782542\") " pod="kuadrant-system/authorino-58cd955675-mnkdh" Jul 02 21:43:51.925159 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:51.925133 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bgmc\" (UniqueName: \"kubernetes.io/projected/20394ae0-55e3-42d9-b7af-645119782542-kube-api-access-5bgmc\") pod \"authorino-58cd955675-mnkdh\" (UID: \"20394ae0-55e3-42d9-b7af-645119782542\") " pod="kuadrant-system/authorino-58cd955675-mnkdh" Jul 02 21:43:52.005568 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:52.005543 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-mnkdh" Jul 02 21:43:52.171151 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:52.171119 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-mnkdh"] Jul 02 21:43:52.174527 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:43:52.174497 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod20394ae0_55e3_42d9_b7af_645119782542.slice/crio-0c2e89960015173e583c1d085f2c5e0b5eaf81cbb02b4c9ae9fe73960edb6a8a WatchSource:0}: Error finding container 0c2e89960015173e583c1d085f2c5e0b5eaf81cbb02b4c9ae9fe73960edb6a8a: Status 404 returned error can't find the container with id 0c2e89960015173e583c1d085f2c5e0b5eaf81cbb02b4c9ae9fe73960edb6a8a Jul 02 21:43:52.238055 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:52.238023 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-mnkdh" event={"ID":"20394ae0-55e3-42d9-b7af-645119782542","Type":"ContainerStarted","Data":"0c2e89960015173e583c1d085f2c5e0b5eaf81cbb02b4c9ae9fe73960edb6a8a"} Jul 02 21:43:56.266158 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:56.266119 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-mnkdh" event={"ID":"20394ae0-55e3-42d9-b7af-645119782542","Type":"ContainerStarted","Data":"7ba6fb44802a3fa70e4a8a53dbc327a417300fe5039d29d7ca88426935450186"} Jul 02 21:43:56.281055 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:56.281000 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-mnkdh" podStartSLOduration=2.093964402 podStartE2EDuration="5.280986226s" totalImagesPullingTime="3.187021824s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:43:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:43:56.280768364 +0000 UTC m=+475.619738800" watchObservedRunningTime="2026-07-02 21:43:56.280986226 +0000 UTC m=+475.619956660" Jul 02 21:43:57.151777 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:57.151740 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-mnkdh"] Jul 02 21:43:58.277345 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:58.277302 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-mnkdh" podUID="20394ae0-55e3-42d9-b7af-645119782542" containerName="authorino" containerID="cri-o://7ba6fb44802a3fa70e4a8a53dbc327a417300fe5039d29d7ca88426935450186" gracePeriod=30 Jul 02 21:43:58.534629 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:58.534571 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-mnkdh" Jul 02 21:43:58.682251 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:58.682224 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5bgmc\" (UniqueName: \"kubernetes.io/projected/20394ae0-55e3-42d9-b7af-645119782542-kube-api-access-5bgmc\") pod \"20394ae0-55e3-42d9-b7af-645119782542\" (UID: \"20394ae0-55e3-42d9-b7af-645119782542\") " Jul 02 21:43:58.684346 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:58.684316 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20394ae0-55e3-42d9-b7af-645119782542-kube-api-access-5bgmc" (OuterVolumeSpecName: "kube-api-access-5bgmc") pod "20394ae0-55e3-42d9-b7af-645119782542" (UID: "20394ae0-55e3-42d9-b7af-645119782542"). InnerVolumeSpecName "kube-api-access-5bgmc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 21:43:58.782976 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:58.782929 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5bgmc\" (UniqueName: \"kubernetes.io/projected/20394ae0-55e3-42d9-b7af-645119782542-kube-api-access-5bgmc\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 21:43:59.283142 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:59.283108 2569 generic.go:356] "Generic (PLEG): container finished" podID="20394ae0-55e3-42d9-b7af-645119782542" containerID="7ba6fb44802a3fa70e4a8a53dbc327a417300fe5039d29d7ca88426935450186" exitCode=0 Jul 02 21:43:59.283142 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:59.283141 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-mnkdh" Jul 02 21:43:59.283572 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:59.283195 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-mnkdh" event={"ID":"20394ae0-55e3-42d9-b7af-645119782542","Type":"ContainerDied","Data":"7ba6fb44802a3fa70e4a8a53dbc327a417300fe5039d29d7ca88426935450186"} Jul 02 21:43:59.283572 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:59.283233 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-mnkdh" event={"ID":"20394ae0-55e3-42d9-b7af-645119782542","Type":"ContainerDied","Data":"0c2e89960015173e583c1d085f2c5e0b5eaf81cbb02b4c9ae9fe73960edb6a8a"} Jul 02 21:43:59.283572 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:59.283250 2569 scope.go:117] "RemoveContainer" containerID="7ba6fb44802a3fa70e4a8a53dbc327a417300fe5039d29d7ca88426935450186" Jul 02 21:43:59.291931 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:59.291916 2569 scope.go:117] "RemoveContainer" containerID="7ba6fb44802a3fa70e4a8a53dbc327a417300fe5039d29d7ca88426935450186" Jul 02 21:43:59.292177 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:43:59.292160 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ba6fb44802a3fa70e4a8a53dbc327a417300fe5039d29d7ca88426935450186\": container with ID starting with 7ba6fb44802a3fa70e4a8a53dbc327a417300fe5039d29d7ca88426935450186 not found: ID does not exist" containerID="7ba6fb44802a3fa70e4a8a53dbc327a417300fe5039d29d7ca88426935450186" Jul 02 21:43:59.292230 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:59.292188 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ba6fb44802a3fa70e4a8a53dbc327a417300fe5039d29d7ca88426935450186"} err="failed to get container status \"7ba6fb44802a3fa70e4a8a53dbc327a417300fe5039d29d7ca88426935450186\": rpc error: code = NotFound desc = could not find container \"7ba6fb44802a3fa70e4a8a53dbc327a417300fe5039d29d7ca88426935450186\": container with ID starting with 7ba6fb44802a3fa70e4a8a53dbc327a417300fe5039d29d7ca88426935450186 not found: ID does not exist" Jul 02 21:43:59.298556 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:59.298531 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-mnkdh"] Jul 02 21:43:59.302077 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:43:59.302047 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-mnkdh"] Jul 02 21:44:00.956411 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:44:00.956379 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20394ae0-55e3-42d9-b7af-645119782542" path="/var/lib/kubelet/pods/20394ae0-55e3-42d9-b7af-645119782542/volumes" Jul 02 21:44:12.051878 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:44:12.051839 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-6khkg"] Jul 02 21:44:12.052349 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:44:12.052228 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20394ae0-55e3-42d9-b7af-645119782542" containerName="authorino" Jul 02 21:44:12.052349 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:44:12.052241 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="20394ae0-55e3-42d9-b7af-645119782542" containerName="authorino" Jul 02 21:44:12.052349 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:44:12.052309 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="20394ae0-55e3-42d9-b7af-645119782542" containerName="authorino" Jul 02 21:44:12.056763 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:44:12.056739 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-6khkg" Jul 02 21:44:12.059202 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:44:12.059179 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jul 02 21:44:12.059202 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:44:12.059193 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-d2w6w\"" Jul 02 21:44:12.067066 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:44:12.067041 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-6khkg"] Jul 02 21:44:12.205666 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:44:12.205618 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-768st\" (UniqueName: \"kubernetes.io/projected/34d3904e-29e4-413f-9f5f-8a259c6d2279-kube-api-access-768st\") pod \"postgres-69fb4594cf-6khkg\" (UID: \"34d3904e-29e4-413f-9f5f-8a259c6d2279\") " pod="opendatahub/postgres-69fb4594cf-6khkg" Jul 02 21:44:12.205869 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:44:12.205717 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/34d3904e-29e4-413f-9f5f-8a259c6d2279-data\") pod \"postgres-69fb4594cf-6khkg\" (UID: \"34d3904e-29e4-413f-9f5f-8a259c6d2279\") " pod="opendatahub/postgres-69fb4594cf-6khkg" Jul 02 21:44:12.306467 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:44:12.306383 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-768st\" (UniqueName: \"kubernetes.io/projected/34d3904e-29e4-413f-9f5f-8a259c6d2279-kube-api-access-768st\") pod \"postgres-69fb4594cf-6khkg\" (UID: \"34d3904e-29e4-413f-9f5f-8a259c6d2279\") " pod="opendatahub/postgres-69fb4594cf-6khkg" Jul 02 21:44:12.306467 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:44:12.306441 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/34d3904e-29e4-413f-9f5f-8a259c6d2279-data\") pod \"postgres-69fb4594cf-6khkg\" (UID: \"34d3904e-29e4-413f-9f5f-8a259c6d2279\") " pod="opendatahub/postgres-69fb4594cf-6khkg" Jul 02 21:44:12.306851 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:44:12.306833 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/34d3904e-29e4-413f-9f5f-8a259c6d2279-data\") pod \"postgres-69fb4594cf-6khkg\" (UID: \"34d3904e-29e4-413f-9f5f-8a259c6d2279\") " pod="opendatahub/postgres-69fb4594cf-6khkg" Jul 02 21:44:12.315905 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:44:12.315879 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-768st\" (UniqueName: \"kubernetes.io/projected/34d3904e-29e4-413f-9f5f-8a259c6d2279-kube-api-access-768st\") pod \"postgres-69fb4594cf-6khkg\" (UID: \"34d3904e-29e4-413f-9f5f-8a259c6d2279\") " pod="opendatahub/postgres-69fb4594cf-6khkg" Jul 02 21:44:12.368531 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:44:12.368495 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-6khkg" Jul 02 21:44:12.527098 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:44:12.527075 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-6khkg"] Jul 02 21:44:12.529272 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:44:12.529235 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34d3904e_29e4_413f_9f5f_8a259c6d2279.slice/crio-2d2d7b009ce5dc9cf1c9c571f9ffcec9b5895b8c7978eb55092623084768c282 WatchSource:0}: Error finding container 2d2d7b009ce5dc9cf1c9c571f9ffcec9b5895b8c7978eb55092623084768c282: Status 404 returned error can't find the container with id 2d2d7b009ce5dc9cf1c9c571f9ffcec9b5895b8c7978eb55092623084768c282 Jul 02 21:44:13.356130 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:44:13.356085 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-6khkg" event={"ID":"34d3904e-29e4-413f-9f5f-8a259c6d2279","Type":"ContainerStarted","Data":"2d2d7b009ce5dc9cf1c9c571f9ffcec9b5895b8c7978eb55092623084768c282"} Jul 02 21:44:45.512192 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:44:45.512152 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-6khkg" event={"ID":"34d3904e-29e4-413f-9f5f-8a259c6d2279","Type":"ContainerStarted","Data":"5be195f7b6256b7e3b624ddec5187005f0a93278f80d424f769a42dc73e6e89a"} Jul 02 21:44:45.512647 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:44:45.512211 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-6khkg" Jul 02 21:44:45.537265 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:44:45.533068 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-6khkg" podStartSLOduration=1.6002438780000001 podStartE2EDuration="33.533046602s" totalImagesPullingTime="31.932802724s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:44:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:44:45.528393371 +0000 UTC m=+524.867363806" watchObservedRunningTime="2026-07-02 21:44:45.533046602 +0000 UTC m=+524.872017040" Jul 02 21:44:51.547455 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:44:51.547428 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-6khkg" Jul 02 21:44:58.964800 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:44:58.964715 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-544f8c6b68-9kc57"] Jul 02 21:44:58.968377 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:44:58.968356 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-544f8c6b68-9kc57" Jul 02 21:44:58.970889 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:44:58.970861 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-2vqz5\"" Jul 02 21:44:58.971778 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:44:58.971746 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 02 21:44:58.971778 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:44:58.971773 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 02 21:44:58.977611 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:44:58.977590 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-544f8c6b68-9kc57"] Jul 02 21:44:59.136738 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:44:59.136708 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmf7t\" (UniqueName: \"kubernetes.io/projected/3b00865b-e410-4830-97e5-43ebb328b81c-kube-api-access-nmf7t\") pod \"maas-controller-544f8c6b68-9kc57\" (UID: \"3b00865b-e410-4830-97e5-43ebb328b81c\") " pod="opendatahub/maas-controller-544f8c6b68-9kc57" Jul 02 21:44:59.136738 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:44:59.136742 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3b00865b-e410-4830-97e5-43ebb328b81c-webhook-cert\") pod \"maas-controller-544f8c6b68-9kc57\" (UID: \"3b00865b-e410-4830-97e5-43ebb328b81c\") " pod="opendatahub/maas-controller-544f8c6b68-9kc57" Jul 02 21:44:59.237957 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:44:59.237890 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmf7t\" (UniqueName: \"kubernetes.io/projected/3b00865b-e410-4830-97e5-43ebb328b81c-kube-api-access-nmf7t\") pod \"maas-controller-544f8c6b68-9kc57\" (UID: \"3b00865b-e410-4830-97e5-43ebb328b81c\") " pod="opendatahub/maas-controller-544f8c6b68-9kc57" Jul 02 21:44:59.237957 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:44:59.237922 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3b00865b-e410-4830-97e5-43ebb328b81c-webhook-cert\") pod \"maas-controller-544f8c6b68-9kc57\" (UID: \"3b00865b-e410-4830-97e5-43ebb328b81c\") " pod="opendatahub/maas-controller-544f8c6b68-9kc57" Jul 02 21:44:59.240181 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:44:59.240159 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3b00865b-e410-4830-97e5-43ebb328b81c-webhook-cert\") pod \"maas-controller-544f8c6b68-9kc57\" (UID: \"3b00865b-e410-4830-97e5-43ebb328b81c\") " pod="opendatahub/maas-controller-544f8c6b68-9kc57" Jul 02 21:44:59.245390 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:44:59.245373 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmf7t\" (UniqueName: \"kubernetes.io/projected/3b00865b-e410-4830-97e5-43ebb328b81c-kube-api-access-nmf7t\") pod \"maas-controller-544f8c6b68-9kc57\" (UID: \"3b00865b-e410-4830-97e5-43ebb328b81c\") " pod="opendatahub/maas-controller-544f8c6b68-9kc57" Jul 02 21:44:59.280423 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:44:59.280400 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-544f8c6b68-9kc57" Jul 02 21:44:59.414465 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:44:59.414440 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-544f8c6b68-9kc57"] Jul 02 21:44:59.417011 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:44:59.416985 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b00865b_e410_4830_97e5_43ebb328b81c.slice/crio-8659a16fe0af9a1865e654a8af84cc0c0f6ede1a3c9b05d8e338cb1dc805b055 WatchSource:0}: Error finding container 8659a16fe0af9a1865e654a8af84cc0c0f6ede1a3c9b05d8e338cb1dc805b055: Status 404 returned error can't find the container with id 8659a16fe0af9a1865e654a8af84cc0c0f6ede1a3c9b05d8e338cb1dc805b055 Jul 02 21:44:59.577619 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:44:59.577581 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-544f8c6b68-9kc57" event={"ID":"3b00865b-e410-4830-97e5-43ebb328b81c","Type":"ContainerStarted","Data":"8659a16fe0af9a1865e654a8af84cc0c0f6ede1a3c9b05d8e338cb1dc805b055"} Jul 02 21:45:01.881158 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:01.881136 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 02 21:45:02.599401 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:02.599355 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-544f8c6b68-9kc57" event={"ID":"3b00865b-e410-4830-97e5-43ebb328b81c","Type":"ContainerStarted","Data":"078f0f966b3163893fea57689cb2a7809c6b6d681737dd9d13c86d9bac80daeb"} Jul 02 21:45:02.599596 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:02.599476 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-544f8c6b68-9kc57" Jul 02 21:45:02.619354 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:02.619306 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-544f8c6b68-9kc57" podStartSLOduration=2.159208154 podStartE2EDuration="4.619293332s" totalImagesPullingTime="2.460085178s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:44:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:45:02.617409276 +0000 UTC m=+541.956379711" watchObservedRunningTime="2026-07-02 21:45:02.619293332 +0000 UTC m=+541.958263766" Jul 02 21:45:13.610461 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:13.610427 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-544f8c6b68-9kc57" Jul 02 21:45:34.522849 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:34.522810 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-jqgtg"] Jul 02 21:45:34.527375 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:34.527350 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-jqgtg" Jul 02 21:45:34.529634 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:34.529611 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 02 21:45:34.529735 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:34.529688 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-jgjw7\"" Jul 02 21:45:34.534667 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:34.534643 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-jqgtg"] Jul 02 21:45:34.663838 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:34.663804 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5k8rz\" (UniqueName: \"kubernetes.io/projected/0fe11bea-2cf2-4c63-943a-3edd673560af-kube-api-access-5k8rz\") pod \"payload-processing-58f86c7798-jqgtg\" (UID: \"0fe11bea-2cf2-4c63-943a-3edd673560af\") " pod="openshift-ingress/payload-processing-58f86c7798-jqgtg" Jul 02 21:45:34.664003 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:34.663916 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0fe11bea-2cf2-4c63-943a-3edd673560af-plugins-config-volume\") pod \"payload-processing-58f86c7798-jqgtg\" (UID: \"0fe11bea-2cf2-4c63-943a-3edd673560af\") " pod="openshift-ingress/payload-processing-58f86c7798-jqgtg" Jul 02 21:45:34.764910 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:34.764876 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5k8rz\" (UniqueName: \"kubernetes.io/projected/0fe11bea-2cf2-4c63-943a-3edd673560af-kube-api-access-5k8rz\") pod \"payload-processing-58f86c7798-jqgtg\" (UID: \"0fe11bea-2cf2-4c63-943a-3edd673560af\") " pod="openshift-ingress/payload-processing-58f86c7798-jqgtg" Jul 02 21:45:34.765134 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:34.764972 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0fe11bea-2cf2-4c63-943a-3edd673560af-plugins-config-volume\") pod \"payload-processing-58f86c7798-jqgtg\" (UID: \"0fe11bea-2cf2-4c63-943a-3edd673560af\") " pod="openshift-ingress/payload-processing-58f86c7798-jqgtg" Jul 02 21:45:34.765713 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:34.765688 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0fe11bea-2cf2-4c63-943a-3edd673560af-plugins-config-volume\") pod \"payload-processing-58f86c7798-jqgtg\" (UID: \"0fe11bea-2cf2-4c63-943a-3edd673560af\") " pod="openshift-ingress/payload-processing-58f86c7798-jqgtg" Jul 02 21:45:34.773052 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:34.772991 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5k8rz\" (UniqueName: \"kubernetes.io/projected/0fe11bea-2cf2-4c63-943a-3edd673560af-kube-api-access-5k8rz\") pod \"payload-processing-58f86c7798-jqgtg\" (UID: \"0fe11bea-2cf2-4c63-943a-3edd673560af\") " pod="openshift-ingress/payload-processing-58f86c7798-jqgtg" Jul 02 21:45:34.839223 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:34.839191 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-jqgtg" Jul 02 21:45:34.982845 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:34.982648 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-jqgtg"] Jul 02 21:45:34.985417 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:45:34.985388 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fe11bea_2cf2_4c63_943a_3edd673560af.slice/crio-4f2bc7f7d5aa7bdd7ab9eb5af9894a9c8412f97d6452bd51a913d523b85fd7bd WatchSource:0}: Error finding container 4f2bc7f7d5aa7bdd7ab9eb5af9894a9c8412f97d6452bd51a913d523b85fd7bd: Status 404 returned error can't find the container with id 4f2bc7f7d5aa7bdd7ab9eb5af9894a9c8412f97d6452bd51a913d523b85fd7bd Jul 02 21:45:35.798826 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:35.798731 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-jqgtg" event={"ID":"0fe11bea-2cf2-4c63-943a-3edd673560af","Type":"ContainerStarted","Data":"4f2bc7f7d5aa7bdd7ab9eb5af9894a9c8412f97d6452bd51a913d523b85fd7bd"} Jul 02 21:45:36.807335 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:36.807287 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-jqgtg" event={"ID":"0fe11bea-2cf2-4c63-943a-3edd673560af","Type":"ContainerStarted","Data":"8e94ddf7eecfa587145f316da0401bbf5e8bf1072c42b62437d40504f4f316cd"} Jul 02 21:45:36.807743 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:36.807360 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-jqgtg" Jul 02 21:45:36.822699 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:36.822644 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-jqgtg" podStartSLOduration=1.1822860259999999 podStartE2EDuration="2.82262549s" totalImagesPullingTime="1.640339464s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:45:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:45:36.822056517 +0000 UTC m=+576.161026964" watchObservedRunningTime="2026-07-02 21:45:36.82262549 +0000 UTC m=+576.161595926" Jul 02 21:45:42.077517 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:42.077486 2569 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-5tjmq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": read tcp 10.132.0.2:54292->10.132.0.42:15021: read: connection reset by peer" start-of-body= Jul 02 21:45:42.078085 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:42.077556 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5tjmq" podUID="2749fe4b-86bd-4342-863c-ab4e0bd8bd24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": read tcp 10.132.0.2:54292->10.132.0.42:15021: read: connection reset by peer" Jul 02 21:45:42.840859 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:42.840827 2569 generic.go:356] "Generic (PLEG): container finished" podID="2749fe4b-86bd-4342-863c-ab4e0bd8bd24" containerID="6aa8ac15281040fcd64d3632832db524796b2708800739cd91effce9abc31e48" exitCode=0 Jul 02 21:45:42.841013 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:42.840905 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5tjmq" event={"ID":"2749fe4b-86bd-4342-863c-ab4e0bd8bd24","Type":"ContainerDied","Data":"6aa8ac15281040fcd64d3632832db524796b2708800739cd91effce9abc31e48"} Jul 02 21:45:42.841281 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:42.841268 2569 scope.go:117] "RemoveContainer" containerID="6aa8ac15281040fcd64d3632832db524796b2708800739cd91effce9abc31e48" Jul 02 21:45:42.842192 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:42.842167 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Jul 02 21:45:42.842245 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:42.842233 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Jul 02 21:45:42.842279 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:42.842268 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Jul 02 21:45:43.848050 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:43.848016 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5tjmq" event={"ID":"2749fe4b-86bd-4342-863c-ab4e0bd8bd24","Type":"ContainerStarted","Data":"51e28c75197a2ffa11019ef5cfaa7ddb76930c5ea7870c94b73ca02cdfe7ac6d"} Jul 02 21:45:44.810551 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:44.810514 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5tjmq" Jul 02 21:45:44.810744 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:44.810717 2569 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-5tjmq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 02 21:45:44.810834 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:44.810760 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5tjmq" podUID="2749fe4b-86bd-4342-863c-ab4e0bd8bd24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 02 21:45:45.810643 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:45.810603 2569 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-5tjmq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 02 21:45:45.811046 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:45.810658 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5tjmq" podUID="2749fe4b-86bd-4342-863c-ab4e0bd8bd24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 02 21:45:46.811223 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:46.811186 2569 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-5tjmq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 02 21:45:46.811642 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:46.811243 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5tjmq" podUID="2749fe4b-86bd-4342-863c-ab4e0bd8bd24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 02 21:45:47.811235 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:47.811197 2569 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-5tjmq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 02 21:45:47.811651 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:47.811258 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5tjmq" podUID="2749fe4b-86bd-4342-863c-ab4e0bd8bd24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 02 21:45:47.815279 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:47.815255 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-jqgtg" Jul 02 21:45:48.813635 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:48.813606 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5tjmq" Jul 02 21:45:48.814157 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:48.814018 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5tjmq" Jul 02 21:45:48.814595 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:48.814577 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5tjmq" Jul 02 21:45:49.842872 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:49.842839 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dgchj"] Jul 02 21:45:49.845339 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:49.845324 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dgchj" Jul 02 21:45:49.847932 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:49.847908 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 02 21:45:49.848091 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:49.848037 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-m6vlx\"" Jul 02 21:45:49.848764 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:49.848749 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 02 21:45:49.848885 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:49.848761 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 02 21:45:49.855012 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:49.854993 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dgchj"] Jul 02 21:45:49.933157 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:49.933126 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd2bf3ef-4f49-46cf-87cf-ee102b6d8fe5-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dgchj\" (UID: \"dd2bf3ef-4f49-46cf-87cf-ee102b6d8fe5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dgchj" Jul 02 21:45:49.933401 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:49.933192 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd2bf3ef-4f49-46cf-87cf-ee102b6d8fe5-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dgchj\" (UID: \"dd2bf3ef-4f49-46cf-87cf-ee102b6d8fe5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dgchj" Jul 02 21:45:49.933401 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:49.933254 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd2bf3ef-4f49-46cf-87cf-ee102b6d8fe5-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dgchj\" (UID: \"dd2bf3ef-4f49-46cf-87cf-ee102b6d8fe5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dgchj" Jul 02 21:45:49.933401 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:49.933314 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd2bf3ef-4f49-46cf-87cf-ee102b6d8fe5-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dgchj\" (UID: \"dd2bf3ef-4f49-46cf-87cf-ee102b6d8fe5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dgchj" Jul 02 21:45:49.933593 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:49.933397 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd2bf3ef-4f49-46cf-87cf-ee102b6d8fe5-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dgchj\" (UID: \"dd2bf3ef-4f49-46cf-87cf-ee102b6d8fe5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dgchj" Jul 02 21:45:49.933593 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:49.933469 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jrl7\" (UniqueName: \"kubernetes.io/projected/dd2bf3ef-4f49-46cf-87cf-ee102b6d8fe5-kube-api-access-6jrl7\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dgchj\" (UID: \"dd2bf3ef-4f49-46cf-87cf-ee102b6d8fe5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dgchj" Jul 02 21:45:50.034810 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:50.034750 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd2bf3ef-4f49-46cf-87cf-ee102b6d8fe5-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dgchj\" (UID: \"dd2bf3ef-4f49-46cf-87cf-ee102b6d8fe5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dgchj" Jul 02 21:45:50.034970 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:50.034823 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd2bf3ef-4f49-46cf-87cf-ee102b6d8fe5-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dgchj\" (UID: \"dd2bf3ef-4f49-46cf-87cf-ee102b6d8fe5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dgchj" Jul 02 21:45:50.034970 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:50.034865 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jrl7\" (UniqueName: \"kubernetes.io/projected/dd2bf3ef-4f49-46cf-87cf-ee102b6d8fe5-kube-api-access-6jrl7\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dgchj\" (UID: \"dd2bf3ef-4f49-46cf-87cf-ee102b6d8fe5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dgchj" Jul 02 21:45:50.034970 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:50.034933 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd2bf3ef-4f49-46cf-87cf-ee102b6d8fe5-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dgchj\" (UID: \"dd2bf3ef-4f49-46cf-87cf-ee102b6d8fe5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dgchj" Jul 02 21:45:50.035132 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:50.034971 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd2bf3ef-4f49-46cf-87cf-ee102b6d8fe5-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dgchj\" (UID: \"dd2bf3ef-4f49-46cf-87cf-ee102b6d8fe5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dgchj" Jul 02 21:45:50.035132 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:50.034998 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd2bf3ef-4f49-46cf-87cf-ee102b6d8fe5-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dgchj\" (UID: \"dd2bf3ef-4f49-46cf-87cf-ee102b6d8fe5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dgchj" Jul 02 21:45:50.035232 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:50.035215 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd2bf3ef-4f49-46cf-87cf-ee102b6d8fe5-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dgchj\" (UID: \"dd2bf3ef-4f49-46cf-87cf-ee102b6d8fe5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dgchj" Jul 02 21:45:50.035306 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:50.035262 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd2bf3ef-4f49-46cf-87cf-ee102b6d8fe5-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dgchj\" (UID: \"dd2bf3ef-4f49-46cf-87cf-ee102b6d8fe5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dgchj" Jul 02 21:45:50.035369 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:50.035323 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd2bf3ef-4f49-46cf-87cf-ee102b6d8fe5-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dgchj\" (UID: \"dd2bf3ef-4f49-46cf-87cf-ee102b6d8fe5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dgchj" Jul 02 21:45:50.037202 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:50.037179 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd2bf3ef-4f49-46cf-87cf-ee102b6d8fe5-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dgchj\" (UID: \"dd2bf3ef-4f49-46cf-87cf-ee102b6d8fe5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dgchj" Jul 02 21:45:50.037380 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:50.037363 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd2bf3ef-4f49-46cf-87cf-ee102b6d8fe5-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dgchj\" (UID: \"dd2bf3ef-4f49-46cf-87cf-ee102b6d8fe5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dgchj" Jul 02 21:45:50.043066 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:50.043048 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jrl7\" (UniqueName: \"kubernetes.io/projected/dd2bf3ef-4f49-46cf-87cf-ee102b6d8fe5-kube-api-access-6jrl7\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dgchj\" (UID: \"dd2bf3ef-4f49-46cf-87cf-ee102b6d8fe5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dgchj" Jul 02 21:45:50.158049 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:50.157968 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dgchj" Jul 02 21:45:50.295594 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:50.295564 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dgchj"] Jul 02 21:45:50.296915 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:45:50.296886 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd2bf3ef_4f49_46cf_87cf_ee102b6d8fe5.slice/crio-fc9acaa6ad7b3068961d83c0b86b0892ff409f3316e955a2ab2dac1ff117c39e WatchSource:0}: Error finding container fc9acaa6ad7b3068961d83c0b86b0892ff409f3316e955a2ab2dac1ff117c39e: Status 404 returned error can't find the container with id fc9acaa6ad7b3068961d83c0b86b0892ff409f3316e955a2ab2dac1ff117c39e Jul 02 21:45:50.894658 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:50.894619 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dgchj" event={"ID":"dd2bf3ef-4f49-46cf-87cf-ee102b6d8fe5","Type":"ContainerStarted","Data":"fc9acaa6ad7b3068961d83c0b86b0892ff409f3316e955a2ab2dac1ff117c39e"} Jul 02 21:45:51.901226 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:51.901186 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dgchj" event={"ID":"dd2bf3ef-4f49-46cf-87cf-ee102b6d8fe5","Type":"ContainerStarted","Data":"a2d12f27b0744d64475a4aaf63ab27f115c3894a143e1893d7e478e711438458"} Jul 02 21:45:51.920899 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:45:51.920852 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dgchj" podStartSLOduration=1.820610269 podStartE2EDuration="2.920837253s" totalImagesPullingTime="1.100226984s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:45:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:45:51.919128216 +0000 UTC m=+591.258098653" watchObservedRunningTime="2026-07-02 21:45:51.920837253 +0000 UTC m=+591.259807687" Jul 02 21:46:00.158423 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:00.158394 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dgchj" Jul 02 21:46:00.158816 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:00.158487 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dgchj" Jul 02 21:46:00.170756 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:00.170734 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dgchj" Jul 02 21:46:00.960377 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:00.960346 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dgchj" Jul 02 21:46:19.852404 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:19.852371 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-727mw"] Jul 02 21:46:19.858079 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:19.858060 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-727mw" Jul 02 21:46:19.861445 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:19.861427 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 02 21:46:19.867309 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:19.867284 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-727mw"] Jul 02 21:46:20.021237 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:20.021193 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ad662ca-f02d-4364-8486-b462220ed2dc-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-727mw\" (UID: \"0ad662ca-f02d-4364-8486-b462220ed2dc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-727mw" Jul 02 21:46:20.021432 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:20.021272 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ad662ca-f02d-4364-8486-b462220ed2dc-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-727mw\" (UID: \"0ad662ca-f02d-4364-8486-b462220ed2dc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-727mw" Jul 02 21:46:20.021432 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:20.021324 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ad662ca-f02d-4364-8486-b462220ed2dc-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-727mw\" (UID: \"0ad662ca-f02d-4364-8486-b462220ed2dc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-727mw" Jul 02 21:46:20.021432 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:20.021377 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ad662ca-f02d-4364-8486-b462220ed2dc-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-727mw\" (UID: \"0ad662ca-f02d-4364-8486-b462220ed2dc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-727mw" Jul 02 21:46:20.021603 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:20.021441 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ad662ca-f02d-4364-8486-b462220ed2dc-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-727mw\" (UID: \"0ad662ca-f02d-4364-8486-b462220ed2dc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-727mw" Jul 02 21:46:20.021603 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:20.021472 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kc62\" (UniqueName: \"kubernetes.io/projected/0ad662ca-f02d-4364-8486-b462220ed2dc-kube-api-access-7kc62\") pod \"e2e-trlp-test-simulated-kserve-598466548d-727mw\" (UID: \"0ad662ca-f02d-4364-8486-b462220ed2dc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-727mw" Jul 02 21:46:20.122896 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:20.122818 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ad662ca-f02d-4364-8486-b462220ed2dc-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-727mw\" (UID: \"0ad662ca-f02d-4364-8486-b462220ed2dc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-727mw" Jul 02 21:46:20.122896 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:20.122851 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kc62\" (UniqueName: \"kubernetes.io/projected/0ad662ca-f02d-4364-8486-b462220ed2dc-kube-api-access-7kc62\") pod \"e2e-trlp-test-simulated-kserve-598466548d-727mw\" (UID: \"0ad662ca-f02d-4364-8486-b462220ed2dc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-727mw" Jul 02 21:46:20.122896 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:20.122879 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ad662ca-f02d-4364-8486-b462220ed2dc-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-727mw\" (UID: \"0ad662ca-f02d-4364-8486-b462220ed2dc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-727mw" Jul 02 21:46:20.123178 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:20.122916 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ad662ca-f02d-4364-8486-b462220ed2dc-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-727mw\" (UID: \"0ad662ca-f02d-4364-8486-b462220ed2dc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-727mw" Jul 02 21:46:20.123178 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:20.122947 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ad662ca-f02d-4364-8486-b462220ed2dc-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-727mw\" (UID: \"0ad662ca-f02d-4364-8486-b462220ed2dc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-727mw" Jul 02 21:46:20.123178 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:20.122986 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ad662ca-f02d-4364-8486-b462220ed2dc-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-727mw\" (UID: \"0ad662ca-f02d-4364-8486-b462220ed2dc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-727mw" Jul 02 21:46:20.123337 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:20.123277 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ad662ca-f02d-4364-8486-b462220ed2dc-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-727mw\" (UID: \"0ad662ca-f02d-4364-8486-b462220ed2dc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-727mw" Jul 02 21:46:20.123382 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:20.123346 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ad662ca-f02d-4364-8486-b462220ed2dc-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-727mw\" (UID: \"0ad662ca-f02d-4364-8486-b462220ed2dc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-727mw" Jul 02 21:46:20.123382 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:20.123353 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ad662ca-f02d-4364-8486-b462220ed2dc-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-727mw\" (UID: \"0ad662ca-f02d-4364-8486-b462220ed2dc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-727mw" Jul 02 21:46:20.125270 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:20.125243 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ad662ca-f02d-4364-8486-b462220ed2dc-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-727mw\" (UID: \"0ad662ca-f02d-4364-8486-b462220ed2dc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-727mw" Jul 02 21:46:20.125398 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:20.125385 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ad662ca-f02d-4364-8486-b462220ed2dc-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-727mw\" (UID: \"0ad662ca-f02d-4364-8486-b462220ed2dc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-727mw" Jul 02 21:46:20.131912 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:20.131891 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kc62\" (UniqueName: \"kubernetes.io/projected/0ad662ca-f02d-4364-8486-b462220ed2dc-kube-api-access-7kc62\") pod \"e2e-trlp-test-simulated-kserve-598466548d-727mw\" (UID: \"0ad662ca-f02d-4364-8486-b462220ed2dc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-727mw" Jul 02 21:46:20.170200 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:20.170172 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-727mw" Jul 02 21:46:20.311254 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:20.311225 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-727mw"] Jul 02 21:46:20.316971 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:46:20.316944 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ad662ca_f02d_4364_8486_b462220ed2dc.slice/crio-e86707723794b727fb021a3af3f927a7872225486dc382d346213b2a565e70c0 WatchSource:0}: Error finding container e86707723794b727fb021a3af3f927a7872225486dc382d346213b2a565e70c0: Status 404 returned error can't find the container with id e86707723794b727fb021a3af3f927a7872225486dc382d346213b2a565e70c0 Jul 02 21:46:20.318433 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:20.318416 2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 21:46:21.048321 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:21.048288 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-727mw" event={"ID":"0ad662ca-f02d-4364-8486-b462220ed2dc","Type":"ContainerStarted","Data":"c6ea18e016122e33ea58fe348094754004bf45d1b23f57b34086b7b841c5af4d"} Jul 02 21:46:21.048685 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:21.048334 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-727mw" event={"ID":"0ad662ca-f02d-4364-8486-b462220ed2dc","Type":"ContainerStarted","Data":"e86707723794b727fb021a3af3f927a7872225486dc382d346213b2a565e70c0"} Jul 02 21:46:21.066957 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:21.066910 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-727mw" podStartSLOduration=1.877040693 podStartE2EDuration="2.066896051s" totalImagesPullingTime="189.855358ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:46:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:46:21.06454401 +0000 UTC m=+620.403514445" watchObservedRunningTime="2026-07-02 21:46:21.066896051 +0000 UTC m=+620.405866487" Jul 02 21:46:30.171429 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:30.171354 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-727mw" Jul 02 21:46:30.171429 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:30.171428 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-727mw" Jul 02 21:46:30.183719 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:30.183690 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-727mw" Jul 02 21:46:31.117215 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:31.117187 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-727mw" Jul 02 21:46:51.337398 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:51.337360 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-9clz6"] Jul 02 21:46:51.343160 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:51.343135 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9clz6" Jul 02 21:46:51.345836 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:51.345809 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 02 21:46:51.349360 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:51.349335 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-9clz6"] Jul 02 21:46:51.411744 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:51.411692 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a374984-8c70-4e94-b998-7882bf32e372-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9clz6\" (UID: \"4a374984-8c70-4e94-b998-7882bf32e372\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9clz6" Jul 02 21:46:51.411744 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:51.411747 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a374984-8c70-4e94-b998-7882bf32e372-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9clz6\" (UID: \"4a374984-8c70-4e94-b998-7882bf32e372\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9clz6" Jul 02 21:46:51.412019 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:51.411840 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a374984-8c70-4e94-b998-7882bf32e372-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9clz6\" (UID: \"4a374984-8c70-4e94-b998-7882bf32e372\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9clz6" Jul 02 21:46:51.412019 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:51.411901 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a374984-8c70-4e94-b998-7882bf32e372-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9clz6\" (UID: \"4a374984-8c70-4e94-b998-7882bf32e372\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9clz6" Jul 02 21:46:51.412124 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:51.412038 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a374984-8c70-4e94-b998-7882bf32e372-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9clz6\" (UID: \"4a374984-8c70-4e94-b998-7882bf32e372\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9clz6" Jul 02 21:46:51.412124 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:51.412099 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cj9nk\" (UniqueName: \"kubernetes.io/projected/4a374984-8c70-4e94-b998-7882bf32e372-kube-api-access-cj9nk\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9clz6\" (UID: \"4a374984-8c70-4e94-b998-7882bf32e372\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9clz6" Jul 02 21:46:51.512628 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:51.512585 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a374984-8c70-4e94-b998-7882bf32e372-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9clz6\" (UID: \"4a374984-8c70-4e94-b998-7882bf32e372\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9clz6" Jul 02 21:46:51.512835 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:51.512698 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a374984-8c70-4e94-b998-7882bf32e372-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9clz6\" (UID: \"4a374984-8c70-4e94-b998-7882bf32e372\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9clz6" Jul 02 21:46:51.512835 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:51.512734 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cj9nk\" (UniqueName: \"kubernetes.io/projected/4a374984-8c70-4e94-b998-7882bf32e372-kube-api-access-cj9nk\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9clz6\" (UID: \"4a374984-8c70-4e94-b998-7882bf32e372\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9clz6" Jul 02 21:46:51.512835 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:51.512825 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a374984-8c70-4e94-b998-7882bf32e372-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9clz6\" (UID: \"4a374984-8c70-4e94-b998-7882bf32e372\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9clz6" Jul 02 21:46:51.513021 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:51.512854 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a374984-8c70-4e94-b998-7882bf32e372-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9clz6\" (UID: \"4a374984-8c70-4e94-b998-7882bf32e372\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9clz6" Jul 02 21:46:51.513021 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:51.512882 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a374984-8c70-4e94-b998-7882bf32e372-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9clz6\" (UID: \"4a374984-8c70-4e94-b998-7882bf32e372\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9clz6" Jul 02 21:46:51.513167 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:51.513141 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a374984-8c70-4e94-b998-7882bf32e372-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9clz6\" (UID: \"4a374984-8c70-4e94-b998-7882bf32e372\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9clz6" Jul 02 21:46:51.513237 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:51.513168 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a374984-8c70-4e94-b998-7882bf32e372-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9clz6\" (UID: \"4a374984-8c70-4e94-b998-7882bf32e372\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9clz6" Jul 02 21:46:51.513237 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:51.513211 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a374984-8c70-4e94-b998-7882bf32e372-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9clz6\" (UID: \"4a374984-8c70-4e94-b998-7882bf32e372\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9clz6" Jul 02 21:46:51.515987 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:51.515964 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a374984-8c70-4e94-b998-7882bf32e372-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9clz6\" (UID: \"4a374984-8c70-4e94-b998-7882bf32e372\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9clz6" Jul 02 21:46:51.516133 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:51.516112 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a374984-8c70-4e94-b998-7882bf32e372-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9clz6\" (UID: \"4a374984-8c70-4e94-b998-7882bf32e372\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9clz6" Jul 02 21:46:51.520562 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:51.520537 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cj9nk\" (UniqueName: \"kubernetes.io/projected/4a374984-8c70-4e94-b998-7882bf32e372-kube-api-access-cj9nk\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9clz6\" (UID: \"4a374984-8c70-4e94-b998-7882bf32e372\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9clz6" Jul 02 21:46:51.656574 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:51.656472 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9clz6" Jul 02 21:46:51.800347 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:51.800319 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-9clz6"] Jul 02 21:46:51.803010 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:46:51.802978 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a374984_8c70_4e94_b998_7882bf32e372.slice/crio-7d2e64a74a2a284ff29ea875966cd15dfeeafa20f3675e18a208fca03c786e4e WatchSource:0}: Error finding container 7d2e64a74a2a284ff29ea875966cd15dfeeafa20f3675e18a208fca03c786e4e: Status 404 returned error can't find the container with id 7d2e64a74a2a284ff29ea875966cd15dfeeafa20f3675e18a208fca03c786e4e Jul 02 21:46:52.214105 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:52.214062 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9clz6" event={"ID":"4a374984-8c70-4e94-b998-7882bf32e372","Type":"ContainerStarted","Data":"ba201adb158f10909bd8a7303e27ed3159748fc7e2ad0a86f6b68d1ca1d38574"} Jul 02 21:46:52.214298 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:52.214129 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9clz6" event={"ID":"4a374984-8c70-4e94-b998-7882bf32e372","Type":"ContainerStarted","Data":"7d2e64a74a2a284ff29ea875966cd15dfeeafa20f3675e18a208fca03c786e4e"} Jul 02 21:46:52.233063 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:46:52.233014 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9clz6" podStartSLOduration=1.036560075 podStartE2EDuration="1.232998898s" totalImagesPullingTime="196.438823ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:46:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:46:52.230877234 +0000 UTC m=+651.569847669" watchObservedRunningTime="2026-07-02 21:46:52.232998898 +0000 UTC m=+651.571969333" Jul 02 21:47:01.656902 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:47:01.656853 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9clz6" Jul 02 21:47:01.657324 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:47:01.656924 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9clz6" Jul 02 21:47:01.670072 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:47:01.670047 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9clz6" Jul 02 21:47:02.278233 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:47:02.278196 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9clz6" Jul 02 21:49:46.642847 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:49:46.642813 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-544f8c6b68-9kc57"] Jul 02 21:49:46.643339 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:49:46.643051 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-544f8c6b68-9kc57" podUID="3b00865b-e410-4830-97e5-43ebb328b81c" containerName="manager" containerID="cri-o://078f0f966b3163893fea57689cb2a7809c6b6d681737dd9d13c86d9bac80daeb" gracePeriod=30 Jul 02 21:49:46.894350 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:49:46.894292 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-544f8c6b68-9kc57" Jul 02 21:49:47.007226 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:49:47.007193 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nmf7t\" (UniqueName: \"kubernetes.io/projected/3b00865b-e410-4830-97e5-43ebb328b81c-kube-api-access-nmf7t\") pod \"3b00865b-e410-4830-97e5-43ebb328b81c\" (UID: \"3b00865b-e410-4830-97e5-43ebb328b81c\") " Jul 02 21:49:47.007389 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:49:47.007307 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3b00865b-e410-4830-97e5-43ebb328b81c-webhook-cert\") pod \"3b00865b-e410-4830-97e5-43ebb328b81c\" (UID: \"3b00865b-e410-4830-97e5-43ebb328b81c\") " Jul 02 21:49:47.009443 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:49:47.009415 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b00865b-e410-4830-97e5-43ebb328b81c-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "3b00865b-e410-4830-97e5-43ebb328b81c" (UID: "3b00865b-e410-4830-97e5-43ebb328b81c"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 21:49:47.009552 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:49:47.009490 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b00865b-e410-4830-97e5-43ebb328b81c-kube-api-access-nmf7t" (OuterVolumeSpecName: "kube-api-access-nmf7t") pod "3b00865b-e410-4830-97e5-43ebb328b81c" (UID: "3b00865b-e410-4830-97e5-43ebb328b81c"). InnerVolumeSpecName "kube-api-access-nmf7t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 21:49:47.108999 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:49:47.108963 2569 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3b00865b-e410-4830-97e5-43ebb328b81c-webhook-cert\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 21:49:47.108999 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:49:47.108993 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nmf7t\" (UniqueName: \"kubernetes.io/projected/3b00865b-e410-4830-97e5-43ebb328b81c-kube-api-access-nmf7t\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 21:49:47.134699 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:49:47.134661 2569 generic.go:356] "Generic (PLEG): container finished" podID="3b00865b-e410-4830-97e5-43ebb328b81c" containerID="078f0f966b3163893fea57689cb2a7809c6b6d681737dd9d13c86d9bac80daeb" exitCode=0 Jul 02 21:49:47.134909 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:49:47.134740 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-544f8c6b68-9kc57" event={"ID":"3b00865b-e410-4830-97e5-43ebb328b81c","Type":"ContainerDied","Data":"078f0f966b3163893fea57689cb2a7809c6b6d681737dd9d13c86d9bac80daeb"} Jul 02 21:49:47.134909 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:49:47.134752 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-544f8c6b68-9kc57" Jul 02 21:49:47.134909 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:49:47.134835 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-544f8c6b68-9kc57" event={"ID":"3b00865b-e410-4830-97e5-43ebb328b81c","Type":"ContainerDied","Data":"8659a16fe0af9a1865e654a8af84cc0c0f6ede1a3c9b05d8e338cb1dc805b055"} Jul 02 21:49:47.134909 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:49:47.134870 2569 scope.go:117] "RemoveContainer" containerID="078f0f966b3163893fea57689cb2a7809c6b6d681737dd9d13c86d9bac80daeb" Jul 02 21:49:47.145154 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:49:47.145137 2569 scope.go:117] "RemoveContainer" containerID="078f0f966b3163893fea57689cb2a7809c6b6d681737dd9d13c86d9bac80daeb" Jul 02 21:49:47.145402 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:49:47.145380 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"078f0f966b3163893fea57689cb2a7809c6b6d681737dd9d13c86d9bac80daeb\": container with ID starting with 078f0f966b3163893fea57689cb2a7809c6b6d681737dd9d13c86d9bac80daeb not found: ID does not exist" containerID="078f0f966b3163893fea57689cb2a7809c6b6d681737dd9d13c86d9bac80daeb" Jul 02 21:49:47.145467 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:49:47.145425 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"078f0f966b3163893fea57689cb2a7809c6b6d681737dd9d13c86d9bac80daeb"} err="failed to get container status \"078f0f966b3163893fea57689cb2a7809c6b6d681737dd9d13c86d9bac80daeb\": rpc error: code = NotFound desc = could not find container \"078f0f966b3163893fea57689cb2a7809c6b6d681737dd9d13c86d9bac80daeb\": container with ID starting with 078f0f966b3163893fea57689cb2a7809c6b6d681737dd9d13c86d9bac80daeb not found: ID does not exist" Jul 02 21:49:47.156875 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:49:47.156848 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-544f8c6b68-9kc57"] Jul 02 21:49:47.158976 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:49:47.158954 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-544f8c6b68-9kc57"] Jul 02 21:49:48.956611 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:49:48.956580 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b00865b-e410-4830-97e5-43ebb328b81c" path="/var/lib/kubelet/pods/3b00865b-e410-4830-97e5-43ebb328b81c/volumes" Jul 02 21:54:26.857977 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:54:26.857899 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rppgc"] Jul 02 21:54:26.858495 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:54:26.858084 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rppgc" podUID="37f15a25-f233-47ad-acde-94dcb577b1d2" containerName="manager" containerID="cri-o://be1642754aa06cd086dc70791c9c94865ffc6f3a715766932b08aeaa6a894df5" gracePeriod=10 Jul 02 21:54:30.515915 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:54:30.515894 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rppgc" Jul 02 21:54:30.600849 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:54:30.600781 2569 generic.go:356] "Generic (PLEG): container finished" podID="37f15a25-f233-47ad-acde-94dcb577b1d2" containerID="be1642754aa06cd086dc70791c9c94865ffc6f3a715766932b08aeaa6a894df5" exitCode=0 Jul 02 21:54:30.600849 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:54:30.600845 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rppgc" Jul 02 21:54:30.601081 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:54:30.600853 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rppgc" event={"ID":"37f15a25-f233-47ad-acde-94dcb577b1d2","Type":"ContainerDied","Data":"be1642754aa06cd086dc70791c9c94865ffc6f3a715766932b08aeaa6a894df5"} Jul 02 21:54:30.601081 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:54:30.600913 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rppgc" event={"ID":"37f15a25-f233-47ad-acde-94dcb577b1d2","Type":"ContainerDied","Data":"ee0400b3e8141e50856d9d69033be34150c7c40240162d8fdad3ea06934dcdca"} Jul 02 21:54:30.601081 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:54:30.600940 2569 scope.go:117] "RemoveContainer" containerID="be1642754aa06cd086dc70791c9c94865ffc6f3a715766932b08aeaa6a894df5" Jul 02 21:54:30.610020 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:54:30.610003 2569 scope.go:117] "RemoveContainer" containerID="be1642754aa06cd086dc70791c9c94865ffc6f3a715766932b08aeaa6a894df5" Jul 02 21:54:30.610258 ip-10-0-139-75 kubenswrapper[2569]: E0702 21:54:30.610242 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be1642754aa06cd086dc70791c9c94865ffc6f3a715766932b08aeaa6a894df5\": container with ID starting with be1642754aa06cd086dc70791c9c94865ffc6f3a715766932b08aeaa6a894df5 not found: ID does not exist" containerID="be1642754aa06cd086dc70791c9c94865ffc6f3a715766932b08aeaa6a894df5" Jul 02 21:54:30.610320 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:54:30.610265 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be1642754aa06cd086dc70791c9c94865ffc6f3a715766932b08aeaa6a894df5"} err="failed to get container status \"be1642754aa06cd086dc70791c9c94865ffc6f3a715766932b08aeaa6a894df5\": rpc error: code = NotFound desc = could not find container \"be1642754aa06cd086dc70791c9c94865ffc6f3a715766932b08aeaa6a894df5\": container with ID starting with be1642754aa06cd086dc70791c9c94865ffc6f3a715766932b08aeaa6a894df5 not found: ID does not exist" Jul 02 21:54:30.700512 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:54:30.700430 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5mzsw\" (UniqueName: \"kubernetes.io/projected/37f15a25-f233-47ad-acde-94dcb577b1d2-kube-api-access-5mzsw\") pod \"37f15a25-f233-47ad-acde-94dcb577b1d2\" (UID: \"37f15a25-f233-47ad-acde-94dcb577b1d2\") " Jul 02 21:54:30.700650 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:54:30.700534 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/37f15a25-f233-47ad-acde-94dcb577b1d2-extensions-socket-volume\") pod \"37f15a25-f233-47ad-acde-94dcb577b1d2\" (UID: \"37f15a25-f233-47ad-acde-94dcb577b1d2\") " Jul 02 21:54:30.700898 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:54:30.700876 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37f15a25-f233-47ad-acde-94dcb577b1d2-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "37f15a25-f233-47ad-acde-94dcb577b1d2" (UID: "37f15a25-f233-47ad-acde-94dcb577b1d2"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 21:54:30.702460 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:54:30.702434 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37f15a25-f233-47ad-acde-94dcb577b1d2-kube-api-access-5mzsw" (OuterVolumeSpecName: "kube-api-access-5mzsw") pod "37f15a25-f233-47ad-acde-94dcb577b1d2" (UID: "37f15a25-f233-47ad-acde-94dcb577b1d2"). InnerVolumeSpecName "kube-api-access-5mzsw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 21:54:30.801676 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:54:30.801644 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5mzsw\" (UniqueName: \"kubernetes.io/projected/37f15a25-f233-47ad-acde-94dcb577b1d2-kube-api-access-5mzsw\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 21:54:30.801676 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:54:30.801673 2569 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/37f15a25-f233-47ad-acde-94dcb577b1d2-extensions-socket-volume\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 21:54:30.927589 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:54:30.927560 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rppgc"] Jul 02 21:54:30.932484 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:54:30.932455 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rppgc"] Jul 02 21:54:30.956944 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:54:30.956882 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37f15a25-f233-47ad-acde-94dcb577b1d2" path="/var/lib/kubelet/pods/37f15a25-f233-47ad-acde-94dcb577b1d2/volumes" Jul 02 21:55:28.046554 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:55:28.046475 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rgb8z"] Jul 02 21:55:28.047026 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:55:28.046937 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b00865b-e410-4830-97e5-43ebb328b81c" containerName="manager" Jul 02 21:55:28.047026 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:55:28.046949 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b00865b-e410-4830-97e5-43ebb328b81c" containerName="manager" Jul 02 21:55:28.047026 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:55:28.046966 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37f15a25-f233-47ad-acde-94dcb577b1d2" containerName="manager" Jul 02 21:55:28.047026 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:55:28.046973 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="37f15a25-f233-47ad-acde-94dcb577b1d2" containerName="manager" Jul 02 21:55:28.047164 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:55:28.047040 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="37f15a25-f233-47ad-acde-94dcb577b1d2" containerName="manager" Jul 02 21:55:28.047164 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:55:28.047049 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b00865b-e410-4830-97e5-43ebb328b81c" containerName="manager" Jul 02 21:55:28.051340 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:55:28.051323 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rgb8z" Jul 02 21:55:28.057562 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:55:28.057543 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-q9lv2\"" Jul 02 21:55:28.078497 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:55:28.078472 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rgb8z"] Jul 02 21:55:28.147685 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:55:28.147650 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/13dad243-1dd3-4e64-a799-15e96c4caadf-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-rgb8z\" (UID: \"13dad243-1dd3-4e64-a799-15e96c4caadf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rgb8z" Jul 02 21:55:28.147870 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:55:28.147697 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tj7z\" (UniqueName: \"kubernetes.io/projected/13dad243-1dd3-4e64-a799-15e96c4caadf-kube-api-access-7tj7z\") pod \"kuadrant-operator-controller-manager-86878c769f-rgb8z\" (UID: \"13dad243-1dd3-4e64-a799-15e96c4caadf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rgb8z" Jul 02 21:55:28.249148 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:55:28.249111 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/13dad243-1dd3-4e64-a799-15e96c4caadf-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-rgb8z\" (UID: \"13dad243-1dd3-4e64-a799-15e96c4caadf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rgb8z" Jul 02 21:55:28.249369 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:55:28.249158 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tj7z\" (UniqueName: \"kubernetes.io/projected/13dad243-1dd3-4e64-a799-15e96c4caadf-kube-api-access-7tj7z\") pod \"kuadrant-operator-controller-manager-86878c769f-rgb8z\" (UID: \"13dad243-1dd3-4e64-a799-15e96c4caadf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rgb8z" Jul 02 21:55:28.249568 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:55:28.249543 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/13dad243-1dd3-4e64-a799-15e96c4caadf-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-rgb8z\" (UID: \"13dad243-1dd3-4e64-a799-15e96c4caadf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rgb8z" Jul 02 21:55:28.260205 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:55:28.260180 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tj7z\" (UniqueName: \"kubernetes.io/projected/13dad243-1dd3-4e64-a799-15e96c4caadf-kube-api-access-7tj7z\") pod \"kuadrant-operator-controller-manager-86878c769f-rgb8z\" (UID: \"13dad243-1dd3-4e64-a799-15e96c4caadf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rgb8z" Jul 02 21:55:28.361531 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:55:28.361447 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rgb8z" Jul 02 21:55:28.499641 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:55:28.499616 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rgb8z"] Jul 02 21:55:28.502190 ip-10-0-139-75 kubenswrapper[2569]: W0702 21:55:28.502158 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13dad243_1dd3_4e64_a799_15e96c4caadf.slice/crio-410f42948b4a782a8807b1ba009a2945c4ca93832ccb681a35d97e37f42f25b1 WatchSource:0}: Error finding container 410f42948b4a782a8807b1ba009a2945c4ca93832ccb681a35d97e37f42f25b1: Status 404 returned error can't find the container with id 410f42948b4a782a8807b1ba009a2945c4ca93832ccb681a35d97e37f42f25b1 Jul 02 21:55:28.904941 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:55:28.904902 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rgb8z" event={"ID":"13dad243-1dd3-4e64-a799-15e96c4caadf","Type":"ContainerStarted","Data":"fc372c77f064e5c2f26280acf53bf68045121c8ef0263ee776552498e82731c3"} Jul 02 21:55:28.905151 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:55:28.904956 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rgb8z" event={"ID":"13dad243-1dd3-4e64-a799-15e96c4caadf","Type":"ContainerStarted","Data":"410f42948b4a782a8807b1ba009a2945c4ca93832ccb681a35d97e37f42f25b1"} Jul 02 21:55:28.905151 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:55:28.905036 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rgb8z" Jul 02 21:55:28.927042 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:55:28.926990 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rgb8z" podStartSLOduration=0.926974069 podStartE2EDuration="926.974069ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:55:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:55:28.92471877 +0000 UTC m=+1168.263689206" watchObservedRunningTime="2026-07-02 21:55:28.926974069 +0000 UTC m=+1168.265944505" Jul 02 21:55:39.912771 ip-10-0-139-75 kubenswrapper[2569]: I0702 21:55:39.912730 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rgb8z" Jul 02 22:01:46.548400 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:46.547945 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg"] Jul 02 22:01:46.551367 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:46.551347 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg" Jul 02 22:01:46.554035 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:46.553971 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-a9c5b595-openshift-default-dockercfg-clk6x\"" Jul 02 22:01:46.572156 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:46.570994 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg"] Jul 02 22:01:46.607596 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:46.607558 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7e190b7-563b-4ec0-8304-bce573fdfa86-credential-socket\") pod \"e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg\" (UID: \"f7e190b7-563b-4ec0-8304-bce573fdfa86\") " pod="openshift-ingress/e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg" Jul 02 22:01:46.607761 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:46.607614 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7e190b7-563b-4ec0-8304-bce573fdfa86-istio-data\") pod \"e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg\" (UID: \"f7e190b7-563b-4ec0-8304-bce573fdfa86\") " pod="openshift-ingress/e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg" Jul 02 22:01:46.607761 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:46.607671 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9z8tp\" (UniqueName: \"kubernetes.io/projected/f7e190b7-563b-4ec0-8304-bce573fdfa86-kube-api-access-9z8tp\") pod \"e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg\" (UID: \"f7e190b7-563b-4ec0-8304-bce573fdfa86\") " pod="openshift-ingress/e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg" Jul 02 22:01:46.607761 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:46.607708 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7e190b7-563b-4ec0-8304-bce573fdfa86-istio-envoy\") pod \"e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg\" (UID: \"f7e190b7-563b-4ec0-8304-bce573fdfa86\") " pod="openshift-ingress/e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg" Jul 02 22:01:46.607761 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:46.607725 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7e190b7-563b-4ec0-8304-bce573fdfa86-istiod-ca-cert\") pod \"e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg\" (UID: \"f7e190b7-563b-4ec0-8304-bce573fdfa86\") " pod="openshift-ingress/e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg" Jul 02 22:01:46.607761 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:46.607759 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7e190b7-563b-4ec0-8304-bce573fdfa86-workload-socket\") pod \"e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg\" (UID: \"f7e190b7-563b-4ec0-8304-bce573fdfa86\") " pod="openshift-ingress/e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg" Jul 02 22:01:46.607965 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:46.607812 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7e190b7-563b-4ec0-8304-bce573fdfa86-workload-certs\") pod \"e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg\" (UID: \"f7e190b7-563b-4ec0-8304-bce573fdfa86\") " pod="openshift-ingress/e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg" Jul 02 22:01:46.607965 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:46.607855 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7e190b7-563b-4ec0-8304-bce573fdfa86-istio-podinfo\") pod \"e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg\" (UID: \"f7e190b7-563b-4ec0-8304-bce573fdfa86\") " pod="openshift-ingress/e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg" Jul 02 22:01:46.607965 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:46.607879 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7e190b7-563b-4ec0-8304-bce573fdfa86-istio-token\") pod \"e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg\" (UID: \"f7e190b7-563b-4ec0-8304-bce573fdfa86\") " pod="openshift-ingress/e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg" Jul 02 22:01:46.709289 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:46.709260 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7e190b7-563b-4ec0-8304-bce573fdfa86-istio-podinfo\") pod \"e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg\" (UID: \"f7e190b7-563b-4ec0-8304-bce573fdfa86\") " pod="openshift-ingress/e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg" Jul 02 22:01:46.709289 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:46.709301 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7e190b7-563b-4ec0-8304-bce573fdfa86-istio-token\") pod \"e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg\" (UID: \"f7e190b7-563b-4ec0-8304-bce573fdfa86\") " pod="openshift-ingress/e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg" Jul 02 22:01:46.709553 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:46.709328 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7e190b7-563b-4ec0-8304-bce573fdfa86-credential-socket\") pod \"e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg\" (UID: \"f7e190b7-563b-4ec0-8304-bce573fdfa86\") " pod="openshift-ingress/e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg" Jul 02 22:01:46.709553 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:46.709510 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7e190b7-563b-4ec0-8304-bce573fdfa86-istio-data\") pod \"e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg\" (UID: \"f7e190b7-563b-4ec0-8304-bce573fdfa86\") " pod="openshift-ingress/e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg" Jul 02 22:01:46.709669 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:46.709558 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9z8tp\" (UniqueName: \"kubernetes.io/projected/f7e190b7-563b-4ec0-8304-bce573fdfa86-kube-api-access-9z8tp\") pod \"e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg\" (UID: \"f7e190b7-563b-4ec0-8304-bce573fdfa86\") " pod="openshift-ingress/e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg" Jul 02 22:01:46.709669 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:46.709597 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7e190b7-563b-4ec0-8304-bce573fdfa86-istio-envoy\") pod \"e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg\" (UID: \"f7e190b7-563b-4ec0-8304-bce573fdfa86\") " pod="openshift-ingress/e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg" Jul 02 22:01:46.709669 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:46.709621 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7e190b7-563b-4ec0-8304-bce573fdfa86-istiod-ca-cert\") pod \"e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg\" (UID: \"f7e190b7-563b-4ec0-8304-bce573fdfa86\") " pod="openshift-ingress/e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg" Jul 02 22:01:46.709932 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:46.709676 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7e190b7-563b-4ec0-8304-bce573fdfa86-workload-socket\") pod \"e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg\" (UID: \"f7e190b7-563b-4ec0-8304-bce573fdfa86\") " pod="openshift-ingress/e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg" Jul 02 22:01:46.709932 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:46.709699 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7e190b7-563b-4ec0-8304-bce573fdfa86-credential-socket\") pod \"e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg\" (UID: \"f7e190b7-563b-4ec0-8304-bce573fdfa86\") " pod="openshift-ingress/e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg" Jul 02 22:01:46.709932 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:46.709701 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7e190b7-563b-4ec0-8304-bce573fdfa86-workload-certs\") pod \"e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg\" (UID: \"f7e190b7-563b-4ec0-8304-bce573fdfa86\") " pod="openshift-ingress/e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg" Jul 02 22:01:46.710089 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:46.709931 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7e190b7-563b-4ec0-8304-bce573fdfa86-istio-data\") pod \"e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg\" (UID: \"f7e190b7-563b-4ec0-8304-bce573fdfa86\") " pod="openshift-ingress/e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg" Jul 02 22:01:46.710089 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:46.710042 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7e190b7-563b-4ec0-8304-bce573fdfa86-workload-certs\") pod \"e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg\" (UID: \"f7e190b7-563b-4ec0-8304-bce573fdfa86\") " pod="openshift-ingress/e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg" Jul 02 22:01:46.710194 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:46.710161 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7e190b7-563b-4ec0-8304-bce573fdfa86-workload-socket\") pod \"e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg\" (UID: \"f7e190b7-563b-4ec0-8304-bce573fdfa86\") " pod="openshift-ingress/e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg" Jul 02 22:01:46.710355 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:46.710335 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7e190b7-563b-4ec0-8304-bce573fdfa86-istiod-ca-cert\") pod \"e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg\" (UID: \"f7e190b7-563b-4ec0-8304-bce573fdfa86\") " pod="openshift-ingress/e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg" Jul 02 22:01:46.711738 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:46.711715 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7e190b7-563b-4ec0-8304-bce573fdfa86-istio-envoy\") pod \"e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg\" (UID: \"f7e190b7-563b-4ec0-8304-bce573fdfa86\") " pod="openshift-ingress/e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg" Jul 02 22:01:46.711895 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:46.711877 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7e190b7-563b-4ec0-8304-bce573fdfa86-istio-podinfo\") pod \"e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg\" (UID: \"f7e190b7-563b-4ec0-8304-bce573fdfa86\") " pod="openshift-ingress/e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg" Jul 02 22:01:46.717579 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:46.717559 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9z8tp\" (UniqueName: \"kubernetes.io/projected/f7e190b7-563b-4ec0-8304-bce573fdfa86-kube-api-access-9z8tp\") pod \"e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg\" (UID: \"f7e190b7-563b-4ec0-8304-bce573fdfa86\") " pod="openshift-ingress/e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg" Jul 02 22:01:46.717673 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:46.717623 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7e190b7-563b-4ec0-8304-bce573fdfa86-istio-token\") pod \"e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg\" (UID: \"f7e190b7-563b-4ec0-8304-bce573fdfa86\") " pod="openshift-ingress/e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg" Jul 02 22:01:46.863118 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:46.863035 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg" Jul 02 22:01:47.033963 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:47.033926 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg"] Jul 02 22:01:47.035237 ip-10-0-139-75 kubenswrapper[2569]: W0702 22:01:47.035204 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7e190b7_563b_4ec0_8304_bce573fdfa86.slice/crio-0ff9fe33c7f8aa8fa123bf1dfb7eb560550b43c7a8e7283c5b7cc6d647646b07 WatchSource:0}: Error finding container 0ff9fe33c7f8aa8fa123bf1dfb7eb560550b43c7a8e7283c5b7cc6d647646b07: Status 404 returned error can't find the container with id 0ff9fe33c7f8aa8fa123bf1dfb7eb560550b43c7a8e7283c5b7cc6d647646b07 Jul 02 22:01:47.037509 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:47.037466 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Jul 02 22:01:47.037586 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:47.037569 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Jul 02 22:01:47.037636 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:47.037617 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Jul 02 22:01:47.866011 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:47.865969 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg" event={"ID":"f7e190b7-563b-4ec0-8304-bce573fdfa86","Type":"ContainerStarted","Data":"7fb83fc19b03cae9dac7f5eefb0dc9670091216691c7f3983c10f3cfe21d9a7f"} Jul 02 22:01:47.866475 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:47.866045 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg" event={"ID":"f7e190b7-563b-4ec0-8304-bce573fdfa86","Type":"ContainerStarted","Data":"0ff9fe33c7f8aa8fa123bf1dfb7eb560550b43c7a8e7283c5b7cc6d647646b07"} Jul 02 22:01:47.892740 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:47.892681 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg" podStartSLOduration=1.8926619649999998 podStartE2EDuration="1.892661965s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:01:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:01:47.891725919 +0000 UTC m=+1547.230696354" watchObservedRunningTime="2026-07-02 22:01:47.892661965 +0000 UTC m=+1547.231632401" Jul 02 22:01:48.863612 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:48.863580 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg" Jul 02 22:01:48.868594 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:48.868570 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg" Jul 02 22:01:48.872141 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:48.872120 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg" Jul 02 22:01:48.872708 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:48.872690 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg" Jul 02 22:01:53.039472 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:53.039436 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg"] Jul 02 22:01:53.040071 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:53.039623 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg" podUID="f7e190b7-563b-4ec0-8304-bce573fdfa86" containerName="istio-proxy" containerID="cri-o://7fb83fc19b03cae9dac7f5eefb0dc9670091216691c7f3983c10f3cfe21d9a7f" gracePeriod=30 Jul 02 22:01:59.192303 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.192272 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7"] Jul 02 22:01:59.195688 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.195659 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7" Jul 02 22:01:59.198275 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.198253 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-8bfa68a6-openshift-default-dockercfg-6492w\"" Jul 02 22:01:59.206448 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.206423 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7"] Jul 02 22:01:59.212245 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.212225 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg" Jul 02 22:01:59.346218 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.346136 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7e190b7-563b-4ec0-8304-bce573fdfa86-workload-certs\") pod \"f7e190b7-563b-4ec0-8304-bce573fdfa86\" (UID: \"f7e190b7-563b-4ec0-8304-bce573fdfa86\") " Jul 02 22:01:59.346218 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.346198 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7e190b7-563b-4ec0-8304-bce573fdfa86-istiod-ca-cert\") pod \"f7e190b7-563b-4ec0-8304-bce573fdfa86\" (UID: \"f7e190b7-563b-4ec0-8304-bce573fdfa86\") " Jul 02 22:01:59.346440 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.346223 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7e190b7-563b-4ec0-8304-bce573fdfa86-istio-podinfo\") pod \"f7e190b7-563b-4ec0-8304-bce573fdfa86\" (UID: \"f7e190b7-563b-4ec0-8304-bce573fdfa86\") " Jul 02 22:01:59.346440 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.346240 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7e190b7-563b-4ec0-8304-bce573fdfa86-istio-token\") pod \"f7e190b7-563b-4ec0-8304-bce573fdfa86\" (UID: \"f7e190b7-563b-4ec0-8304-bce573fdfa86\") " Jul 02 22:01:59.346440 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.346270 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7e190b7-563b-4ec0-8304-bce573fdfa86-credential-socket\") pod \"f7e190b7-563b-4ec0-8304-bce573fdfa86\" (UID: \"f7e190b7-563b-4ec0-8304-bce573fdfa86\") " Jul 02 22:01:59.346440 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.346295 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7e190b7-563b-4ec0-8304-bce573fdfa86-istio-data\") pod \"f7e190b7-563b-4ec0-8304-bce573fdfa86\" (UID: \"f7e190b7-563b-4ec0-8304-bce573fdfa86\") " Jul 02 22:01:59.346440 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.346348 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7e190b7-563b-4ec0-8304-bce573fdfa86-workload-socket\") pod \"f7e190b7-563b-4ec0-8304-bce573fdfa86\" (UID: \"f7e190b7-563b-4ec0-8304-bce573fdfa86\") " Jul 02 22:01:59.346440 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.346428 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9z8tp\" (UniqueName: \"kubernetes.io/projected/f7e190b7-563b-4ec0-8304-bce573fdfa86-kube-api-access-9z8tp\") pod \"f7e190b7-563b-4ec0-8304-bce573fdfa86\" (UID: \"f7e190b7-563b-4ec0-8304-bce573fdfa86\") " Jul 02 22:01:59.346740 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.346456 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7e190b7-563b-4ec0-8304-bce573fdfa86-istio-envoy\") pod \"f7e190b7-563b-4ec0-8304-bce573fdfa86\" (UID: \"f7e190b7-563b-4ec0-8304-bce573fdfa86\") " Jul 02 22:01:59.346740 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.346457 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7e190b7-563b-4ec0-8304-bce573fdfa86-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f7e190b7-563b-4ec0-8304-bce573fdfa86" (UID: "f7e190b7-563b-4ec0-8304-bce573fdfa86"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:01:59.346740 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.346593 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7dd97d4b-d4a0-4792-a267-faa23c382741-istio-envoy\") pod \"e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7\" (UID: \"7dd97d4b-d4a0-4792-a267-faa23c382741\") " pod="openshift-ingress/e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7" Jul 02 22:01:59.346740 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.346614 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7e190b7-563b-4ec0-8304-bce573fdfa86-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f7e190b7-563b-4ec0-8304-bce573fdfa86" (UID: "f7e190b7-563b-4ec0-8304-bce573fdfa86"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:01:59.346740 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.346712 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7e190b7-563b-4ec0-8304-bce573fdfa86-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f7e190b7-563b-4ec0-8304-bce573fdfa86" (UID: "f7e190b7-563b-4ec0-8304-bce573fdfa86"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:01:59.347092 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.346737 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7dd97d4b-d4a0-4792-a267-faa23c382741-istio-podinfo\") pod \"e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7\" (UID: \"7dd97d4b-d4a0-4792-a267-faa23c382741\") " pod="openshift-ingress/e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7" Jul 02 22:01:59.347092 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.346757 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7e190b7-563b-4ec0-8304-bce573fdfa86-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f7e190b7-563b-4ec0-8304-bce573fdfa86" (UID: "f7e190b7-563b-4ec0-8304-bce573fdfa86"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 22:01:59.347092 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.346770 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7dd97d4b-d4a0-4792-a267-faa23c382741-workload-certs\") pod \"e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7\" (UID: \"7dd97d4b-d4a0-4792-a267-faa23c382741\") " pod="openshift-ingress/e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7" Jul 02 22:01:59.347092 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.346865 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7dd97d4b-d4a0-4792-a267-faa23c382741-istio-token\") pod \"e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7\" (UID: \"7dd97d4b-d4a0-4792-a267-faa23c382741\") " pod="openshift-ingress/e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7" Jul 02 22:01:59.347092 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.346894 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7dd97d4b-d4a0-4792-a267-faa23c382741-istiod-ca-cert\") pod \"e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7\" (UID: \"7dd97d4b-d4a0-4792-a267-faa23c382741\") " pod="openshift-ingress/e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7" Jul 02 22:01:59.347092 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.346921 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dd97d4b-d4a0-4792-a267-faa23c382741-credential-socket\") pod \"e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7\" (UID: \"7dd97d4b-d4a0-4792-a267-faa23c382741\") " pod="openshift-ingress/e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7" Jul 02 22:01:59.347092 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.346965 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dd97d4b-d4a0-4792-a267-faa23c382741-workload-socket\") pod \"e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7\" (UID: \"7dd97d4b-d4a0-4792-a267-faa23c382741\") " pod="openshift-ingress/e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7" Jul 02 22:01:59.347092 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.346997 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpw2w\" (UniqueName: \"kubernetes.io/projected/7dd97d4b-d4a0-4792-a267-faa23c382741-kube-api-access-lpw2w\") pod \"e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7\" (UID: \"7dd97d4b-d4a0-4792-a267-faa23c382741\") " pod="openshift-ingress/e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7" Jul 02 22:01:59.347449 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.347133 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7dd97d4b-d4a0-4792-a267-faa23c382741-istio-data\") pod \"e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7\" (UID: \"7dd97d4b-d4a0-4792-a267-faa23c382741\") " pod="openshift-ingress/e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7" Jul 02 22:01:59.347449 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.347229 2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7e190b7-563b-4ec0-8304-bce573fdfa86-istiod-ca-cert\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:01:59.347449 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.347246 2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7e190b7-563b-4ec0-8304-bce573fdfa86-credential-socket\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:01:59.347449 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.347262 2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7e190b7-563b-4ec0-8304-bce573fdfa86-workload-socket\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:01:59.347449 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.347278 2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7e190b7-563b-4ec0-8304-bce573fdfa86-workload-certs\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:01:59.347449 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.347312 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7e190b7-563b-4ec0-8304-bce573fdfa86-istio-data" (OuterVolumeSpecName: "istio-data") pod "f7e190b7-563b-4ec0-8304-bce573fdfa86" (UID: "f7e190b7-563b-4ec0-8304-bce573fdfa86"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:01:59.348649 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.348630 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7e190b7-563b-4ec0-8304-bce573fdfa86-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f7e190b7-563b-4ec0-8304-bce573fdfa86" (UID: "f7e190b7-563b-4ec0-8304-bce573fdfa86"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:01:59.348746 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.348695 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7e190b7-563b-4ec0-8304-bce573fdfa86-istio-token" (OuterVolumeSpecName: "istio-token") pod "f7e190b7-563b-4ec0-8304-bce573fdfa86" (UID: "f7e190b7-563b-4ec0-8304-bce573fdfa86"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:01:59.348882 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.348867 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f7e190b7-563b-4ec0-8304-bce573fdfa86-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f7e190b7-563b-4ec0-8304-bce573fdfa86" (UID: "f7e190b7-563b-4ec0-8304-bce573fdfa86"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 22:01:59.348946 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.348913 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7e190b7-563b-4ec0-8304-bce573fdfa86-kube-api-access-9z8tp" (OuterVolumeSpecName: "kube-api-access-9z8tp") pod "f7e190b7-563b-4ec0-8304-bce573fdfa86" (UID: "f7e190b7-563b-4ec0-8304-bce573fdfa86"). InnerVolumeSpecName "kube-api-access-9z8tp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:01:59.448603 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.448570 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7dd97d4b-d4a0-4792-a267-faa23c382741-istio-podinfo\") pod \"e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7\" (UID: \"7dd97d4b-d4a0-4792-a267-faa23c382741\") " pod="openshift-ingress/e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7" Jul 02 22:01:59.448603 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.448606 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7dd97d4b-d4a0-4792-a267-faa23c382741-workload-certs\") pod \"e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7\" (UID: \"7dd97d4b-d4a0-4792-a267-faa23c382741\") " pod="openshift-ingress/e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7" Jul 02 22:01:59.448858 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.448630 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7dd97d4b-d4a0-4792-a267-faa23c382741-istio-token\") pod \"e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7\" (UID: \"7dd97d4b-d4a0-4792-a267-faa23c382741\") " pod="openshift-ingress/e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7" Jul 02 22:01:59.448858 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.448676 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7dd97d4b-d4a0-4792-a267-faa23c382741-istiod-ca-cert\") pod \"e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7\" (UID: \"7dd97d4b-d4a0-4792-a267-faa23c382741\") " pod="openshift-ingress/e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7" Jul 02 22:01:59.448858 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.448713 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dd97d4b-d4a0-4792-a267-faa23c382741-credential-socket\") pod \"e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7\" (UID: \"7dd97d4b-d4a0-4792-a267-faa23c382741\") " pod="openshift-ingress/e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7" Jul 02 22:01:59.448858 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.448732 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dd97d4b-d4a0-4792-a267-faa23c382741-workload-socket\") pod \"e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7\" (UID: \"7dd97d4b-d4a0-4792-a267-faa23c382741\") " pod="openshift-ingress/e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7" Jul 02 22:01:59.448858 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.448750 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpw2w\" (UniqueName: \"kubernetes.io/projected/7dd97d4b-d4a0-4792-a267-faa23c382741-kube-api-access-lpw2w\") pod \"e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7\" (UID: \"7dd97d4b-d4a0-4792-a267-faa23c382741\") " pod="openshift-ingress/e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7" Jul 02 22:01:59.449104 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.448879 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7dd97d4b-d4a0-4792-a267-faa23c382741-istio-data\") pod \"e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7\" (UID: \"7dd97d4b-d4a0-4792-a267-faa23c382741\") " pod="openshift-ingress/e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7" Jul 02 22:01:59.449104 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.448950 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7dd97d4b-d4a0-4792-a267-faa23c382741-istio-envoy\") pod \"e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7\" (UID: \"7dd97d4b-d4a0-4792-a267-faa23c382741\") " pod="openshift-ingress/e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7" Jul 02 22:01:59.449104 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.449035 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9z8tp\" (UniqueName: \"kubernetes.io/projected/f7e190b7-563b-4ec0-8304-bce573fdfa86-kube-api-access-9z8tp\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:01:59.449104 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.449054 2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7e190b7-563b-4ec0-8304-bce573fdfa86-istio-envoy\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:01:59.449104 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.449071 2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7e190b7-563b-4ec0-8304-bce573fdfa86-istio-podinfo\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:01:59.449104 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.449087 2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7e190b7-563b-4ec0-8304-bce573fdfa86-istio-token\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:01:59.449104 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.449099 2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7e190b7-563b-4ec0-8304-bce573fdfa86-istio-data\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:01:59.449425 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.449240 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dd97d4b-d4a0-4792-a267-faa23c382741-workload-socket\") pod \"e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7\" (UID: \"7dd97d4b-d4a0-4792-a267-faa23c382741\") " pod="openshift-ingress/e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7" Jul 02 22:01:59.449425 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.449263 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7dd97d4b-d4a0-4792-a267-faa23c382741-workload-certs\") pod \"e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7\" (UID: \"7dd97d4b-d4a0-4792-a267-faa23c382741\") " pod="openshift-ingress/e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7" Jul 02 22:01:59.449425 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.449315 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dd97d4b-d4a0-4792-a267-faa23c382741-credential-socket\") pod \"e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7\" (UID: \"7dd97d4b-d4a0-4792-a267-faa23c382741\") " pod="openshift-ingress/e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7" Jul 02 22:01:59.449425 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.449348 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7dd97d4b-d4a0-4792-a267-faa23c382741-istio-data\") pod \"e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7\" (UID: \"7dd97d4b-d4a0-4792-a267-faa23c382741\") " pod="openshift-ingress/e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7" Jul 02 22:01:59.449550 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.449520 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7dd97d4b-d4a0-4792-a267-faa23c382741-istiod-ca-cert\") pod \"e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7\" (UID: \"7dd97d4b-d4a0-4792-a267-faa23c382741\") " pod="openshift-ingress/e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7" Jul 02 22:01:59.451046 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.451029 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7dd97d4b-d4a0-4792-a267-faa23c382741-istio-envoy\") pod \"e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7\" (UID: \"7dd97d4b-d4a0-4792-a267-faa23c382741\") " pod="openshift-ingress/e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7" Jul 02 22:01:59.451111 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.451073 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7dd97d4b-d4a0-4792-a267-faa23c382741-istio-podinfo\") pod \"e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7\" (UID: \"7dd97d4b-d4a0-4792-a267-faa23c382741\") " pod="openshift-ingress/e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7" Jul 02 22:01:59.456889 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.456867 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7dd97d4b-d4a0-4792-a267-faa23c382741-istio-token\") pod \"e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7\" (UID: \"7dd97d4b-d4a0-4792-a267-faa23c382741\") " pod="openshift-ingress/e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7" Jul 02 22:01:59.457252 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.457236 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpw2w\" (UniqueName: \"kubernetes.io/projected/7dd97d4b-d4a0-4792-a267-faa23c382741-kube-api-access-lpw2w\") pod \"e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7\" (UID: \"7dd97d4b-d4a0-4792-a267-faa23c382741\") " pod="openshift-ingress/e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7" Jul 02 22:01:59.519233 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.519202 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7" Jul 02 22:01:59.662102 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.662075 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7"] Jul 02 22:01:59.664731 ip-10-0-139-75 kubenswrapper[2569]: W0702 22:01:59.664704 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7dd97d4b_d4a0_4792_a267_faa23c382741.slice/crio-9c36afd95ee8dbc001949f152a8ad82128e8fb9091ef6ff5ca1f9babc601e52e WatchSource:0}: Error finding container 9c36afd95ee8dbc001949f152a8ad82128e8fb9091ef6ff5ca1f9babc601e52e: Status 404 returned error can't find the container with id 9c36afd95ee8dbc001949f152a8ad82128e8fb9091ef6ff5ca1f9babc601e52e Jul 02 22:01:59.666763 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.666733 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Jul 02 22:01:59.666857 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.666832 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Jul 02 22:01:59.666901 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.666864 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Jul 02 22:01:59.937538 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.937450 2569 generic.go:356] "Generic (PLEG): container finished" podID="f7e190b7-563b-4ec0-8304-bce573fdfa86" containerID="7fb83fc19b03cae9dac7f5eefb0dc9670091216691c7f3983c10f3cfe21d9a7f" exitCode=0 Jul 02 22:01:59.937538 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.937495 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg" Jul 02 22:01:59.937769 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.937532 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg" event={"ID":"f7e190b7-563b-4ec0-8304-bce573fdfa86","Type":"ContainerDied","Data":"7fb83fc19b03cae9dac7f5eefb0dc9670091216691c7f3983c10f3cfe21d9a7f"} Jul 02 22:01:59.937769 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.937596 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg" event={"ID":"f7e190b7-563b-4ec0-8304-bce573fdfa86","Type":"ContainerDied","Data":"0ff9fe33c7f8aa8fa123bf1dfb7eb560550b43c7a8e7283c5b7cc6d647646b07"} Jul 02 22:01:59.937769 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.937627 2569 scope.go:117] "RemoveContainer" containerID="7fb83fc19b03cae9dac7f5eefb0dc9670091216691c7f3983c10f3cfe21d9a7f" Jul 02 22:01:59.939209 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.939175 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7" event={"ID":"7dd97d4b-d4a0-4792-a267-faa23c382741","Type":"ContainerStarted","Data":"71b4d6203ab06da38ffe849af38a9cd92c3ad48f33dff8a69989f10c5260c862"} Jul 02 22:01:59.939318 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.939226 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7" event={"ID":"7dd97d4b-d4a0-4792-a267-faa23c382741","Type":"ContainerStarted","Data":"9c36afd95ee8dbc001949f152a8ad82128e8fb9091ef6ff5ca1f9babc601e52e"} Jul 02 22:01:59.947169 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.947152 2569 scope.go:117] "RemoveContainer" containerID="7fb83fc19b03cae9dac7f5eefb0dc9670091216691c7f3983c10f3cfe21d9a7f" Jul 02 22:01:59.947418 ip-10-0-139-75 kubenswrapper[2569]: E0702 22:01:59.947400 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fb83fc19b03cae9dac7f5eefb0dc9670091216691c7f3983c10f3cfe21d9a7f\": container with ID starting with 7fb83fc19b03cae9dac7f5eefb0dc9670091216691c7f3983c10f3cfe21d9a7f not found: ID does not exist" containerID="7fb83fc19b03cae9dac7f5eefb0dc9670091216691c7f3983c10f3cfe21d9a7f" Jul 02 22:01:59.947465 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.947426 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fb83fc19b03cae9dac7f5eefb0dc9670091216691c7f3983c10f3cfe21d9a7f"} err="failed to get container status \"7fb83fc19b03cae9dac7f5eefb0dc9670091216691c7f3983c10f3cfe21d9a7f\": rpc error: code = NotFound desc = could not find container \"7fb83fc19b03cae9dac7f5eefb0dc9670091216691c7f3983c10f3cfe21d9a7f\": container with ID starting with 7fb83fc19b03cae9dac7f5eefb0dc9670091216691c7f3983c10f3cfe21d9a7f not found: ID does not exist" Jul 02 22:01:59.963611 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.963565 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7" podStartSLOduration=0.963546839 podStartE2EDuration="963.546839ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:01:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:01:59.961988564 +0000 UTC m=+1559.300959002" watchObservedRunningTime="2026-07-02 22:01:59.963546839 +0000 UTC m=+1559.302517275" Jul 02 22:01:59.979035 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.979011 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg"] Jul 02 22:01:59.982983 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:01:59.982963 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-a9c5b595-openshift-default-646488bbd5-srbmg"] Jul 02 22:02:00.519464 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:00.519434 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7" Jul 02 22:02:00.523943 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:00.523920 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7" Jul 02 22:02:00.959694 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:00.959611 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7e190b7-563b-4ec0-8304-bce573fdfa86" path="/var/lib/kubelet/pods/f7e190b7-563b-4ec0-8304-bce573fdfa86/volumes" Jul 02 22:02:00.959958 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:00.959943 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7" Jul 02 22:02:00.960030 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:00.960010 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7" Jul 02 22:02:11.842881 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:11.842843 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7"] Jul 02 22:02:11.843260 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:11.843085 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7" podUID="7dd97d4b-d4a0-4792-a267-faa23c382741" containerName="istio-proxy" containerID="cri-o://71b4d6203ab06da38ffe849af38a9cd92c3ad48f33dff8a69989f10c5260c862" gracePeriod=30 Jul 02 22:02:12.496356 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:12.496334 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7" Jul 02 22:02:12.589208 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:12.589179 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7dd97d4b-d4a0-4792-a267-faa23c382741-istiod-ca-cert\") pod \"7dd97d4b-d4a0-4792-a267-faa23c382741\" (UID: \"7dd97d4b-d4a0-4792-a267-faa23c382741\") " Jul 02 22:02:12.589208 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:12.589212 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lpw2w\" (UniqueName: \"kubernetes.io/projected/7dd97d4b-d4a0-4792-a267-faa23c382741-kube-api-access-lpw2w\") pod \"7dd97d4b-d4a0-4792-a267-faa23c382741\" (UID: \"7dd97d4b-d4a0-4792-a267-faa23c382741\") " Jul 02 22:02:12.589416 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:12.589239 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7dd97d4b-d4a0-4792-a267-faa23c382741-workload-certs\") pod \"7dd97d4b-d4a0-4792-a267-faa23c382741\" (UID: \"7dd97d4b-d4a0-4792-a267-faa23c382741\") " Jul 02 22:02:12.589416 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:12.589373 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dd97d4b-d4a0-4792-a267-faa23c382741-credential-socket\") pod \"7dd97d4b-d4a0-4792-a267-faa23c382741\" (UID: \"7dd97d4b-d4a0-4792-a267-faa23c382741\") " Jul 02 22:02:12.589505 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:12.589419 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dd97d4b-d4a0-4792-a267-faa23c382741-workload-socket\") pod \"7dd97d4b-d4a0-4792-a267-faa23c382741\" (UID: \"7dd97d4b-d4a0-4792-a267-faa23c382741\") " Jul 02 22:02:12.589505 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:12.589468 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7dd97d4b-d4a0-4792-a267-faa23c382741-istio-data\") pod \"7dd97d4b-d4a0-4792-a267-faa23c382741\" (UID: \"7dd97d4b-d4a0-4792-a267-faa23c382741\") " Jul 02 22:02:12.589614 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:12.589500 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7dd97d4b-d4a0-4792-a267-faa23c382741-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7dd97d4b-d4a0-4792-a267-faa23c382741" (UID: "7dd97d4b-d4a0-4792-a267-faa23c382741"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:02:12.589614 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:12.589540 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7dd97d4b-d4a0-4792-a267-faa23c382741-istio-token\") pod \"7dd97d4b-d4a0-4792-a267-faa23c382741\" (UID: \"7dd97d4b-d4a0-4792-a267-faa23c382741\") " Jul 02 22:02:12.589720 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:12.589631 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7dd97d4b-d4a0-4792-a267-faa23c382741-istio-podinfo\") pod \"7dd97d4b-d4a0-4792-a267-faa23c382741\" (UID: \"7dd97d4b-d4a0-4792-a267-faa23c382741\") " Jul 02 22:02:12.589769 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:12.589536 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7dd97d4b-d4a0-4792-a267-faa23c382741-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7dd97d4b-d4a0-4792-a267-faa23c382741" (UID: "7dd97d4b-d4a0-4792-a267-faa23c382741"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 22:02:12.589769 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:12.589606 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7dd97d4b-d4a0-4792-a267-faa23c382741-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7dd97d4b-d4a0-4792-a267-faa23c382741" (UID: "7dd97d4b-d4a0-4792-a267-faa23c382741"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:02:12.589769 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:12.589695 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7dd97d4b-d4a0-4792-a267-faa23c382741-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7dd97d4b-d4a0-4792-a267-faa23c382741" (UID: "7dd97d4b-d4a0-4792-a267-faa23c382741"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:02:12.589769 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:12.589667 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7dd97d4b-d4a0-4792-a267-faa23c382741-istio-envoy\") pod \"7dd97d4b-d4a0-4792-a267-faa23c382741\" (UID: \"7dd97d4b-d4a0-4792-a267-faa23c382741\") " Jul 02 22:02:12.590162 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:12.590136 2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7dd97d4b-d4a0-4792-a267-faa23c382741-istiod-ca-cert\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:02:12.590257 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:12.590165 2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7dd97d4b-d4a0-4792-a267-faa23c382741-workload-certs\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:02:12.590257 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:12.590178 2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dd97d4b-d4a0-4792-a267-faa23c382741-credential-socket\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:02:12.590257 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:12.590193 2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dd97d4b-d4a0-4792-a267-faa23c382741-workload-socket\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:02:12.590773 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:12.590748 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7dd97d4b-d4a0-4792-a267-faa23c382741-istio-data" (OuterVolumeSpecName: "istio-data") pod "7dd97d4b-d4a0-4792-a267-faa23c382741" (UID: "7dd97d4b-d4a0-4792-a267-faa23c382741"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:02:12.591539 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:12.591514 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7dd97d4b-d4a0-4792-a267-faa23c382741-istio-token" (OuterVolumeSpecName: "istio-token") pod "7dd97d4b-d4a0-4792-a267-faa23c382741" (UID: "7dd97d4b-d4a0-4792-a267-faa23c382741"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:02:12.591628 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:12.591541 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7dd97d4b-d4a0-4792-a267-faa23c382741-kube-api-access-lpw2w" (OuterVolumeSpecName: "kube-api-access-lpw2w") pod "7dd97d4b-d4a0-4792-a267-faa23c382741" (UID: "7dd97d4b-d4a0-4792-a267-faa23c382741"). InnerVolumeSpecName "kube-api-access-lpw2w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:02:12.591677 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:12.591662 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7dd97d4b-d4a0-4792-a267-faa23c382741-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7dd97d4b-d4a0-4792-a267-faa23c382741" (UID: "7dd97d4b-d4a0-4792-a267-faa23c382741"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 22:02:12.591718 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:12.591702 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7dd97d4b-d4a0-4792-a267-faa23c382741-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7dd97d4b-d4a0-4792-a267-faa23c382741" (UID: "7dd97d4b-d4a0-4792-a267-faa23c382741"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:02:12.691145 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:12.691120 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lpw2w\" (UniqueName: \"kubernetes.io/projected/7dd97d4b-d4a0-4792-a267-faa23c382741-kube-api-access-lpw2w\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:02:12.691145 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:12.691143 2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7dd97d4b-d4a0-4792-a267-faa23c382741-istio-data\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:02:12.691291 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:12.691152 2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7dd97d4b-d4a0-4792-a267-faa23c382741-istio-token\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:02:12.691291 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:12.691161 2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7dd97d4b-d4a0-4792-a267-faa23c382741-istio-podinfo\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:02:12.691291 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:12.691169 2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7dd97d4b-d4a0-4792-a267-faa23c382741-istio-envoy\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:02:13.034300 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:13.034267 2569 generic.go:356] "Generic (PLEG): container finished" podID="7dd97d4b-d4a0-4792-a267-faa23c382741" containerID="71b4d6203ab06da38ffe849af38a9cd92c3ad48f33dff8a69989f10c5260c862" exitCode=0 Jul 02 22:02:13.034694 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:13.034307 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7" Jul 02 22:02:13.034694 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:13.034348 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7" event={"ID":"7dd97d4b-d4a0-4792-a267-faa23c382741","Type":"ContainerDied","Data":"71b4d6203ab06da38ffe849af38a9cd92c3ad48f33dff8a69989f10c5260c862"} Jul 02 22:02:13.034694 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:13.034397 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7" event={"ID":"7dd97d4b-d4a0-4792-a267-faa23c382741","Type":"ContainerDied","Data":"9c36afd95ee8dbc001949f152a8ad82128e8fb9091ef6ff5ca1f9babc601e52e"} Jul 02 22:02:13.034694 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:13.034420 2569 scope.go:117] "RemoveContainer" containerID="71b4d6203ab06da38ffe849af38a9cd92c3ad48f33dff8a69989f10c5260c862" Jul 02 22:02:13.043237 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:13.043220 2569 scope.go:117] "RemoveContainer" containerID="71b4d6203ab06da38ffe849af38a9cd92c3ad48f33dff8a69989f10c5260c862" Jul 02 22:02:13.043493 ip-10-0-139-75 kubenswrapper[2569]: E0702 22:02:13.043474 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71b4d6203ab06da38ffe849af38a9cd92c3ad48f33dff8a69989f10c5260c862\": container with ID starting with 71b4d6203ab06da38ffe849af38a9cd92c3ad48f33dff8a69989f10c5260c862 not found: ID does not exist" containerID="71b4d6203ab06da38ffe849af38a9cd92c3ad48f33dff8a69989f10c5260c862" Jul 02 22:02:13.043538 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:13.043502 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71b4d6203ab06da38ffe849af38a9cd92c3ad48f33dff8a69989f10c5260c862"} err="failed to get container status \"71b4d6203ab06da38ffe849af38a9cd92c3ad48f33dff8a69989f10c5260c862\": rpc error: code = NotFound desc = could not find container \"71b4d6203ab06da38ffe849af38a9cd92c3ad48f33dff8a69989f10c5260c862\": container with ID starting with 71b4d6203ab06da38ffe849af38a9cd92c3ad48f33dff8a69989f10c5260c862 not found: ID does not exist" Jul 02 22:02:13.052632 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:13.052610 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7"] Jul 02 22:02:13.055995 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:13.055972 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-8bfa68a6-openshift-default-5c578ddb67-dfrs7"] Jul 02 22:02:14.956149 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:14.956109 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7dd97d4b-d4a0-4792-a267-faa23c382741" path="/var/lib/kubelet/pods/7dd97d4b-d4a0-4792-a267-faa23c382741/volumes" Jul 02 22:02:17.902708 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:17.902672 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5"] Jul 02 22:02:17.903700 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:17.903669 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7dd97d4b-d4a0-4792-a267-faa23c382741" containerName="istio-proxy" Jul 02 22:02:17.903700 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:17.903697 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="7dd97d4b-d4a0-4792-a267-faa23c382741" containerName="istio-proxy" Jul 02 22:02:17.903949 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:17.903711 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7e190b7-563b-4ec0-8304-bce573fdfa86" containerName="istio-proxy" Jul 02 22:02:17.903949 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:17.903718 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7e190b7-563b-4ec0-8304-bce573fdfa86" containerName="istio-proxy" Jul 02 22:02:17.903949 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:17.903816 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="7dd97d4b-d4a0-4792-a267-faa23c382741" containerName="istio-proxy" Jul 02 22:02:17.903949 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:17.903827 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7e190b7-563b-4ec0-8304-bce573fdfa86" containerName="istio-proxy" Jul 02 22:02:17.907270 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:17.907250 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5" Jul 02 22:02:17.909990 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:17.909968 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-e6c9d74c-openshift-default-dockercfg-krckd\"" Jul 02 22:02:17.918403 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:17.918369 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5"] Jul 02 22:02:17.946655 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:17.946615 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b03a2ca-af48-490e-89ae-76d295ec9fe1-workload-certs\") pod \"e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5\" (UID: \"9b03a2ca-af48-490e-89ae-76d295ec9fe1\") " pod="openshift-ingress/e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5" Jul 02 22:02:17.946844 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:17.946706 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b03a2ca-af48-490e-89ae-76d295ec9fe1-istiod-ca-cert\") pod \"e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5\" (UID: \"9b03a2ca-af48-490e-89ae-76d295ec9fe1\") " pod="openshift-ingress/e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5" Jul 02 22:02:17.946844 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:17.946732 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlbfk\" (UniqueName: \"kubernetes.io/projected/9b03a2ca-af48-490e-89ae-76d295ec9fe1-kube-api-access-rlbfk\") pod \"e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5\" (UID: \"9b03a2ca-af48-490e-89ae-76d295ec9fe1\") " pod="openshift-ingress/e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5" Jul 02 22:02:17.946844 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:17.946750 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b03a2ca-af48-490e-89ae-76d295ec9fe1-workload-socket\") pod \"e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5\" (UID: \"9b03a2ca-af48-490e-89ae-76d295ec9fe1\") " pod="openshift-ingress/e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5" Jul 02 22:02:17.946844 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:17.946769 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b03a2ca-af48-490e-89ae-76d295ec9fe1-istio-envoy\") pod \"e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5\" (UID: \"9b03a2ca-af48-490e-89ae-76d295ec9fe1\") " pod="openshift-ingress/e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5" Jul 02 22:02:17.946844 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:17.946806 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b03a2ca-af48-490e-89ae-76d295ec9fe1-istio-data\") pod \"e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5\" (UID: \"9b03a2ca-af48-490e-89ae-76d295ec9fe1\") " pod="openshift-ingress/e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5" Jul 02 22:02:17.947016 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:17.946851 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b03a2ca-af48-490e-89ae-76d295ec9fe1-istio-token\") pod \"e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5\" (UID: \"9b03a2ca-af48-490e-89ae-76d295ec9fe1\") " pod="openshift-ingress/e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5" Jul 02 22:02:17.947016 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:17.946872 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b03a2ca-af48-490e-89ae-76d295ec9fe1-istio-podinfo\") pod \"e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5\" (UID: \"9b03a2ca-af48-490e-89ae-76d295ec9fe1\") " pod="openshift-ingress/e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5" Jul 02 22:02:17.947016 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:17.946926 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b03a2ca-af48-490e-89ae-76d295ec9fe1-credential-socket\") pod \"e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5\" (UID: \"9b03a2ca-af48-490e-89ae-76d295ec9fe1\") " pod="openshift-ingress/e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5" Jul 02 22:02:18.047828 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:18.047776 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b03a2ca-af48-490e-89ae-76d295ec9fe1-istio-token\") pod \"e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5\" (UID: \"9b03a2ca-af48-490e-89ae-76d295ec9fe1\") " pod="openshift-ingress/e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5" Jul 02 22:02:18.047828 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:18.047832 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b03a2ca-af48-490e-89ae-76d295ec9fe1-istio-podinfo\") pod \"e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5\" (UID: \"9b03a2ca-af48-490e-89ae-76d295ec9fe1\") " pod="openshift-ingress/e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5" Jul 02 22:02:18.048048 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:18.047885 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b03a2ca-af48-490e-89ae-76d295ec9fe1-credential-socket\") pod \"e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5\" (UID: \"9b03a2ca-af48-490e-89ae-76d295ec9fe1\") " pod="openshift-ingress/e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5" Jul 02 22:02:18.048048 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:18.047910 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b03a2ca-af48-490e-89ae-76d295ec9fe1-workload-certs\") pod \"e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5\" (UID: \"9b03a2ca-af48-490e-89ae-76d295ec9fe1\") " pod="openshift-ingress/e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5" Jul 02 22:02:18.048048 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:18.047961 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b03a2ca-af48-490e-89ae-76d295ec9fe1-istiod-ca-cert\") pod \"e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5\" (UID: \"9b03a2ca-af48-490e-89ae-76d295ec9fe1\") " pod="openshift-ingress/e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5" Jul 02 22:02:18.048048 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:18.047982 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlbfk\" (UniqueName: \"kubernetes.io/projected/9b03a2ca-af48-490e-89ae-76d295ec9fe1-kube-api-access-rlbfk\") pod \"e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5\" (UID: \"9b03a2ca-af48-490e-89ae-76d295ec9fe1\") " pod="openshift-ingress/e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5" Jul 02 22:02:18.048048 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:18.048002 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b03a2ca-af48-490e-89ae-76d295ec9fe1-workload-socket\") pod \"e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5\" (UID: \"9b03a2ca-af48-490e-89ae-76d295ec9fe1\") " pod="openshift-ingress/e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5" Jul 02 22:02:18.048048 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:18.048020 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b03a2ca-af48-490e-89ae-76d295ec9fe1-istio-envoy\") pod \"e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5\" (UID: \"9b03a2ca-af48-490e-89ae-76d295ec9fe1\") " pod="openshift-ingress/e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5" Jul 02 22:02:18.048048 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:18.048036 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b03a2ca-af48-490e-89ae-76d295ec9fe1-istio-data\") pod \"e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5\" (UID: \"9b03a2ca-af48-490e-89ae-76d295ec9fe1\") " pod="openshift-ingress/e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5" Jul 02 22:02:18.048394 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:18.048360 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b03a2ca-af48-490e-89ae-76d295ec9fe1-credential-socket\") pod \"e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5\" (UID: \"9b03a2ca-af48-490e-89ae-76d295ec9fe1\") " pod="openshift-ingress/e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5" Jul 02 22:02:18.048506 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:18.048454 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b03a2ca-af48-490e-89ae-76d295ec9fe1-workload-certs\") pod \"e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5\" (UID: \"9b03a2ca-af48-490e-89ae-76d295ec9fe1\") " pod="openshift-ingress/e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5" Jul 02 22:02:18.048506 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:18.048486 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b03a2ca-af48-490e-89ae-76d295ec9fe1-istio-data\") pod \"e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5\" (UID: \"9b03a2ca-af48-490e-89ae-76d295ec9fe1\") " pod="openshift-ingress/e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5" Jul 02 22:02:18.048687 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:18.048658 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b03a2ca-af48-490e-89ae-76d295ec9fe1-workload-socket\") pod \"e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5\" (UID: \"9b03a2ca-af48-490e-89ae-76d295ec9fe1\") " pod="openshift-ingress/e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5" Jul 02 22:02:18.048824 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:18.048752 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b03a2ca-af48-490e-89ae-76d295ec9fe1-istiod-ca-cert\") pod \"e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5\" (UID: \"9b03a2ca-af48-490e-89ae-76d295ec9fe1\") " pod="openshift-ingress/e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5" Jul 02 22:02:18.050272 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:18.050254 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b03a2ca-af48-490e-89ae-76d295ec9fe1-istio-envoy\") pod \"e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5\" (UID: \"9b03a2ca-af48-490e-89ae-76d295ec9fe1\") " pod="openshift-ingress/e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5" Jul 02 22:02:18.050603 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:18.050579 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b03a2ca-af48-490e-89ae-76d295ec9fe1-istio-podinfo\") pod \"e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5\" (UID: \"9b03a2ca-af48-490e-89ae-76d295ec9fe1\") " pod="openshift-ingress/e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5" Jul 02 22:02:18.057088 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:18.057065 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b03a2ca-af48-490e-89ae-76d295ec9fe1-istio-token\") pod \"e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5\" (UID: \"9b03a2ca-af48-490e-89ae-76d295ec9fe1\") " pod="openshift-ingress/e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5" Jul 02 22:02:18.057197 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:18.057087 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlbfk\" (UniqueName: \"kubernetes.io/projected/9b03a2ca-af48-490e-89ae-76d295ec9fe1-kube-api-access-rlbfk\") pod \"e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5\" (UID: \"9b03a2ca-af48-490e-89ae-76d295ec9fe1\") " pod="openshift-ingress/e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5" Jul 02 22:02:18.220849 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:18.220743 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5" Jul 02 22:02:18.364420 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:18.364395 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5"] Jul 02 22:02:18.366396 ip-10-0-139-75 kubenswrapper[2569]: W0702 22:02:18.366369 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b03a2ca_af48_490e_89ae_76d295ec9fe1.slice/crio-4f55a4edd5234a4757c52018d12a407408269a573acd06e3191a32c99206a633 WatchSource:0}: Error finding container 4f55a4edd5234a4757c52018d12a407408269a573acd06e3191a32c99206a633: Status 404 returned error can't find the container with id 4f55a4edd5234a4757c52018d12a407408269a573acd06e3191a32c99206a633 Jul 02 22:02:18.368488 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:18.368454 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Jul 02 22:02:18.368569 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:18.368524 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Jul 02 22:02:18.368612 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:18.368569 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Jul 02 22:02:19.069548 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:19.069519 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5" event={"ID":"9b03a2ca-af48-490e-89ae-76d295ec9fe1","Type":"ContainerStarted","Data":"aac27110beeab85de75ce23829491d11429e0cf37f67bd5c2186327d51e0049a"} Jul 02 22:02:19.069947 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:19.069569 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5" event={"ID":"9b03a2ca-af48-490e-89ae-76d295ec9fe1","Type":"ContainerStarted","Data":"4f55a4edd5234a4757c52018d12a407408269a573acd06e3191a32c99206a633"} Jul 02 22:02:19.100939 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:19.100880 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5" podStartSLOduration=2.100865613 podStartE2EDuration="2.100865613s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:02:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:02:19.099953898 +0000 UTC m=+1578.438924336" watchObservedRunningTime="2026-07-02 22:02:19.100865613 +0000 UTC m=+1578.439836048" Jul 02 22:02:19.221680 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:19.221645 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5" Jul 02 22:02:19.227248 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:19.227223 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5" Jul 02 22:02:20.075673 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:20.075641 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5" Jul 02 22:02:20.076919 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:20.076897 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5" Jul 02 22:02:23.715900 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:23.715861 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5"] Jul 02 22:02:23.716313 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:23.716057 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5" podUID="9b03a2ca-af48-490e-89ae-76d295ec9fe1" containerName="istio-proxy" containerID="cri-o://aac27110beeab85de75ce23829491d11429e0cf37f67bd5c2186327d51e0049a" gracePeriod=30 Jul 02 22:02:28.993559 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:28.993536 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5" Jul 02 22:02:29.067593 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:29.067566 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b03a2ca-af48-490e-89ae-76d295ec9fe1-credential-socket\") pod \"9b03a2ca-af48-490e-89ae-76d295ec9fe1\" (UID: \"9b03a2ca-af48-490e-89ae-76d295ec9fe1\") " Jul 02 22:02:29.067754 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:29.067611 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b03a2ca-af48-490e-89ae-76d295ec9fe1-istio-data\") pod \"9b03a2ca-af48-490e-89ae-76d295ec9fe1\" (UID: \"9b03a2ca-af48-490e-89ae-76d295ec9fe1\") " Jul 02 22:02:29.067754 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:29.067633 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b03a2ca-af48-490e-89ae-76d295ec9fe1-istiod-ca-cert\") pod \"9b03a2ca-af48-490e-89ae-76d295ec9fe1\" (UID: \"9b03a2ca-af48-490e-89ae-76d295ec9fe1\") " Jul 02 22:02:29.067754 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:29.067664 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rlbfk\" (UniqueName: \"kubernetes.io/projected/9b03a2ca-af48-490e-89ae-76d295ec9fe1-kube-api-access-rlbfk\") pod \"9b03a2ca-af48-490e-89ae-76d295ec9fe1\" (UID: \"9b03a2ca-af48-490e-89ae-76d295ec9fe1\") " Jul 02 22:02:29.067754 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:29.067696 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b03a2ca-af48-490e-89ae-76d295ec9fe1-workload-socket\") pod \"9b03a2ca-af48-490e-89ae-76d295ec9fe1\" (UID: \"9b03a2ca-af48-490e-89ae-76d295ec9fe1\") " Jul 02 22:02:29.067754 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:29.067715 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b03a2ca-af48-490e-89ae-76d295ec9fe1-istio-token\") pod \"9b03a2ca-af48-490e-89ae-76d295ec9fe1\" (UID: \"9b03a2ca-af48-490e-89ae-76d295ec9fe1\") " Jul 02 22:02:29.068005 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:29.067803 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b03a2ca-af48-490e-89ae-76d295ec9fe1-istio-envoy\") pod \"9b03a2ca-af48-490e-89ae-76d295ec9fe1\" (UID: \"9b03a2ca-af48-490e-89ae-76d295ec9fe1\") " Jul 02 22:02:29.068005 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:29.067895 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b03a2ca-af48-490e-89ae-76d295ec9fe1-istio-data" (OuterVolumeSpecName: "istio-data") pod "9b03a2ca-af48-490e-89ae-76d295ec9fe1" (UID: "9b03a2ca-af48-490e-89ae-76d295ec9fe1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:02:29.068005 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:29.067901 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b03a2ca-af48-490e-89ae-76d295ec9fe1-workload-certs\") pod \"9b03a2ca-af48-490e-89ae-76d295ec9fe1\" (UID: \"9b03a2ca-af48-490e-89ae-76d295ec9fe1\") " Jul 02 22:02:29.068005 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:29.067926 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b03a2ca-af48-490e-89ae-76d295ec9fe1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9b03a2ca-af48-490e-89ae-76d295ec9fe1" (UID: "9b03a2ca-af48-490e-89ae-76d295ec9fe1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:02:29.068005 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:29.067999 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b03a2ca-af48-490e-89ae-76d295ec9fe1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9b03a2ca-af48-490e-89ae-76d295ec9fe1" (UID: "9b03a2ca-af48-490e-89ae-76d295ec9fe1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:02:29.068264 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:29.068005 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b03a2ca-af48-490e-89ae-76d295ec9fe1-istio-podinfo\") pod \"9b03a2ca-af48-490e-89ae-76d295ec9fe1\" (UID: \"9b03a2ca-af48-490e-89ae-76d295ec9fe1\") " Jul 02 22:02:29.068264 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:29.068095 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b03a2ca-af48-490e-89ae-76d295ec9fe1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9b03a2ca-af48-490e-89ae-76d295ec9fe1" (UID: "9b03a2ca-af48-490e-89ae-76d295ec9fe1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:02:29.068264 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:29.068185 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b03a2ca-af48-490e-89ae-76d295ec9fe1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9b03a2ca-af48-490e-89ae-76d295ec9fe1" (UID: "9b03a2ca-af48-490e-89ae-76d295ec9fe1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 22:02:29.068439 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:29.068419 2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b03a2ca-af48-490e-89ae-76d295ec9fe1-workload-socket\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:02:29.068496 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:29.068443 2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b03a2ca-af48-490e-89ae-76d295ec9fe1-workload-certs\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:02:29.068496 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:29.068457 2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b03a2ca-af48-490e-89ae-76d295ec9fe1-credential-socket\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:02:29.068496 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:29.068471 2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b03a2ca-af48-490e-89ae-76d295ec9fe1-istio-data\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:02:29.068496 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:29.068485 2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b03a2ca-af48-490e-89ae-76d295ec9fe1-istiod-ca-cert\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:02:29.070121 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:29.070097 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9b03a2ca-af48-490e-89ae-76d295ec9fe1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9b03a2ca-af48-490e-89ae-76d295ec9fe1" (UID: "9b03a2ca-af48-490e-89ae-76d295ec9fe1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 22:02:29.070238 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:29.070194 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b03a2ca-af48-490e-89ae-76d295ec9fe1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9b03a2ca-af48-490e-89ae-76d295ec9fe1" (UID: "9b03a2ca-af48-490e-89ae-76d295ec9fe1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:02:29.070238 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:29.070200 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b03a2ca-af48-490e-89ae-76d295ec9fe1-istio-token" (OuterVolumeSpecName: "istio-token") pod "9b03a2ca-af48-490e-89ae-76d295ec9fe1" (UID: "9b03a2ca-af48-490e-89ae-76d295ec9fe1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:02:29.070323 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:29.070261 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b03a2ca-af48-490e-89ae-76d295ec9fe1-kube-api-access-rlbfk" (OuterVolumeSpecName: "kube-api-access-rlbfk") pod "9b03a2ca-af48-490e-89ae-76d295ec9fe1" (UID: "9b03a2ca-af48-490e-89ae-76d295ec9fe1"). InnerVolumeSpecName "kube-api-access-rlbfk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:02:29.128986 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:29.128959 2569 generic.go:356] "Generic (PLEG): container finished" podID="9b03a2ca-af48-490e-89ae-76d295ec9fe1" containerID="aac27110beeab85de75ce23829491d11429e0cf37f67bd5c2186327d51e0049a" exitCode=0 Jul 02 22:02:29.129107 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:29.129003 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5" Jul 02 22:02:29.129107 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:29.129050 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5" event={"ID":"9b03a2ca-af48-490e-89ae-76d295ec9fe1","Type":"ContainerDied","Data":"aac27110beeab85de75ce23829491d11429e0cf37f67bd5c2186327d51e0049a"} Jul 02 22:02:29.129226 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:29.129106 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5" event={"ID":"9b03a2ca-af48-490e-89ae-76d295ec9fe1","Type":"ContainerDied","Data":"4f55a4edd5234a4757c52018d12a407408269a573acd06e3191a32c99206a633"} Jul 02 22:02:29.129226 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:29.129137 2569 scope.go:117] "RemoveContainer" containerID="aac27110beeab85de75ce23829491d11429e0cf37f67bd5c2186327d51e0049a" Jul 02 22:02:29.138544 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:29.138528 2569 scope.go:117] "RemoveContainer" containerID="aac27110beeab85de75ce23829491d11429e0cf37f67bd5c2186327d51e0049a" Jul 02 22:02:29.138766 ip-10-0-139-75 kubenswrapper[2569]: E0702 22:02:29.138749 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aac27110beeab85de75ce23829491d11429e0cf37f67bd5c2186327d51e0049a\": container with ID starting with aac27110beeab85de75ce23829491d11429e0cf37f67bd5c2186327d51e0049a not found: ID does not exist" containerID="aac27110beeab85de75ce23829491d11429e0cf37f67bd5c2186327d51e0049a" Jul 02 22:02:29.138907 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:29.138774 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aac27110beeab85de75ce23829491d11429e0cf37f67bd5c2186327d51e0049a"} err="failed to get container status \"aac27110beeab85de75ce23829491d11429e0cf37f67bd5c2186327d51e0049a\": rpc error: code = NotFound desc = could not find container \"aac27110beeab85de75ce23829491d11429e0cf37f67bd5c2186327d51e0049a\": container with ID starting with aac27110beeab85de75ce23829491d11429e0cf37f67bd5c2186327d51e0049a not found: ID does not exist" Jul 02 22:02:29.151594 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:29.151574 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5"] Jul 02 22:02:29.155212 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:29.155188 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-e6c9d74c-openshift-default-cf4cb6d9f-bdhj5"] Jul 02 22:02:29.169328 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:29.169309 2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b03a2ca-af48-490e-89ae-76d295ec9fe1-istio-podinfo\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:02:29.169328 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:29.169330 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rlbfk\" (UniqueName: \"kubernetes.io/projected/9b03a2ca-af48-490e-89ae-76d295ec9fe1-kube-api-access-rlbfk\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:02:29.169452 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:29.169338 2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b03a2ca-af48-490e-89ae-76d295ec9fe1-istio-token\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:02:29.169452 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:29.169349 2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b03a2ca-af48-490e-89ae-76d295ec9fe1-istio-envoy\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:02:30.956237 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:02:30.956207 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b03a2ca-af48-490e-89ae-76d295ec9fe1" path="/var/lib/kubelet/pods/9b03a2ca-af48-490e-89ae-76d295ec9fe1/volumes" Jul 02 22:04:06.213075 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:06.213043 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p"] Jul 02 22:04:06.213689 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:06.213669 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b03a2ca-af48-490e-89ae-76d295ec9fe1" containerName="istio-proxy" Jul 02 22:04:06.213754 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:06.213693 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b03a2ca-af48-490e-89ae-76d295ec9fe1" containerName="istio-proxy" Jul 02 22:04:06.213824 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:06.213812 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b03a2ca-af48-490e-89ae-76d295ec9fe1" containerName="istio-proxy" Jul 02 22:04:06.217005 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:06.216987 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p" Jul 02 22:04:06.219751 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:06.219732 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-1861b388-openshift-default-dockercfg-xfkq7\"" Jul 02 22:04:06.244524 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:06.244506 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p"] Jul 02 22:04:06.379565 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:06.379529 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/213f6914-9020-4147-91c4-59299beeffcd-credential-socket\") pod \"e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p\" (UID: \"213f6914-9020-4147-91c4-59299beeffcd\") " pod="openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p" Jul 02 22:04:06.379724 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:06.379577 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/213f6914-9020-4147-91c4-59299beeffcd-workload-socket\") pod \"e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p\" (UID: \"213f6914-9020-4147-91c4-59299beeffcd\") " pod="openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p" Jul 02 22:04:06.379724 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:06.379660 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/213f6914-9020-4147-91c4-59299beeffcd-istiod-ca-cert\") pod \"e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p\" (UID: \"213f6914-9020-4147-91c4-59299beeffcd\") " pod="openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p" Jul 02 22:04:06.379724 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:06.379693 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/213f6914-9020-4147-91c4-59299beeffcd-istio-envoy\") pod \"e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p\" (UID: \"213f6914-9020-4147-91c4-59299beeffcd\") " pod="openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p" Jul 02 22:04:06.379912 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:06.379721 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/213f6914-9020-4147-91c4-59299beeffcd-istio-podinfo\") pod \"e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p\" (UID: \"213f6914-9020-4147-91c4-59299beeffcd\") " pod="openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p" Jul 02 22:04:06.379912 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:06.379832 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/213f6914-9020-4147-91c4-59299beeffcd-istio-data\") pod \"e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p\" (UID: \"213f6914-9020-4147-91c4-59299beeffcd\") " pod="openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p" Jul 02 22:04:06.379912 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:06.379901 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7zkp\" (UniqueName: \"kubernetes.io/projected/213f6914-9020-4147-91c4-59299beeffcd-kube-api-access-r7zkp\") pod \"e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p\" (UID: \"213f6914-9020-4147-91c4-59299beeffcd\") " pod="openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p" Jul 02 22:04:06.380071 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:06.379966 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/213f6914-9020-4147-91c4-59299beeffcd-workload-certs\") pod \"e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p\" (UID: \"213f6914-9020-4147-91c4-59299beeffcd\") " pod="openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p" Jul 02 22:04:06.380071 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:06.379999 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/213f6914-9020-4147-91c4-59299beeffcd-istio-token\") pod \"e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p\" (UID: \"213f6914-9020-4147-91c4-59299beeffcd\") " pod="openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p" Jul 02 22:04:06.481377 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:06.481297 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/213f6914-9020-4147-91c4-59299beeffcd-credential-socket\") pod \"e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p\" (UID: \"213f6914-9020-4147-91c4-59299beeffcd\") " pod="openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p" Jul 02 22:04:06.481377 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:06.481332 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/213f6914-9020-4147-91c4-59299beeffcd-workload-socket\") pod \"e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p\" (UID: \"213f6914-9020-4147-91c4-59299beeffcd\") " pod="openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p" Jul 02 22:04:06.481377 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:06.481363 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/213f6914-9020-4147-91c4-59299beeffcd-istiod-ca-cert\") pod \"e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p\" (UID: \"213f6914-9020-4147-91c4-59299beeffcd\") " pod="openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p" Jul 02 22:04:06.481652 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:06.481391 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/213f6914-9020-4147-91c4-59299beeffcd-istio-envoy\") pod \"e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p\" (UID: \"213f6914-9020-4147-91c4-59299beeffcd\") " pod="openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p" Jul 02 22:04:06.481652 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:06.481420 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/213f6914-9020-4147-91c4-59299beeffcd-istio-podinfo\") pod \"e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p\" (UID: \"213f6914-9020-4147-91c4-59299beeffcd\") " pod="openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p" Jul 02 22:04:06.481652 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:06.481458 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/213f6914-9020-4147-91c4-59299beeffcd-istio-data\") pod \"e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p\" (UID: \"213f6914-9020-4147-91c4-59299beeffcd\") " pod="openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p" Jul 02 22:04:06.481652 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:06.481506 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7zkp\" (UniqueName: \"kubernetes.io/projected/213f6914-9020-4147-91c4-59299beeffcd-kube-api-access-r7zkp\") pod \"e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p\" (UID: \"213f6914-9020-4147-91c4-59299beeffcd\") " pod="openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p" Jul 02 22:04:06.481652 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:06.481546 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/213f6914-9020-4147-91c4-59299beeffcd-workload-certs\") pod \"e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p\" (UID: \"213f6914-9020-4147-91c4-59299beeffcd\") " pod="openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p" Jul 02 22:04:06.481652 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:06.481572 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/213f6914-9020-4147-91c4-59299beeffcd-istio-token\") pod \"e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p\" (UID: \"213f6914-9020-4147-91c4-59299beeffcd\") " pod="openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p" Jul 02 22:04:06.482101 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:06.482075 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/213f6914-9020-4147-91c4-59299beeffcd-workload-socket\") pod \"e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p\" (UID: \"213f6914-9020-4147-91c4-59299beeffcd\") " pod="openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p" Jul 02 22:04:06.482194 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:06.482112 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/213f6914-9020-4147-91c4-59299beeffcd-credential-socket\") pod \"e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p\" (UID: \"213f6914-9020-4147-91c4-59299beeffcd\") " pod="openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p" Jul 02 22:04:06.482378 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:06.482356 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/213f6914-9020-4147-91c4-59299beeffcd-workload-certs\") pod \"e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p\" (UID: \"213f6914-9020-4147-91c4-59299beeffcd\") " pod="openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p" Jul 02 22:04:06.482454 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:06.482378 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/213f6914-9020-4147-91c4-59299beeffcd-istiod-ca-cert\") pod \"e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p\" (UID: \"213f6914-9020-4147-91c4-59299beeffcd\") " pod="openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p" Jul 02 22:04:06.482518 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:06.482474 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/213f6914-9020-4147-91c4-59299beeffcd-istio-data\") pod \"e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p\" (UID: \"213f6914-9020-4147-91c4-59299beeffcd\") " pod="openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p" Jul 02 22:04:06.484243 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:06.484215 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/213f6914-9020-4147-91c4-59299beeffcd-istio-envoy\") pod \"e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p\" (UID: \"213f6914-9020-4147-91c4-59299beeffcd\") " pod="openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p" Jul 02 22:04:06.484365 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:06.484303 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/213f6914-9020-4147-91c4-59299beeffcd-istio-podinfo\") pod \"e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p\" (UID: \"213f6914-9020-4147-91c4-59299beeffcd\") " pod="openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p" Jul 02 22:04:06.491893 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:06.491869 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/213f6914-9020-4147-91c4-59299beeffcd-istio-token\") pod \"e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p\" (UID: \"213f6914-9020-4147-91c4-59299beeffcd\") " pod="openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p" Jul 02 22:04:06.492032 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:06.492014 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7zkp\" (UniqueName: \"kubernetes.io/projected/213f6914-9020-4147-91c4-59299beeffcd-kube-api-access-r7zkp\") pod \"e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p\" (UID: \"213f6914-9020-4147-91c4-59299beeffcd\") " pod="openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p" Jul 02 22:04:06.527272 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:06.527253 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p" Jul 02 22:04:06.669554 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:06.669531 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p"] Jul 02 22:04:06.671321 ip-10-0-139-75 kubenswrapper[2569]: W0702 22:04:06.671294 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod213f6914_9020_4147_91c4_59299beeffcd.slice/crio-9e35f9c712e35eeb57721f21020e3594bfdf95130faf4ed4ee23bc4297ccef15 WatchSource:0}: Error finding container 9e35f9c712e35eeb57721f21020e3594bfdf95130faf4ed4ee23bc4297ccef15: Status 404 returned error can't find the container with id 9e35f9c712e35eeb57721f21020e3594bfdf95130faf4ed4ee23bc4297ccef15 Jul 02 22:04:06.673782 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:06.673742 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Jul 02 22:04:06.674954 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:06.674221 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Jul 02 22:04:06.674954 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:06.674285 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Jul 02 22:04:07.644557 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:07.644521 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p" event={"ID":"213f6914-9020-4147-91c4-59299beeffcd","Type":"ContainerStarted","Data":"7a23e53c391b4c0d5310fef7380c247b501c15f36f58fcbf61df06a6acb9f5a2"} Jul 02 22:04:07.645035 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:07.644587 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p" event={"ID":"213f6914-9020-4147-91c4-59299beeffcd","Type":"ContainerStarted","Data":"9e35f9c712e35eeb57721f21020e3594bfdf95130faf4ed4ee23bc4297ccef15"} Jul 02 22:04:07.681121 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:07.681072 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p" podStartSLOduration=1.681055297 podStartE2EDuration="1.681055297s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:04:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:04:07.67823971 +0000 UTC m=+1687.017210146" watchObservedRunningTime="2026-07-02 22:04:07.681055297 +0000 UTC m=+1687.020025734" Jul 02 22:04:08.528407 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:08.528374 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p" Jul 02 22:04:08.533060 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:08.533034 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p" Jul 02 22:04:08.649854 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:08.649821 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p" Jul 02 22:04:08.650865 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:08.650846 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p" Jul 02 22:04:22.730573 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:22.730447 2569 generic.go:356] "Generic (PLEG): container finished" podID="213f6914-9020-4147-91c4-59299beeffcd" containerID="7a23e53c391b4c0d5310fef7380c247b501c15f36f58fcbf61df06a6acb9f5a2" exitCode=0 Jul 02 22:04:22.730573 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:22.730527 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p" event={"ID":"213f6914-9020-4147-91c4-59299beeffcd","Type":"ContainerDied","Data":"7a23e53c391b4c0d5310fef7380c247b501c15f36f58fcbf61df06a6acb9f5a2"} Jul 02 22:04:22.731033 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:22.730957 2569 scope.go:117] "RemoveContainer" containerID="7a23e53c391b4c0d5310fef7380c247b501c15f36f58fcbf61df06a6acb9f5a2" Jul 02 22:04:22.732032 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:22.732000 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Jul 02 22:04:22.732139 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:22.732068 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Jul 02 22:04:22.732139 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:22.732099 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Jul 02 22:04:23.649948 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:23.649913 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p" Jul 02 22:04:23.737324 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:23.737288 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p" event={"ID":"213f6914-9020-4147-91c4-59299beeffcd","Type":"ContainerStarted","Data":"94f1411e62c629408e7c95b30c7c71c65bf61ef7da4ae031ca226fbdd2618ca5"} Jul 02 22:04:24.527858 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:24.527820 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p" Jul 02 22:04:24.529071 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:24.529044 2569 patch_prober.go:28] interesting pod/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 02 22:04:24.529208 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:24.529102 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p" podUID="213f6914-9020-4147-91c4-59299beeffcd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 02 22:04:25.528105 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:25.528071 2569 patch_prober.go:28] interesting pod/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 02 22:04:25.528481 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:25.528139 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p" podUID="213f6914-9020-4147-91c4-59299beeffcd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 02 22:04:26.528348 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:26.528311 2569 patch_prober.go:28] interesting pod/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 02 22:04:26.528710 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:26.528373 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p" podUID="213f6914-9020-4147-91c4-59299beeffcd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 02 22:04:27.552018 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:27.551988 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p" Jul 02 22:04:27.552503 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:27.552269 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p" Jul 02 22:04:27.553073 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:27.552991 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p" Jul 02 22:04:28.325484 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:28.325455 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p"] Jul 02 22:04:28.762088 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:28.762054 2569 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p" secret="" err="secret \"e2e-mt-1861b388-openshift-default-dockercfg-xfkq7\" not found" Jul 02 22:04:29.767452 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:29.767409 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p" podUID="213f6914-9020-4147-91c4-59299beeffcd" containerName="istio-proxy" containerID="cri-o://94f1411e62c629408e7c95b30c7c71c65bf61ef7da4ae031ca226fbdd2618ca5" gracePeriod=30 Jul 02 22:04:35.025392 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:35.025366 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p" Jul 02 22:04:35.184982 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:35.184897 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/213f6914-9020-4147-91c4-59299beeffcd-istiod-ca-cert\") pod \"213f6914-9020-4147-91c4-59299beeffcd\" (UID: \"213f6914-9020-4147-91c4-59299beeffcd\") " Jul 02 22:04:35.184982 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:35.184939 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/213f6914-9020-4147-91c4-59299beeffcd-istio-envoy\") pod \"213f6914-9020-4147-91c4-59299beeffcd\" (UID: \"213f6914-9020-4147-91c4-59299beeffcd\") " Jul 02 22:04:35.184982 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:35.184965 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/213f6914-9020-4147-91c4-59299beeffcd-workload-certs\") pod \"213f6914-9020-4147-91c4-59299beeffcd\" (UID: \"213f6914-9020-4147-91c4-59299beeffcd\") " Jul 02 22:04:35.184982 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:35.184983 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/213f6914-9020-4147-91c4-59299beeffcd-istio-data\") pod \"213f6914-9020-4147-91c4-59299beeffcd\" (UID: \"213f6914-9020-4147-91c4-59299beeffcd\") " Jul 02 22:04:35.185300 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:35.185010 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r7zkp\" (UniqueName: \"kubernetes.io/projected/213f6914-9020-4147-91c4-59299beeffcd-kube-api-access-r7zkp\") pod \"213f6914-9020-4147-91c4-59299beeffcd\" (UID: \"213f6914-9020-4147-91c4-59299beeffcd\") " Jul 02 22:04:35.185300 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:35.185036 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/213f6914-9020-4147-91c4-59299beeffcd-credential-socket\") pod \"213f6914-9020-4147-91c4-59299beeffcd\" (UID: \"213f6914-9020-4147-91c4-59299beeffcd\") " Jul 02 22:04:35.185300 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:35.185097 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/213f6914-9020-4147-91c4-59299beeffcd-istio-podinfo\") pod \"213f6914-9020-4147-91c4-59299beeffcd\" (UID: \"213f6914-9020-4147-91c4-59299beeffcd\") " Jul 02 22:04:35.185300 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:35.185199 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/213f6914-9020-4147-91c4-59299beeffcd-workload-socket\") pod \"213f6914-9020-4147-91c4-59299beeffcd\" (UID: \"213f6914-9020-4147-91c4-59299beeffcd\") " Jul 02 22:04:35.185300 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:35.185222 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/213f6914-9020-4147-91c4-59299beeffcd-istio-token\") pod \"213f6914-9020-4147-91c4-59299beeffcd\" (UID: \"213f6914-9020-4147-91c4-59299beeffcd\") " Jul 02 22:04:35.185300 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:35.185280 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/213f6914-9020-4147-91c4-59299beeffcd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "213f6914-9020-4147-91c4-59299beeffcd" (UID: "213f6914-9020-4147-91c4-59299beeffcd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:04:35.185611 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:35.185441 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/213f6914-9020-4147-91c4-59299beeffcd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "213f6914-9020-4147-91c4-59299beeffcd" (UID: "213f6914-9020-4147-91c4-59299beeffcd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:04:35.185611 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:35.185462 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/213f6914-9020-4147-91c4-59299beeffcd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "213f6914-9020-4147-91c4-59299beeffcd" (UID: "213f6914-9020-4147-91c4-59299beeffcd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 22:04:35.185611 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:35.185533 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/213f6914-9020-4147-91c4-59299beeffcd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "213f6914-9020-4147-91c4-59299beeffcd" (UID: "213f6914-9020-4147-91c4-59299beeffcd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:04:35.185781 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:35.185665 2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/213f6914-9020-4147-91c4-59299beeffcd-credential-socket\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:04:35.185781 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:35.185686 2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/213f6914-9020-4147-91c4-59299beeffcd-workload-socket\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:04:35.185781 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:35.185701 2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/213f6914-9020-4147-91c4-59299beeffcd-istiod-ca-cert\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:04:35.185781 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:35.185714 2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/213f6914-9020-4147-91c4-59299beeffcd-workload-certs\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:04:35.186260 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:35.186237 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/213f6914-9020-4147-91c4-59299beeffcd-istio-data" (OuterVolumeSpecName: "istio-data") pod "213f6914-9020-4147-91c4-59299beeffcd" (UID: "213f6914-9020-4147-91c4-59299beeffcd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:04:35.187328 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:35.187300 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/213f6914-9020-4147-91c4-59299beeffcd-kube-api-access-r7zkp" (OuterVolumeSpecName: "kube-api-access-r7zkp") pod "213f6914-9020-4147-91c4-59299beeffcd" (UID: "213f6914-9020-4147-91c4-59299beeffcd"). InnerVolumeSpecName "kube-api-access-r7zkp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:04:35.187439 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:35.187344 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/213f6914-9020-4147-91c4-59299beeffcd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "213f6914-9020-4147-91c4-59299beeffcd" (UID: "213f6914-9020-4147-91c4-59299beeffcd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:04:35.187686 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:35.187659 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/213f6914-9020-4147-91c4-59299beeffcd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "213f6914-9020-4147-91c4-59299beeffcd" (UID: "213f6914-9020-4147-91c4-59299beeffcd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 22:04:35.187686 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:35.187670 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/213f6914-9020-4147-91c4-59299beeffcd-istio-token" (OuterVolumeSpecName: "istio-token") pod "213f6914-9020-4147-91c4-59299beeffcd" (UID: "213f6914-9020-4147-91c4-59299beeffcd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:04:35.286886 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:35.286858 2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/213f6914-9020-4147-91c4-59299beeffcd-istio-envoy\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:04:35.286886 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:35.286879 2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/213f6914-9020-4147-91c4-59299beeffcd-istio-data\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:04:35.286886 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:35.286889 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r7zkp\" (UniqueName: \"kubernetes.io/projected/213f6914-9020-4147-91c4-59299beeffcd-kube-api-access-r7zkp\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:04:35.287084 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:35.286899 2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/213f6914-9020-4147-91c4-59299beeffcd-istio-podinfo\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:04:35.287084 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:35.286908 2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/213f6914-9020-4147-91c4-59299beeffcd-istio-token\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:04:35.801307 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:35.801273 2569 generic.go:356] "Generic (PLEG): container finished" podID="213f6914-9020-4147-91c4-59299beeffcd" containerID="94f1411e62c629408e7c95b30c7c71c65bf61ef7da4ae031ca226fbdd2618ca5" exitCode=0 Jul 02 22:04:35.801474 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:35.801326 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p" event={"ID":"213f6914-9020-4147-91c4-59299beeffcd","Type":"ContainerDied","Data":"94f1411e62c629408e7c95b30c7c71c65bf61ef7da4ae031ca226fbdd2618ca5"} Jul 02 22:04:35.801474 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:35.801398 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p" event={"ID":"213f6914-9020-4147-91c4-59299beeffcd","Type":"ContainerDied","Data":"9e35f9c712e35eeb57721f21020e3594bfdf95130faf4ed4ee23bc4297ccef15"} Jul 02 22:04:35.801474 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:35.801330 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p" Jul 02 22:04:35.801474 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:35.801425 2569 scope.go:117] "RemoveContainer" containerID="94f1411e62c629408e7c95b30c7c71c65bf61ef7da4ae031ca226fbdd2618ca5" Jul 02 22:04:35.812742 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:35.812724 2569 scope.go:117] "RemoveContainer" containerID="7a23e53c391b4c0d5310fef7380c247b501c15f36f58fcbf61df06a6acb9f5a2" Jul 02 22:04:35.821577 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:35.821561 2569 scope.go:117] "RemoveContainer" containerID="94f1411e62c629408e7c95b30c7c71c65bf61ef7da4ae031ca226fbdd2618ca5" Jul 02 22:04:35.821838 ip-10-0-139-75 kubenswrapper[2569]: E0702 22:04:35.821821 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94f1411e62c629408e7c95b30c7c71c65bf61ef7da4ae031ca226fbdd2618ca5\": container with ID starting with 94f1411e62c629408e7c95b30c7c71c65bf61ef7da4ae031ca226fbdd2618ca5 not found: ID does not exist" containerID="94f1411e62c629408e7c95b30c7c71c65bf61ef7da4ae031ca226fbdd2618ca5" Jul 02 22:04:35.821895 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:35.821846 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94f1411e62c629408e7c95b30c7c71c65bf61ef7da4ae031ca226fbdd2618ca5"} err="failed to get container status \"94f1411e62c629408e7c95b30c7c71c65bf61ef7da4ae031ca226fbdd2618ca5\": rpc error: code = NotFound desc = could not find container \"94f1411e62c629408e7c95b30c7c71c65bf61ef7da4ae031ca226fbdd2618ca5\": container with ID starting with 94f1411e62c629408e7c95b30c7c71c65bf61ef7da4ae031ca226fbdd2618ca5 not found: ID does not exist" Jul 02 22:04:35.821895 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:35.821863 2569 scope.go:117] "RemoveContainer" containerID="7a23e53c391b4c0d5310fef7380c247b501c15f36f58fcbf61df06a6acb9f5a2" Jul 02 22:04:35.822117 ip-10-0-139-75 kubenswrapper[2569]: E0702 22:04:35.822099 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a23e53c391b4c0d5310fef7380c247b501c15f36f58fcbf61df06a6acb9f5a2\": container with ID starting with 7a23e53c391b4c0d5310fef7380c247b501c15f36f58fcbf61df06a6acb9f5a2 not found: ID does not exist" containerID="7a23e53c391b4c0d5310fef7380c247b501c15f36f58fcbf61df06a6acb9f5a2" Jul 02 22:04:35.822171 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:35.822122 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a23e53c391b4c0d5310fef7380c247b501c15f36f58fcbf61df06a6acb9f5a2"} err="failed to get container status \"7a23e53c391b4c0d5310fef7380c247b501c15f36f58fcbf61df06a6acb9f5a2\": rpc error: code = NotFound desc = could not find container \"7a23e53c391b4c0d5310fef7380c247b501c15f36f58fcbf61df06a6acb9f5a2\": container with ID starting with 7a23e53c391b4c0d5310fef7380c247b501c15f36f58fcbf61df06a6acb9f5a2 not found: ID does not exist" Jul 02 22:04:35.828121 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:35.828099 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p"] Jul 02 22:04:35.832864 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:35.832844 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-1861b388-openshift-default-76cd8fd4cf-hzv4p"] Jul 02 22:04:36.957313 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:36.957281 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="213f6914-9020-4147-91c4-59299beeffcd" path="/var/lib/kubelet/pods/213f6914-9020-4147-91c4-59299beeffcd/volumes" Jul 02 22:04:48.877552 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:48.877521 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5"] Jul 02 22:04:48.878169 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:48.878152 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="213f6914-9020-4147-91c4-59299beeffcd" containerName="istio-proxy" Jul 02 22:04:48.878233 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:48.878172 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="213f6914-9020-4147-91c4-59299beeffcd" containerName="istio-proxy" Jul 02 22:04:48.878299 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:48.878287 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="213f6914-9020-4147-91c4-59299beeffcd" containerName="istio-proxy" Jul 02 22:04:48.881873 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:48.881853 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5" Jul 02 22:04:48.884401 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:48.884381 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-a72fe752-openshift-default-dockercfg-zjgft\"" Jul 02 22:04:48.895591 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:48.895567 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5"] Jul 02 22:04:48.913744 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:48.913722 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36821903-10c6-4219-9bb9-56e0be83661e-istio-data\") pod \"e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5\" (UID: \"36821903-10c6-4219-9bb9-56e0be83661e\") " pod="openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5" Jul 02 22:04:48.913877 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:48.913768 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36821903-10c6-4219-9bb9-56e0be83661e-workload-certs\") pod \"e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5\" (UID: \"36821903-10c6-4219-9bb9-56e0be83661e\") " pod="openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5" Jul 02 22:04:48.913877 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:48.913817 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36821903-10c6-4219-9bb9-56e0be83661e-credential-socket\") pod \"e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5\" (UID: \"36821903-10c6-4219-9bb9-56e0be83661e\") " pod="openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5" Jul 02 22:04:48.913877 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:48.913837 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36821903-10c6-4219-9bb9-56e0be83661e-istio-envoy\") pod \"e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5\" (UID: \"36821903-10c6-4219-9bb9-56e0be83661e\") " pod="openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5" Jul 02 22:04:48.913877 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:48.913874 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36821903-10c6-4219-9bb9-56e0be83661e-istio-token\") pod \"e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5\" (UID: \"36821903-10c6-4219-9bb9-56e0be83661e\") " pod="openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5" Jul 02 22:04:48.914054 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:48.913929 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36821903-10c6-4219-9bb9-56e0be83661e-istio-podinfo\") pod \"e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5\" (UID: \"36821903-10c6-4219-9bb9-56e0be83661e\") " pod="openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5" Jul 02 22:04:48.914054 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:48.914010 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36821903-10c6-4219-9bb9-56e0be83661e-workload-socket\") pod \"e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5\" (UID: \"36821903-10c6-4219-9bb9-56e0be83661e\") " pod="openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5" Jul 02 22:04:48.914208 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:48.914081 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36821903-10c6-4219-9bb9-56e0be83661e-istiod-ca-cert\") pod \"e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5\" (UID: \"36821903-10c6-4219-9bb9-56e0be83661e\") " pod="openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5" Jul 02 22:04:48.914208 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:48.914131 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggzzv\" (UniqueName: \"kubernetes.io/projected/36821903-10c6-4219-9bb9-56e0be83661e-kube-api-access-ggzzv\") pod \"e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5\" (UID: \"36821903-10c6-4219-9bb9-56e0be83661e\") " pod="openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5" Jul 02 22:04:49.015545 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:49.015508 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36821903-10c6-4219-9bb9-56e0be83661e-workload-certs\") pod \"e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5\" (UID: \"36821903-10c6-4219-9bb9-56e0be83661e\") " pod="openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5" Jul 02 22:04:49.015737 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:49.015561 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36821903-10c6-4219-9bb9-56e0be83661e-credential-socket\") pod \"e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5\" (UID: \"36821903-10c6-4219-9bb9-56e0be83661e\") " pod="openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5" Jul 02 22:04:49.015737 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:49.015580 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36821903-10c6-4219-9bb9-56e0be83661e-istio-envoy\") pod \"e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5\" (UID: \"36821903-10c6-4219-9bb9-56e0be83661e\") " pod="openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5" Jul 02 22:04:49.015737 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:49.015614 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36821903-10c6-4219-9bb9-56e0be83661e-istio-token\") pod \"e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5\" (UID: \"36821903-10c6-4219-9bb9-56e0be83661e\") " pod="openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5" Jul 02 22:04:49.015737 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:49.015633 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36821903-10c6-4219-9bb9-56e0be83661e-istio-podinfo\") pod \"e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5\" (UID: \"36821903-10c6-4219-9bb9-56e0be83661e\") " pod="openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5" Jul 02 22:04:49.016004 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:49.015808 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36821903-10c6-4219-9bb9-56e0be83661e-workload-socket\") pod \"e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5\" (UID: \"36821903-10c6-4219-9bb9-56e0be83661e\") " pod="openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5" Jul 02 22:04:49.016004 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:49.015862 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36821903-10c6-4219-9bb9-56e0be83661e-istiod-ca-cert\") pod \"e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5\" (UID: \"36821903-10c6-4219-9bb9-56e0be83661e\") " pod="openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5" Jul 02 22:04:49.016004 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:49.015903 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggzzv\" (UniqueName: \"kubernetes.io/projected/36821903-10c6-4219-9bb9-56e0be83661e-kube-api-access-ggzzv\") pod \"e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5\" (UID: \"36821903-10c6-4219-9bb9-56e0be83661e\") " pod="openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5" Jul 02 22:04:49.016004 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:49.015956 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36821903-10c6-4219-9bb9-56e0be83661e-istio-data\") pod \"e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5\" (UID: \"36821903-10c6-4219-9bb9-56e0be83661e\") " pod="openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5" Jul 02 22:04:49.016221 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:49.016003 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36821903-10c6-4219-9bb9-56e0be83661e-workload-certs\") pod \"e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5\" (UID: \"36821903-10c6-4219-9bb9-56e0be83661e\") " pod="openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5" Jul 02 22:04:49.016221 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:49.016065 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36821903-10c6-4219-9bb9-56e0be83661e-credential-socket\") pod \"e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5\" (UID: \"36821903-10c6-4219-9bb9-56e0be83661e\") " pod="openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5" Jul 02 22:04:49.016329 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:49.016263 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36821903-10c6-4219-9bb9-56e0be83661e-workload-socket\") pod \"e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5\" (UID: \"36821903-10c6-4219-9bb9-56e0be83661e\") " pod="openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5" Jul 02 22:04:49.016422 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:49.016401 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36821903-10c6-4219-9bb9-56e0be83661e-istio-data\") pod \"e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5\" (UID: \"36821903-10c6-4219-9bb9-56e0be83661e\") " pod="openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5" Jul 02 22:04:49.016695 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:49.016672 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36821903-10c6-4219-9bb9-56e0be83661e-istiod-ca-cert\") pod \"e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5\" (UID: \"36821903-10c6-4219-9bb9-56e0be83661e\") " pod="openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5" Jul 02 22:04:49.017935 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:49.017915 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36821903-10c6-4219-9bb9-56e0be83661e-istio-envoy\") pod \"e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5\" (UID: \"36821903-10c6-4219-9bb9-56e0be83661e\") " pod="openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5" Jul 02 22:04:49.018220 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:49.018204 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36821903-10c6-4219-9bb9-56e0be83661e-istio-podinfo\") pod \"e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5\" (UID: \"36821903-10c6-4219-9bb9-56e0be83661e\") " pod="openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5" Jul 02 22:04:49.024119 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:49.024099 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36821903-10c6-4219-9bb9-56e0be83661e-istio-token\") pod \"e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5\" (UID: \"36821903-10c6-4219-9bb9-56e0be83661e\") " pod="openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5" Jul 02 22:04:49.024563 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:49.024532 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggzzv\" (UniqueName: \"kubernetes.io/projected/36821903-10c6-4219-9bb9-56e0be83661e-kube-api-access-ggzzv\") pod \"e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5\" (UID: \"36821903-10c6-4219-9bb9-56e0be83661e\") " pod="openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5" Jul 02 22:04:49.192757 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:49.192665 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5" Jul 02 22:04:49.339486 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:49.339461 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5"] Jul 02 22:04:49.341726 ip-10-0-139-75 kubenswrapper[2569]: W0702 22:04:49.341700 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36821903_10c6_4219_9bb9_56e0be83661e.slice/crio-def45345f2d0aa3604c83e752c5d52d8d9dae7e93b10d0fd1deab58b04615432 WatchSource:0}: Error finding container def45345f2d0aa3604c83e752c5d52d8d9dae7e93b10d0fd1deab58b04615432: Status 404 returned error can't find the container with id def45345f2d0aa3604c83e752c5d52d8d9dae7e93b10d0fd1deab58b04615432 Jul 02 22:04:49.343981 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:49.343947 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Jul 02 22:04:49.344068 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:49.344014 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Jul 02 22:04:49.344068 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:49.344041 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Jul 02 22:04:49.883139 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:49.883102 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5" event={"ID":"36821903-10c6-4219-9bb9-56e0be83661e","Type":"ContainerStarted","Data":"dc4c72717080086bcc2b32df14aac463da1deb16ebd4540369fb4f27bcf0b340"} Jul 02 22:04:49.883701 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:49.883153 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5" event={"ID":"36821903-10c6-4219-9bb9-56e0be83661e","Type":"ContainerStarted","Data":"def45345f2d0aa3604c83e752c5d52d8d9dae7e93b10d0fd1deab58b04615432"} Jul 02 22:04:49.907730 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:49.907668 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5" podStartSLOduration=1.907652415 podStartE2EDuration="1.907652415s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:04:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:04:49.905935538 +0000 UTC m=+1729.244905976" watchObservedRunningTime="2026-07-02 22:04:49.907652415 +0000 UTC m=+1729.246622912" Jul 02 22:04:50.193295 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:50.193208 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5" Jul 02 22:04:50.198099 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:50.198077 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5" Jul 02 22:04:50.888617 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:50.888586 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5" Jul 02 22:04:50.889447 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:04:50.889428 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5" Jul 02 22:05:06.313105 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:05:06.312677 2569 patch_prober.go:28] interesting pod/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": read tcp 10.132.0.2:59656->10.132.0.55:15021: read: connection reset by peer" start-of-body= Jul 02 22:05:06.313105 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:05:06.312755 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5" podUID="36821903-10c6-4219-9bb9-56e0be83661e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": read tcp 10.132.0.2:59656->10.132.0.55:15021: read: connection reset by peer" Jul 02 22:05:06.971013 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:05:06.970982 2569 generic.go:356] "Generic (PLEG): container finished" podID="36821903-10c6-4219-9bb9-56e0be83661e" containerID="dc4c72717080086bcc2b32df14aac463da1deb16ebd4540369fb4f27bcf0b340" exitCode=0 Jul 02 22:05:06.971239 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:05:06.971024 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5" event={"ID":"36821903-10c6-4219-9bb9-56e0be83661e","Type":"ContainerDied","Data":"dc4c72717080086bcc2b32df14aac463da1deb16ebd4540369fb4f27bcf0b340"} Jul 02 22:05:06.971418 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:05:06.971400 2569 scope.go:117] "RemoveContainer" containerID="dc4c72717080086bcc2b32df14aac463da1deb16ebd4540369fb4f27bcf0b340" Jul 02 22:05:06.972405 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:05:06.972377 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Jul 02 22:05:06.972465 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:05:06.972444 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Jul 02 22:05:06.972501 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:05:06.972477 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Jul 02 22:05:07.995232 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:05:07.995190 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5" event={"ID":"36821903-10c6-4219-9bb9-56e0be83661e","Type":"ContainerStarted","Data":"4101d5d559545d89d69de4e60c7b9f5fddf907a877dd8e93ff17480f659b9182"} Jul 02 22:05:08.193114 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:05:08.193070 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5" Jul 02 22:05:08.193311 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:05:08.193208 2569 patch_prober.go:28] interesting pod/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 02 22:05:08.193311 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:05:08.193251 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5" podUID="36821903-10c6-4219-9bb9-56e0be83661e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 02 22:05:09.194001 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:05:09.193953 2569 patch_prober.go:28] interesting pod/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 02 22:05:09.194473 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:05:09.194028 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5" podUID="36821903-10c6-4219-9bb9-56e0be83661e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 02 22:05:10.193455 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:05:10.193411 2569 patch_prober.go:28] interesting pod/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 02 22:05:10.193664 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:05:10.193478 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5" podUID="36821903-10c6-4219-9bb9-56e0be83661e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 02 22:05:11.194811 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:05:11.194208 2569 patch_prober.go:28] interesting pod/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 02 22:05:11.194811 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:05:11.194295 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5" podUID="36821903-10c6-4219-9bb9-56e0be83661e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 02 22:05:12.196723 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:05:12.196693 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5" Jul 02 22:05:12.197177 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:05:12.197073 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5" Jul 02 22:05:12.197826 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:05:12.197810 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5" Jul 02 22:06:47.121741 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:06:47.121706 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5"] Jul 02 22:06:47.122367 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:06:47.122018 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5" podUID="36821903-10c6-4219-9bb9-56e0be83661e" containerName="istio-proxy" containerID="cri-o://4101d5d559545d89d69de4e60c7b9f5fddf907a877dd8e93ff17480f659b9182" gracePeriod=30 Jul 02 22:06:52.395943 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:06:52.395876 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5" Jul 02 22:06:52.534495 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:06:52.534458 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36821903-10c6-4219-9bb9-56e0be83661e-workload-certs\") pod \"36821903-10c6-4219-9bb9-56e0be83661e\" (UID: \"36821903-10c6-4219-9bb9-56e0be83661e\") " Jul 02 22:06:52.534677 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:06:52.534556 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36821903-10c6-4219-9bb9-56e0be83661e-istio-data\") pod \"36821903-10c6-4219-9bb9-56e0be83661e\" (UID: \"36821903-10c6-4219-9bb9-56e0be83661e\") " Jul 02 22:06:52.534677 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:06:52.534595 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36821903-10c6-4219-9bb9-56e0be83661e-istio-envoy\") pod \"36821903-10c6-4219-9bb9-56e0be83661e\" (UID: \"36821903-10c6-4219-9bb9-56e0be83661e\") " Jul 02 22:06:52.534677 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:06:52.534643 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36821903-10c6-4219-9bb9-56e0be83661e-istiod-ca-cert\") pod \"36821903-10c6-4219-9bb9-56e0be83661e\" (UID: \"36821903-10c6-4219-9bb9-56e0be83661e\") " Jul 02 22:06:52.534860 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:06:52.534689 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36821903-10c6-4219-9bb9-56e0be83661e-credential-socket\") pod \"36821903-10c6-4219-9bb9-56e0be83661e\" (UID: \"36821903-10c6-4219-9bb9-56e0be83661e\") " Jul 02 22:06:52.534860 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:06:52.534740 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36821903-10c6-4219-9bb9-56e0be83661e-istio-token\") pod \"36821903-10c6-4219-9bb9-56e0be83661e\" (UID: \"36821903-10c6-4219-9bb9-56e0be83661e\") " Jul 02 22:06:52.534860 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:06:52.534763 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36821903-10c6-4219-9bb9-56e0be83661e-istio-podinfo\") pod \"36821903-10c6-4219-9bb9-56e0be83661e\" (UID: \"36821903-10c6-4219-9bb9-56e0be83661e\") " Jul 02 22:06:52.534860 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:06:52.534803 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36821903-10c6-4219-9bb9-56e0be83661e-workload-socket\") pod \"36821903-10c6-4219-9bb9-56e0be83661e\" (UID: \"36821903-10c6-4219-9bb9-56e0be83661e\") " Jul 02 22:06:52.534860 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:06:52.534816 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36821903-10c6-4219-9bb9-56e0be83661e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "36821903-10c6-4219-9bb9-56e0be83661e" (UID: "36821903-10c6-4219-9bb9-56e0be83661e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:06:52.534860 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:06:52.534856 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ggzzv\" (UniqueName: \"kubernetes.io/projected/36821903-10c6-4219-9bb9-56e0be83661e-kube-api-access-ggzzv\") pod \"36821903-10c6-4219-9bb9-56e0be83661e\" (UID: \"36821903-10c6-4219-9bb9-56e0be83661e\") " Jul 02 22:06:52.535246 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:06:52.535004 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36821903-10c6-4219-9bb9-56e0be83661e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "36821903-10c6-4219-9bb9-56e0be83661e" (UID: "36821903-10c6-4219-9bb9-56e0be83661e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:06:52.535246 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:06:52.535098 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36821903-10c6-4219-9bb9-56e0be83661e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "36821903-10c6-4219-9bb9-56e0be83661e" (UID: "36821903-10c6-4219-9bb9-56e0be83661e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 22:06:52.535443 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:06:52.535352 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36821903-10c6-4219-9bb9-56e0be83661e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "36821903-10c6-4219-9bb9-56e0be83661e" (UID: "36821903-10c6-4219-9bb9-56e0be83661e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:06:52.535443 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:06:52.535270 2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36821903-10c6-4219-9bb9-56e0be83661e-workload-certs\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:06:52.535443 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:06:52.535388 2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36821903-10c6-4219-9bb9-56e0be83661e-istiod-ca-cert\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:06:52.535443 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:06:52.535403 2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36821903-10c6-4219-9bb9-56e0be83661e-credential-socket\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:06:52.536006 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:06:52.535568 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36821903-10c6-4219-9bb9-56e0be83661e-istio-data" (OuterVolumeSpecName: "istio-data") pod "36821903-10c6-4219-9bb9-56e0be83661e" (UID: "36821903-10c6-4219-9bb9-56e0be83661e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:06:52.536983 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:06:52.536955 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36821903-10c6-4219-9bb9-56e0be83661e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "36821903-10c6-4219-9bb9-56e0be83661e" (UID: "36821903-10c6-4219-9bb9-56e0be83661e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:06:52.537192 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:06:52.537175 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36821903-10c6-4219-9bb9-56e0be83661e-kube-api-access-ggzzv" (OuterVolumeSpecName: "kube-api-access-ggzzv") pod "36821903-10c6-4219-9bb9-56e0be83661e" (UID: "36821903-10c6-4219-9bb9-56e0be83661e"). InnerVolumeSpecName "kube-api-access-ggzzv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:06:52.537275 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:06:52.537262 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/36821903-10c6-4219-9bb9-56e0be83661e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "36821903-10c6-4219-9bb9-56e0be83661e" (UID: "36821903-10c6-4219-9bb9-56e0be83661e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 22:06:52.537375 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:06:52.537346 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36821903-10c6-4219-9bb9-56e0be83661e-istio-token" (OuterVolumeSpecName: "istio-token") pod "36821903-10c6-4219-9bb9-56e0be83661e" (UID: "36821903-10c6-4219-9bb9-56e0be83661e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:06:52.563020 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:06:52.562990 2569 generic.go:356] "Generic (PLEG): container finished" podID="36821903-10c6-4219-9bb9-56e0be83661e" containerID="4101d5d559545d89d69de4e60c7b9f5fddf907a877dd8e93ff17480f659b9182" exitCode=0 Jul 02 22:06:52.563136 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:06:52.563040 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5" Jul 02 22:06:52.563136 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:06:52.563078 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5" event={"ID":"36821903-10c6-4219-9bb9-56e0be83661e","Type":"ContainerDied","Data":"4101d5d559545d89d69de4e60c7b9f5fddf907a877dd8e93ff17480f659b9182"} Jul 02 22:06:52.563211 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:06:52.563135 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5" event={"ID":"36821903-10c6-4219-9bb9-56e0be83661e","Type":"ContainerDied","Data":"def45345f2d0aa3604c83e752c5d52d8d9dae7e93b10d0fd1deab58b04615432"} Jul 02 22:06:52.563211 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:06:52.563165 2569 scope.go:117] "RemoveContainer" containerID="4101d5d559545d89d69de4e60c7b9f5fddf907a877dd8e93ff17480f659b9182" Jul 02 22:06:52.572932 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:06:52.572914 2569 scope.go:117] "RemoveContainer" containerID="dc4c72717080086bcc2b32df14aac463da1deb16ebd4540369fb4f27bcf0b340" Jul 02 22:06:52.580901 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:06:52.580883 2569 scope.go:117] "RemoveContainer" containerID="4101d5d559545d89d69de4e60c7b9f5fddf907a877dd8e93ff17480f659b9182" Jul 02 22:06:52.581150 ip-10-0-139-75 kubenswrapper[2569]: E0702 22:06:52.581134 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4101d5d559545d89d69de4e60c7b9f5fddf907a877dd8e93ff17480f659b9182\": container with ID starting with 4101d5d559545d89d69de4e60c7b9f5fddf907a877dd8e93ff17480f659b9182 not found: ID does not exist" containerID="4101d5d559545d89d69de4e60c7b9f5fddf907a877dd8e93ff17480f659b9182" Jul 02 22:06:52.581200 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:06:52.581166 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4101d5d559545d89d69de4e60c7b9f5fddf907a877dd8e93ff17480f659b9182"} err="failed to get container status \"4101d5d559545d89d69de4e60c7b9f5fddf907a877dd8e93ff17480f659b9182\": rpc error: code = NotFound desc = could not find container \"4101d5d559545d89d69de4e60c7b9f5fddf907a877dd8e93ff17480f659b9182\": container with ID starting with 4101d5d559545d89d69de4e60c7b9f5fddf907a877dd8e93ff17480f659b9182 not found: ID does not exist" Jul 02 22:06:52.581200 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:06:52.581183 2569 scope.go:117] "RemoveContainer" containerID="dc4c72717080086bcc2b32df14aac463da1deb16ebd4540369fb4f27bcf0b340" Jul 02 22:06:52.581414 ip-10-0-139-75 kubenswrapper[2569]: E0702 22:06:52.581396 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc4c72717080086bcc2b32df14aac463da1deb16ebd4540369fb4f27bcf0b340\": container with ID starting with dc4c72717080086bcc2b32df14aac463da1deb16ebd4540369fb4f27bcf0b340 not found: ID does not exist" containerID="dc4c72717080086bcc2b32df14aac463da1deb16ebd4540369fb4f27bcf0b340" Jul 02 22:06:52.581470 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:06:52.581423 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc4c72717080086bcc2b32df14aac463da1deb16ebd4540369fb4f27bcf0b340"} err="failed to get container status \"dc4c72717080086bcc2b32df14aac463da1deb16ebd4540369fb4f27bcf0b340\": rpc error: code = NotFound desc = could not find container \"dc4c72717080086bcc2b32df14aac463da1deb16ebd4540369fb4f27bcf0b340\": container with ID starting with dc4c72717080086bcc2b32df14aac463da1deb16ebd4540369fb4f27bcf0b340 not found: ID does not exist" Jul 02 22:06:52.587299 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:06:52.587278 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5"] Jul 02 22:06:52.591395 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:06:52.591371 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-a72fe752-openshift-default-b779d7b59-mh2m5"] Jul 02 22:06:52.636366 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:06:52.636339 2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36821903-10c6-4219-9bb9-56e0be83661e-istio-data\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:06:52.636366 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:06:52.636363 2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36821903-10c6-4219-9bb9-56e0be83661e-istio-envoy\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:06:52.636366 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:06:52.636372 2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36821903-10c6-4219-9bb9-56e0be83661e-istio-token\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:06:52.636582 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:06:52.636380 2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36821903-10c6-4219-9bb9-56e0be83661e-istio-podinfo\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:06:52.636582 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:06:52.636388 2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36821903-10c6-4219-9bb9-56e0be83661e-workload-socket\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:06:52.636582 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:06:52.636398 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ggzzv\" (UniqueName: \"kubernetes.io/projected/36821903-10c6-4219-9bb9-56e0be83661e-kube-api-access-ggzzv\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:06:52.956199 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:06:52.956166 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36821903-10c6-4219-9bb9-56e0be83661e" path="/var/lib/kubelet/pods/36821903-10c6-4219-9bb9-56e0be83661e/volumes" Jul 02 22:07:42.336746 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:07:42.336712 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75"] Jul 02 22:07:42.337251 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:07:42.337171 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="213f6914-9020-4147-91c4-59299beeffcd" containerName="istio-proxy" Jul 02 22:07:42.337251 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:07:42.337183 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="213f6914-9020-4147-91c4-59299beeffcd" containerName="istio-proxy" Jul 02 22:07:42.337251 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:07:42.337203 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36821903-10c6-4219-9bb9-56e0be83661e" containerName="istio-proxy" Jul 02 22:07:42.337251 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:07:42.337209 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="36821903-10c6-4219-9bb9-56e0be83661e" containerName="istio-proxy" Jul 02 22:07:42.337251 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:07:42.337214 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36821903-10c6-4219-9bb9-56e0be83661e" containerName="istio-proxy" Jul 02 22:07:42.337251 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:07:42.337220 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="36821903-10c6-4219-9bb9-56e0be83661e" containerName="istio-proxy" Jul 02 22:07:42.337466 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:07:42.337285 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="213f6914-9020-4147-91c4-59299beeffcd" containerName="istio-proxy" Jul 02 22:07:42.337466 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:07:42.337292 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="36821903-10c6-4219-9bb9-56e0be83661e" containerName="istio-proxy" Jul 02 22:07:42.340414 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:07:42.340396 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75" Jul 02 22:07:42.343475 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:07:42.343452 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-624219-openshift-default-dockercfg-cqk48\"" Jul 02 22:07:42.353073 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:07:42.353047 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75"] Jul 02 22:07:42.510555 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:07:42.510519 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca911658-4c38-49bc-992f-d16fe85178d9-istio-token\") pod \"e2e-inf-a-624219-openshift-default-fbd54d677-gcx75\" (UID: \"ca911658-4c38-49bc-992f-d16fe85178d9\") " pod="openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75" Jul 02 22:07:42.510746 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:07:42.510568 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca911658-4c38-49bc-992f-d16fe85178d9-istiod-ca-cert\") pod \"e2e-inf-a-624219-openshift-default-fbd54d677-gcx75\" (UID: \"ca911658-4c38-49bc-992f-d16fe85178d9\") " pod="openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75" Jul 02 22:07:42.510746 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:07:42.510654 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca911658-4c38-49bc-992f-d16fe85178d9-credential-socket\") pod \"e2e-inf-a-624219-openshift-default-fbd54d677-gcx75\" (UID: \"ca911658-4c38-49bc-992f-d16fe85178d9\") " pod="openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75" Jul 02 22:07:42.510746 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:07:42.510736 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca911658-4c38-49bc-992f-d16fe85178d9-workload-socket\") pod \"e2e-inf-a-624219-openshift-default-fbd54d677-gcx75\" (UID: \"ca911658-4c38-49bc-992f-d16fe85178d9\") " pod="openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75" Jul 02 22:07:42.510899 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:07:42.510754 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca911658-4c38-49bc-992f-d16fe85178d9-istio-podinfo\") pod \"e2e-inf-a-624219-openshift-default-fbd54d677-gcx75\" (UID: \"ca911658-4c38-49bc-992f-d16fe85178d9\") " pod="openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75" Jul 02 22:07:42.510899 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:07:42.510813 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbvgm\" (UniqueName: \"kubernetes.io/projected/ca911658-4c38-49bc-992f-d16fe85178d9-kube-api-access-kbvgm\") pod \"e2e-inf-a-624219-openshift-default-fbd54d677-gcx75\" (UID: \"ca911658-4c38-49bc-992f-d16fe85178d9\") " pod="openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75" Jul 02 22:07:42.510899 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:07:42.510852 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca911658-4c38-49bc-992f-d16fe85178d9-workload-certs\") pod \"e2e-inf-a-624219-openshift-default-fbd54d677-gcx75\" (UID: \"ca911658-4c38-49bc-992f-d16fe85178d9\") " pod="openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75" Jul 02 22:07:42.510899 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:07:42.510889 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca911658-4c38-49bc-992f-d16fe85178d9-istio-data\") pod \"e2e-inf-a-624219-openshift-default-fbd54d677-gcx75\" (UID: \"ca911658-4c38-49bc-992f-d16fe85178d9\") " pod="openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75" Jul 02 22:07:42.511084 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:07:42.510940 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca911658-4c38-49bc-992f-d16fe85178d9-istio-envoy\") pod \"e2e-inf-a-624219-openshift-default-fbd54d677-gcx75\" (UID: \"ca911658-4c38-49bc-992f-d16fe85178d9\") " pod="openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75" Jul 02 22:07:42.612159 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:07:42.612066 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca911658-4c38-49bc-992f-d16fe85178d9-istiod-ca-cert\") pod \"e2e-inf-a-624219-openshift-default-fbd54d677-gcx75\" (UID: \"ca911658-4c38-49bc-992f-d16fe85178d9\") " pod="openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75" Jul 02 22:07:42.612159 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:07:42.612121 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca911658-4c38-49bc-992f-d16fe85178d9-credential-socket\") pod \"e2e-inf-a-624219-openshift-default-fbd54d677-gcx75\" (UID: \"ca911658-4c38-49bc-992f-d16fe85178d9\") " pod="openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75" Jul 02 22:07:42.612367 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:07:42.612171 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca911658-4c38-49bc-992f-d16fe85178d9-workload-socket\") pod \"e2e-inf-a-624219-openshift-default-fbd54d677-gcx75\" (UID: \"ca911658-4c38-49bc-992f-d16fe85178d9\") " pod="openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75" Jul 02 22:07:42.612367 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:07:42.612189 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca911658-4c38-49bc-992f-d16fe85178d9-istio-podinfo\") pod \"e2e-inf-a-624219-openshift-default-fbd54d677-gcx75\" (UID: \"ca911658-4c38-49bc-992f-d16fe85178d9\") " pod="openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75" Jul 02 22:07:42.612367 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:07:42.612220 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbvgm\" (UniqueName: \"kubernetes.io/projected/ca911658-4c38-49bc-992f-d16fe85178d9-kube-api-access-kbvgm\") pod \"e2e-inf-a-624219-openshift-default-fbd54d677-gcx75\" (UID: \"ca911658-4c38-49bc-992f-d16fe85178d9\") " pod="openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75" Jul 02 22:07:42.612520 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:07:42.612354 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca911658-4c38-49bc-992f-d16fe85178d9-workload-certs\") pod \"e2e-inf-a-624219-openshift-default-fbd54d677-gcx75\" (UID: \"ca911658-4c38-49bc-992f-d16fe85178d9\") " pod="openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75" Jul 02 22:07:42.612520 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:07:42.612422 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca911658-4c38-49bc-992f-d16fe85178d9-istio-data\") pod \"e2e-inf-a-624219-openshift-default-fbd54d677-gcx75\" (UID: \"ca911658-4c38-49bc-992f-d16fe85178d9\") " pod="openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75" Jul 02 22:07:42.612520 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:07:42.612463 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca911658-4c38-49bc-992f-d16fe85178d9-istio-envoy\") pod \"e2e-inf-a-624219-openshift-default-fbd54d677-gcx75\" (UID: \"ca911658-4c38-49bc-992f-d16fe85178d9\") " pod="openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75" Jul 02 22:07:42.612520 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:07:42.612510 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca911658-4c38-49bc-992f-d16fe85178d9-istio-token\") pod \"e2e-inf-a-624219-openshift-default-fbd54d677-gcx75\" (UID: \"ca911658-4c38-49bc-992f-d16fe85178d9\") " pod="openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75" Jul 02 22:07:42.612709 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:07:42.612601 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca911658-4c38-49bc-992f-d16fe85178d9-credential-socket\") pod \"e2e-inf-a-624219-openshift-default-fbd54d677-gcx75\" (UID: \"ca911658-4c38-49bc-992f-d16fe85178d9\") " pod="openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75" Jul 02 22:07:42.612709 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:07:42.612660 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca911658-4c38-49bc-992f-d16fe85178d9-workload-socket\") pod \"e2e-inf-a-624219-openshift-default-fbd54d677-gcx75\" (UID: \"ca911658-4c38-49bc-992f-d16fe85178d9\") " pod="openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75" Jul 02 22:07:42.612709 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:07:42.612698 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca911658-4c38-49bc-992f-d16fe85178d9-workload-certs\") pod \"e2e-inf-a-624219-openshift-default-fbd54d677-gcx75\" (UID: \"ca911658-4c38-49bc-992f-d16fe85178d9\") " pod="openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75" Jul 02 22:07:42.612912 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:07:42.612884 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca911658-4c38-49bc-992f-d16fe85178d9-istio-data\") pod \"e2e-inf-a-624219-openshift-default-fbd54d677-gcx75\" (UID: \"ca911658-4c38-49bc-992f-d16fe85178d9\") " pod="openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75" Jul 02 22:07:42.613029 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:07:42.612993 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca911658-4c38-49bc-992f-d16fe85178d9-istiod-ca-cert\") pod \"e2e-inf-a-624219-openshift-default-fbd54d677-gcx75\" (UID: \"ca911658-4c38-49bc-992f-d16fe85178d9\") " pod="openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75" Jul 02 22:07:42.614868 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:07:42.614850 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca911658-4c38-49bc-992f-d16fe85178d9-istio-envoy\") pod \"e2e-inf-a-624219-openshift-default-fbd54d677-gcx75\" (UID: \"ca911658-4c38-49bc-992f-d16fe85178d9\") " pod="openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75" Jul 02 22:07:42.614937 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:07:42.614903 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca911658-4c38-49bc-992f-d16fe85178d9-istio-podinfo\") pod \"e2e-inf-a-624219-openshift-default-fbd54d677-gcx75\" (UID: \"ca911658-4c38-49bc-992f-d16fe85178d9\") " pod="openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75" Jul 02 22:07:42.621339 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:07:42.621308 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbvgm\" (UniqueName: \"kubernetes.io/projected/ca911658-4c38-49bc-992f-d16fe85178d9-kube-api-access-kbvgm\") pod \"e2e-inf-a-624219-openshift-default-fbd54d677-gcx75\" (UID: \"ca911658-4c38-49bc-992f-d16fe85178d9\") " pod="openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75" Jul 02 22:07:42.621439 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:07:42.621341 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca911658-4c38-49bc-992f-d16fe85178d9-istio-token\") pod \"e2e-inf-a-624219-openshift-default-fbd54d677-gcx75\" (UID: \"ca911658-4c38-49bc-992f-d16fe85178d9\") " pod="openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75" Jul 02 22:07:42.651275 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:07:42.651246 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75" Jul 02 22:07:42.799732 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:07:42.799701 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75"] Jul 02 22:07:42.803585 ip-10-0-139-75 kubenswrapper[2569]: W0702 22:07:42.803556 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca911658_4c38_49bc_992f_d16fe85178d9.slice/crio-6fe32e69b3607c2d455e424a0133dbcea2500de8209ffd971ff3b313db87de72 WatchSource:0}: Error finding container 6fe32e69b3607c2d455e424a0133dbcea2500de8209ffd971ff3b313db87de72: Status 404 returned error can't find the container with id 6fe32e69b3607c2d455e424a0133dbcea2500de8209ffd971ff3b313db87de72 Jul 02 22:07:42.805999 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:07:42.805958 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Jul 02 22:07:42.806068 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:07:42.806034 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Jul 02 22:07:42.806117 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:07:42.806083 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Jul 02 22:07:42.820637 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:07:42.820608 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75" event={"ID":"ca911658-4c38-49bc-992f-d16fe85178d9","Type":"ContainerStarted","Data":"6fe32e69b3607c2d455e424a0133dbcea2500de8209ffd971ff3b313db87de72"} Jul 02 22:07:43.827444 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:07:43.827405 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75" event={"ID":"ca911658-4c38-49bc-992f-d16fe85178d9","Type":"ContainerStarted","Data":"0faac5244bffd2aab91c6e3c0ac726bcf2ddb982e85721f8a93106f1f9eebbc9"} Jul 02 22:07:43.853577 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:07:43.853532 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75" podStartSLOduration=1.853517691 podStartE2EDuration="1.853517691s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:07:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:07:43.851069877 +0000 UTC m=+1903.190040315" watchObservedRunningTime="2026-07-02 22:07:43.853517691 +0000 UTC m=+1903.192488125" Jul 02 22:07:44.651677 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:07:44.651641 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75" Jul 02 22:07:44.656453 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:07:44.656425 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75" Jul 02 22:07:44.832588 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:07:44.832548 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75" Jul 02 22:07:44.833626 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:07:44.833608 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75" Jul 02 22:08:00.873973 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:08:00.873926 2569 patch_prober.go:28] interesting pod/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 02 22:08:00.874434 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:08:00.874022 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75" podUID="ca911658-4c38-49bc-992f-d16fe85178d9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 02 22:08:01.941667 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:08:01.941637 2569 generic.go:356] "Generic (PLEG): container finished" podID="ca911658-4c38-49bc-992f-d16fe85178d9" containerID="0faac5244bffd2aab91c6e3c0ac726bcf2ddb982e85721f8a93106f1f9eebbc9" exitCode=0 Jul 02 22:08:01.942151 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:08:01.941673 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75" event={"ID":"ca911658-4c38-49bc-992f-d16fe85178d9","Type":"ContainerDied","Data":"0faac5244bffd2aab91c6e3c0ac726bcf2ddb982e85721f8a93106f1f9eebbc9"} Jul 02 22:08:01.942151 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:08:01.942091 2569 scope.go:117] "RemoveContainer" containerID="0faac5244bffd2aab91c6e3c0ac726bcf2ddb982e85721f8a93106f1f9eebbc9" Jul 02 22:08:01.946619 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:08:01.946559 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Jul 02 22:08:01.947044 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:08:01.946929 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Jul 02 22:08:01.947375 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:08:01.947349 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Jul 02 22:08:02.948655 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:08:02.948616 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75" event={"ID":"ca911658-4c38-49bc-992f-d16fe85178d9","Type":"ContainerStarted","Data":"b440a36f326753afc78df7722a2329e3f2b507895a056940a05c0c15bce062af"} Jul 02 22:08:03.651926 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:08:03.651885 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75" Jul 02 22:08:03.652207 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:08:03.652183 2569 patch_prober.go:28] interesting pod/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 02 22:08:03.652265 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:08:03.652242 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75" podUID="ca911658-4c38-49bc-992f-d16fe85178d9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 02 22:08:04.652568 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:08:04.652533 2569 patch_prober.go:28] interesting pod/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 02 22:08:04.652970 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:08:04.652598 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75" podUID="ca911658-4c38-49bc-992f-d16fe85178d9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 02 22:08:05.652264 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:08:05.652232 2569 patch_prober.go:28] interesting pod/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 02 22:08:05.652508 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:08:05.652486 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75" podUID="ca911658-4c38-49bc-992f-d16fe85178d9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 02 22:08:06.652067 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:08:06.652036 2569 patch_prober.go:28] interesting pod/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 02 22:08:06.652430 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:08:06.652099 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75" podUID="ca911658-4c38-49bc-992f-d16fe85178d9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 02 22:08:07.655674 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:08:07.655636 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75" Jul 02 22:08:07.656124 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:08:07.656046 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75" Jul 02 22:08:07.656572 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:08:07.656555 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75" Jul 02 22:09:39.656272 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:39.656202 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75"] Jul 02 22:09:39.657283 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:39.656497 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75" podUID="ca911658-4c38-49bc-992f-d16fe85178d9" containerName="istio-proxy" containerID="cri-o://b440a36f326753afc78df7722a2329e3f2b507895a056940a05c0c15bce062af" gracePeriod=30 Jul 02 22:09:44.914155 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:44.914132 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75" Jul 02 22:09:44.993365 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:44.993302 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca911658-4c38-49bc-992f-d16fe85178d9-istio-podinfo\") pod \"ca911658-4c38-49bc-992f-d16fe85178d9\" (UID: \"ca911658-4c38-49bc-992f-d16fe85178d9\") " Jul 02 22:09:44.993365 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:44.993346 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca911658-4c38-49bc-992f-d16fe85178d9-workload-socket\") pod \"ca911658-4c38-49bc-992f-d16fe85178d9\" (UID: \"ca911658-4c38-49bc-992f-d16fe85178d9\") " Jul 02 22:09:44.993520 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:44.993384 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca911658-4c38-49bc-992f-d16fe85178d9-istio-envoy\") pod \"ca911658-4c38-49bc-992f-d16fe85178d9\" (UID: \"ca911658-4c38-49bc-992f-d16fe85178d9\") " Jul 02 22:09:44.993520 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:44.993414 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca911658-4c38-49bc-992f-d16fe85178d9-workload-certs\") pod \"ca911658-4c38-49bc-992f-d16fe85178d9\" (UID: \"ca911658-4c38-49bc-992f-d16fe85178d9\") " Jul 02 22:09:44.993520 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:44.993466 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kbvgm\" (UniqueName: \"kubernetes.io/projected/ca911658-4c38-49bc-992f-d16fe85178d9-kube-api-access-kbvgm\") pod \"ca911658-4c38-49bc-992f-d16fe85178d9\" (UID: \"ca911658-4c38-49bc-992f-d16fe85178d9\") " Jul 02 22:09:44.993520 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:44.993486 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca911658-4c38-49bc-992f-d16fe85178d9-istiod-ca-cert\") pod \"ca911658-4c38-49bc-992f-d16fe85178d9\" (UID: \"ca911658-4c38-49bc-992f-d16fe85178d9\") " Jul 02 22:09:44.993520 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:44.993514 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca911658-4c38-49bc-992f-d16fe85178d9-credential-socket\") pod \"ca911658-4c38-49bc-992f-d16fe85178d9\" (UID: \"ca911658-4c38-49bc-992f-d16fe85178d9\") " Jul 02 22:09:44.993768 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:44.993540 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca911658-4c38-49bc-992f-d16fe85178d9-istio-data\") pod \"ca911658-4c38-49bc-992f-d16fe85178d9\" (UID: \"ca911658-4c38-49bc-992f-d16fe85178d9\") " Jul 02 22:09:44.993768 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:44.993580 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca911658-4c38-49bc-992f-d16fe85178d9-istio-token\") pod \"ca911658-4c38-49bc-992f-d16fe85178d9\" (UID: \"ca911658-4c38-49bc-992f-d16fe85178d9\") " Jul 02 22:09:44.993768 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:44.993660 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca911658-4c38-49bc-992f-d16fe85178d9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ca911658-4c38-49bc-992f-d16fe85178d9" (UID: "ca911658-4c38-49bc-992f-d16fe85178d9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:09:44.993966 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:44.993856 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca911658-4c38-49bc-992f-d16fe85178d9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ca911658-4c38-49bc-992f-d16fe85178d9" (UID: "ca911658-4c38-49bc-992f-d16fe85178d9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:09:44.994034 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:44.993996 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca911658-4c38-49bc-992f-d16fe85178d9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ca911658-4c38-49bc-992f-d16fe85178d9" (UID: "ca911658-4c38-49bc-992f-d16fe85178d9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 22:09:44.994110 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:44.994090 2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca911658-4c38-49bc-992f-d16fe85178d9-credential-socket\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:09:44.994174 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:44.994116 2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca911658-4c38-49bc-992f-d16fe85178d9-workload-socket\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:09:44.994316 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:44.994292 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca911658-4c38-49bc-992f-d16fe85178d9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ca911658-4c38-49bc-992f-d16fe85178d9" (UID: "ca911658-4c38-49bc-992f-d16fe85178d9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:09:44.994655 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:44.994631 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca911658-4c38-49bc-992f-d16fe85178d9-istio-data" (OuterVolumeSpecName: "istio-data") pod "ca911658-4c38-49bc-992f-d16fe85178d9" (UID: "ca911658-4c38-49bc-992f-d16fe85178d9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:09:44.995665 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:44.995637 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca911658-4c38-49bc-992f-d16fe85178d9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ca911658-4c38-49bc-992f-d16fe85178d9" (UID: "ca911658-4c38-49bc-992f-d16fe85178d9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:09:44.995766 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:44.995692 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ca911658-4c38-49bc-992f-d16fe85178d9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ca911658-4c38-49bc-992f-d16fe85178d9" (UID: "ca911658-4c38-49bc-992f-d16fe85178d9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 22:09:44.996294 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:44.996262 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca911658-4c38-49bc-992f-d16fe85178d9-istio-token" (OuterVolumeSpecName: "istio-token") pod "ca911658-4c38-49bc-992f-d16fe85178d9" (UID: "ca911658-4c38-49bc-992f-d16fe85178d9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:09:44.996368 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:44.996300 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca911658-4c38-49bc-992f-d16fe85178d9-kube-api-access-kbvgm" (OuterVolumeSpecName: "kube-api-access-kbvgm") pod "ca911658-4c38-49bc-992f-d16fe85178d9" (UID: "ca911658-4c38-49bc-992f-d16fe85178d9"). InnerVolumeSpecName "kube-api-access-kbvgm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:09:45.095217 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:45.095180 2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca911658-4c38-49bc-992f-d16fe85178d9-istio-envoy\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:09:45.095217 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:45.095209 2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca911658-4c38-49bc-992f-d16fe85178d9-workload-certs\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:09:45.095405 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:45.095223 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kbvgm\" (UniqueName: \"kubernetes.io/projected/ca911658-4c38-49bc-992f-d16fe85178d9-kube-api-access-kbvgm\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:09:45.095405 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:45.095236 2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca911658-4c38-49bc-992f-d16fe85178d9-istiod-ca-cert\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:09:45.095405 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:45.095247 2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca911658-4c38-49bc-992f-d16fe85178d9-istio-data\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:09:45.095405 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:45.095257 2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca911658-4c38-49bc-992f-d16fe85178d9-istio-token\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:09:45.095405 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:45.095268 2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca911658-4c38-49bc-992f-d16fe85178d9-istio-podinfo\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:09:45.479284 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:45.479243 2569 generic.go:356] "Generic (PLEG): container finished" podID="ca911658-4c38-49bc-992f-d16fe85178d9" containerID="b440a36f326753afc78df7722a2329e3f2b507895a056940a05c0c15bce062af" exitCode=0 Jul 02 22:09:45.479465 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:45.479301 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75" Jul 02 22:09:45.479465 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:45.479315 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75" event={"ID":"ca911658-4c38-49bc-992f-d16fe85178d9","Type":"ContainerDied","Data":"b440a36f326753afc78df7722a2329e3f2b507895a056940a05c0c15bce062af"} Jul 02 22:09:45.479465 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:45.479416 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75" event={"ID":"ca911658-4c38-49bc-992f-d16fe85178d9","Type":"ContainerDied","Data":"6fe32e69b3607c2d455e424a0133dbcea2500de8209ffd971ff3b313db87de72"} Jul 02 22:09:45.479465 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:45.479446 2569 scope.go:117] "RemoveContainer" containerID="b440a36f326753afc78df7722a2329e3f2b507895a056940a05c0c15bce062af" Jul 02 22:09:45.489341 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:45.489325 2569 scope.go:117] "RemoveContainer" containerID="0faac5244bffd2aab91c6e3c0ac726bcf2ddb982e85721f8a93106f1f9eebbc9" Jul 02 22:09:45.497276 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:45.497250 2569 scope.go:117] "RemoveContainer" containerID="b440a36f326753afc78df7722a2329e3f2b507895a056940a05c0c15bce062af" Jul 02 22:09:45.497514 ip-10-0-139-75 kubenswrapper[2569]: E0702 22:09:45.497495 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b440a36f326753afc78df7722a2329e3f2b507895a056940a05c0c15bce062af\": container with ID starting with b440a36f326753afc78df7722a2329e3f2b507895a056940a05c0c15bce062af not found: ID does not exist" containerID="b440a36f326753afc78df7722a2329e3f2b507895a056940a05c0c15bce062af" Jul 02 22:09:45.497565 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:45.497521 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b440a36f326753afc78df7722a2329e3f2b507895a056940a05c0c15bce062af"} err="failed to get container status \"b440a36f326753afc78df7722a2329e3f2b507895a056940a05c0c15bce062af\": rpc error: code = NotFound desc = could not find container \"b440a36f326753afc78df7722a2329e3f2b507895a056940a05c0c15bce062af\": container with ID starting with b440a36f326753afc78df7722a2329e3f2b507895a056940a05c0c15bce062af not found: ID does not exist" Jul 02 22:09:45.497565 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:45.497539 2569 scope.go:117] "RemoveContainer" containerID="0faac5244bffd2aab91c6e3c0ac726bcf2ddb982e85721f8a93106f1f9eebbc9" Jul 02 22:09:45.497778 ip-10-0-139-75 kubenswrapper[2569]: E0702 22:09:45.497764 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0faac5244bffd2aab91c6e3c0ac726bcf2ddb982e85721f8a93106f1f9eebbc9\": container with ID starting with 0faac5244bffd2aab91c6e3c0ac726bcf2ddb982e85721f8a93106f1f9eebbc9 not found: ID does not exist" containerID="0faac5244bffd2aab91c6e3c0ac726bcf2ddb982e85721f8a93106f1f9eebbc9" Jul 02 22:09:45.497848 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:45.497805 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0faac5244bffd2aab91c6e3c0ac726bcf2ddb982e85721f8a93106f1f9eebbc9"} err="failed to get container status \"0faac5244bffd2aab91c6e3c0ac726bcf2ddb982e85721f8a93106f1f9eebbc9\": rpc error: code = NotFound desc = could not find container \"0faac5244bffd2aab91c6e3c0ac726bcf2ddb982e85721f8a93106f1f9eebbc9\": container with ID starting with 0faac5244bffd2aab91c6e3c0ac726bcf2ddb982e85721f8a93106f1f9eebbc9 not found: ID does not exist" Jul 02 22:09:45.501037 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:45.501015 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75"] Jul 02 22:09:45.507100 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:45.507074 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-624219-openshift-default-fbd54d677-gcx75"] Jul 02 22:09:46.957513 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:46.957476 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca911658-4c38-49bc-992f-d16fe85178d9" path="/var/lib/kubelet/pods/ca911658-4c38-49bc-992f-d16fe85178d9/volumes" Jul 02 22:09:55.407149 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:55.407117 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc"] Jul 02 22:09:55.407768 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:55.407747 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca911658-4c38-49bc-992f-d16fe85178d9" containerName="istio-proxy" Jul 02 22:09:55.407904 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:55.407771 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca911658-4c38-49bc-992f-d16fe85178d9" containerName="istio-proxy" Jul 02 22:09:55.407964 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:55.407946 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca911658-4c38-49bc-992f-d16fe85178d9" containerName="istio-proxy" Jul 02 22:09:55.408016 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:55.407963 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="36821903-10c6-4219-9bb9-56e0be83661e" containerName="istio-proxy" Jul 02 22:09:55.410218 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:55.410198 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc" Jul 02 22:09:55.412492 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:55.412472 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-906c71-openshift-default-dockercfg-czfxt\"" Jul 02 22:09:55.423280 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:55.423257 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc"] Jul 02 22:09:55.491334 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:55.491305 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea3e06d6-9ee4-4ecf-89fc-ef3451524f87-istio-data\") pod \"e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc\" (UID: \"ea3e06d6-9ee4-4ecf-89fc-ef3451524f87\") " pod="openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc" Jul 02 22:09:55.491471 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:55.491339 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea3e06d6-9ee4-4ecf-89fc-ef3451524f87-workload-certs\") pod \"e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc\" (UID: \"ea3e06d6-9ee4-4ecf-89fc-ef3451524f87\") " pod="openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc" Jul 02 22:09:55.491471 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:55.491382 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea3e06d6-9ee4-4ecf-89fc-ef3451524f87-credential-socket\") pod \"e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc\" (UID: \"ea3e06d6-9ee4-4ecf-89fc-ef3451524f87\") " pod="openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc" Jul 02 22:09:55.491471 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:55.491428 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smqrx\" (UniqueName: \"kubernetes.io/projected/ea3e06d6-9ee4-4ecf-89fc-ef3451524f87-kube-api-access-smqrx\") pod \"e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc\" (UID: \"ea3e06d6-9ee4-4ecf-89fc-ef3451524f87\") " pod="openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc" Jul 02 22:09:55.491588 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:55.491479 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea3e06d6-9ee4-4ecf-89fc-ef3451524f87-istio-envoy\") pod \"e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc\" (UID: \"ea3e06d6-9ee4-4ecf-89fc-ef3451524f87\") " pod="openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc" Jul 02 22:09:55.491588 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:55.491496 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea3e06d6-9ee4-4ecf-89fc-ef3451524f87-istio-podinfo\") pod \"e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc\" (UID: \"ea3e06d6-9ee4-4ecf-89fc-ef3451524f87\") " pod="openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc" Jul 02 22:09:55.491588 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:55.491521 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea3e06d6-9ee4-4ecf-89fc-ef3451524f87-istio-token\") pod \"e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc\" (UID: \"ea3e06d6-9ee4-4ecf-89fc-ef3451524f87\") " pod="openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc" Jul 02 22:09:55.491588 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:55.491550 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea3e06d6-9ee4-4ecf-89fc-ef3451524f87-workload-socket\") pod \"e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc\" (UID: \"ea3e06d6-9ee4-4ecf-89fc-ef3451524f87\") " pod="openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc" Jul 02 22:09:55.491588 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:55.491570 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea3e06d6-9ee4-4ecf-89fc-ef3451524f87-istiod-ca-cert\") pod \"e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc\" (UID: \"ea3e06d6-9ee4-4ecf-89fc-ef3451524f87\") " pod="openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc" Jul 02 22:09:55.592845 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:55.592814 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smqrx\" (UniqueName: \"kubernetes.io/projected/ea3e06d6-9ee4-4ecf-89fc-ef3451524f87-kube-api-access-smqrx\") pod \"e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc\" (UID: \"ea3e06d6-9ee4-4ecf-89fc-ef3451524f87\") " pod="openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc" Jul 02 22:09:55.593036 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:55.592866 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea3e06d6-9ee4-4ecf-89fc-ef3451524f87-istio-envoy\") pod \"e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc\" (UID: \"ea3e06d6-9ee4-4ecf-89fc-ef3451524f87\") " pod="openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc" Jul 02 22:09:55.593036 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:55.592887 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea3e06d6-9ee4-4ecf-89fc-ef3451524f87-istio-podinfo\") pod \"e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc\" (UID: \"ea3e06d6-9ee4-4ecf-89fc-ef3451524f87\") " pod="openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc" Jul 02 22:09:55.593036 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:55.592913 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea3e06d6-9ee4-4ecf-89fc-ef3451524f87-istio-token\") pod \"e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc\" (UID: \"ea3e06d6-9ee4-4ecf-89fc-ef3451524f87\") " pod="openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc" Jul 02 22:09:55.593036 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:55.592937 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea3e06d6-9ee4-4ecf-89fc-ef3451524f87-workload-socket\") pod \"e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc\" (UID: \"ea3e06d6-9ee4-4ecf-89fc-ef3451524f87\") " pod="openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc" Jul 02 22:09:55.593036 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:55.592972 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea3e06d6-9ee4-4ecf-89fc-ef3451524f87-istiod-ca-cert\") pod \"e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc\" (UID: \"ea3e06d6-9ee4-4ecf-89fc-ef3451524f87\") " pod="openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc" Jul 02 22:09:55.593036 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:55.593023 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea3e06d6-9ee4-4ecf-89fc-ef3451524f87-istio-data\") pod \"e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc\" (UID: \"ea3e06d6-9ee4-4ecf-89fc-ef3451524f87\") " pod="openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc" Jul 02 22:09:55.593365 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:55.593050 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea3e06d6-9ee4-4ecf-89fc-ef3451524f87-workload-certs\") pod \"e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc\" (UID: \"ea3e06d6-9ee4-4ecf-89fc-ef3451524f87\") " pod="openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc" Jul 02 22:09:55.593365 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:55.593100 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea3e06d6-9ee4-4ecf-89fc-ef3451524f87-credential-socket\") pod \"e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc\" (UID: \"ea3e06d6-9ee4-4ecf-89fc-ef3451524f87\") " pod="openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc" Jul 02 22:09:55.593471 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:55.593356 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea3e06d6-9ee4-4ecf-89fc-ef3451524f87-workload-socket\") pod \"e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc\" (UID: \"ea3e06d6-9ee4-4ecf-89fc-ef3451524f87\") " pod="openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc" Jul 02 22:09:55.593536 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:55.593518 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea3e06d6-9ee4-4ecf-89fc-ef3451524f87-workload-certs\") pod \"e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc\" (UID: \"ea3e06d6-9ee4-4ecf-89fc-ef3451524f87\") " pod="openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc" Jul 02 22:09:55.593595 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:55.593533 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea3e06d6-9ee4-4ecf-89fc-ef3451524f87-credential-socket\") pod \"e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc\" (UID: \"ea3e06d6-9ee4-4ecf-89fc-ef3451524f87\") " pod="openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc" Jul 02 22:09:55.593692 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:55.593671 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea3e06d6-9ee4-4ecf-89fc-ef3451524f87-istio-data\") pod \"e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc\" (UID: \"ea3e06d6-9ee4-4ecf-89fc-ef3451524f87\") " pod="openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc" Jul 02 22:09:55.593881 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:55.593863 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea3e06d6-9ee4-4ecf-89fc-ef3451524f87-istiod-ca-cert\") pod \"e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc\" (UID: \"ea3e06d6-9ee4-4ecf-89fc-ef3451524f87\") " pod="openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc" Jul 02 22:09:55.595310 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:55.595277 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea3e06d6-9ee4-4ecf-89fc-ef3451524f87-istio-envoy\") pod \"e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc\" (UID: \"ea3e06d6-9ee4-4ecf-89fc-ef3451524f87\") " pod="openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc" Jul 02 22:09:55.595494 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:55.595470 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea3e06d6-9ee4-4ecf-89fc-ef3451524f87-istio-podinfo\") pod \"e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc\" (UID: \"ea3e06d6-9ee4-4ecf-89fc-ef3451524f87\") " pod="openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc" Jul 02 22:09:55.600869 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:55.600849 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-smqrx\" (UniqueName: \"kubernetes.io/projected/ea3e06d6-9ee4-4ecf-89fc-ef3451524f87-kube-api-access-smqrx\") pod \"e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc\" (UID: \"ea3e06d6-9ee4-4ecf-89fc-ef3451524f87\") " pod="openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc" Jul 02 22:09:55.601025 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:55.601006 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea3e06d6-9ee4-4ecf-89fc-ef3451524f87-istio-token\") pod \"e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc\" (UID: \"ea3e06d6-9ee4-4ecf-89fc-ef3451524f87\") " pod="openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc" Jul 02 22:09:55.721382 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:55.721307 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc" Jul 02 22:09:55.860373 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:55.860348 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc"] Jul 02 22:09:55.863246 ip-10-0-139-75 kubenswrapper[2569]: W0702 22:09:55.863218 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea3e06d6_9ee4_4ecf_89fc_ef3451524f87.slice/crio-86bda3cf075dc79d341621b4a5af9b6671b2ce2ea4e186f6a2f44337b7e5d0c0 WatchSource:0}: Error finding container 86bda3cf075dc79d341621b4a5af9b6671b2ce2ea4e186f6a2f44337b7e5d0c0: Status 404 returned error can't find the container with id 86bda3cf075dc79d341621b4a5af9b6671b2ce2ea4e186f6a2f44337b7e5d0c0 Jul 02 22:09:55.865453 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:55.865408 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Jul 02 22:09:55.865555 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:55.865507 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Jul 02 22:09:55.865598 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:55.865552 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Jul 02 22:09:56.540701 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:56.540666 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc" event={"ID":"ea3e06d6-9ee4-4ecf-89fc-ef3451524f87","Type":"ContainerStarted","Data":"d35575890a70807d2ede6b6c4462c618b2f3043e70e50a1d564570f733497d1f"} Jul 02 22:09:56.541092 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:56.540720 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc" event={"ID":"ea3e06d6-9ee4-4ecf-89fc-ef3451524f87","Type":"ContainerStarted","Data":"86bda3cf075dc79d341621b4a5af9b6671b2ce2ea4e186f6a2f44337b7e5d0c0"} Jul 02 22:09:56.560563 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:56.560489 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc" podStartSLOduration=1.560470883 podStartE2EDuration="1.560470883s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:09:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:09:56.558960646 +0000 UTC m=+2035.897931083" watchObservedRunningTime="2026-07-02 22:09:56.560470883 +0000 UTC m=+2035.899441319" Jul 02 22:09:56.721748 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:56.721714 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc" Jul 02 22:09:56.726586 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:56.726560 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc" Jul 02 22:09:57.546421 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:57.546381 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc" Jul 02 22:09:57.547338 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:09:57.547317 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc" Jul 02 22:10:13.588953 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:13.588906 2569 patch_prober.go:28] interesting pod/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 02 22:10:13.589356 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:13.589002 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc" podUID="ea3e06d6-9ee4-4ecf-89fc-ef3451524f87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 02 22:10:14.639804 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:14.639754 2569 generic.go:356] "Generic (PLEG): container finished" podID="ea3e06d6-9ee4-4ecf-89fc-ef3451524f87" containerID="d35575890a70807d2ede6b6c4462c618b2f3043e70e50a1d564570f733497d1f" exitCode=0 Jul 02 22:10:14.640229 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:14.639826 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc" event={"ID":"ea3e06d6-9ee4-4ecf-89fc-ef3451524f87","Type":"ContainerDied","Data":"d35575890a70807d2ede6b6c4462c618b2f3043e70e50a1d564570f733497d1f"} Jul 02 22:10:14.640280 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:14.640238 2569 scope.go:117] "RemoveContainer" containerID="d35575890a70807d2ede6b6c4462c618b2f3043e70e50a1d564570f733497d1f" Jul 02 22:10:14.641281 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:14.641253 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Jul 02 22:10:14.641353 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:14.641340 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Jul 02 22:10:14.641393 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:14.641375 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Jul 02 22:10:15.647576 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:15.647537 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc" event={"ID":"ea3e06d6-9ee4-4ecf-89fc-ef3451524f87","Type":"ContainerStarted","Data":"bb76892f8cb1b4622ed92ae5be4b0ef1db36ded5d92088f3d46dd39dd65d2992"} Jul 02 22:10:15.722025 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:15.721995 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc" Jul 02 22:10:15.722243 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:15.722219 2569 patch_prober.go:28] interesting pod/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 02 22:10:15.722289 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:15.722267 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc" podUID="ea3e06d6-9ee4-4ecf-89fc-ef3451524f87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 02 22:10:16.722432 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:16.722397 2569 patch_prober.go:28] interesting pod/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 02 22:10:16.722849 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:16.722458 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc" podUID="ea3e06d6-9ee4-4ecf-89fc-ef3451524f87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 02 22:10:17.722044 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:17.722005 2569 patch_prober.go:28] interesting pod/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 02 22:10:17.722258 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:17.722061 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc" podUID="ea3e06d6-9ee4-4ecf-89fc-ef3451524f87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 02 22:10:18.721844 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:18.721770 2569 patch_prober.go:28] interesting pod/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 02 22:10:18.722259 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:18.721863 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc" podUID="ea3e06d6-9ee4-4ecf-89fc-ef3451524f87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 02 22:10:19.729120 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:19.729090 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc" Jul 02 22:10:19.729543 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:19.729507 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc" Jul 02 22:10:19.730557 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:19.730536 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc" Jul 02 22:10:24.272205 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:24.272131 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc"] Jul 02 22:10:24.272591 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:24.272314 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc" podUID="ea3e06d6-9ee4-4ecf-89fc-ef3451524f87" containerName="istio-proxy" containerID="cri-o://bb76892f8cb1b4622ed92ae5be4b0ef1db36ded5d92088f3d46dd39dd65d2992" gracePeriod=30 Jul 02 22:10:29.538047 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:29.537978 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc" Jul 02 22:10:29.620151 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:29.620118 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea3e06d6-9ee4-4ecf-89fc-ef3451524f87-istio-data\") pod \"ea3e06d6-9ee4-4ecf-89fc-ef3451524f87\" (UID: \"ea3e06d6-9ee4-4ecf-89fc-ef3451524f87\") " Jul 02 22:10:29.620324 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:29.620163 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea3e06d6-9ee4-4ecf-89fc-ef3451524f87-workload-certs\") pod \"ea3e06d6-9ee4-4ecf-89fc-ef3451524f87\" (UID: \"ea3e06d6-9ee4-4ecf-89fc-ef3451524f87\") " Jul 02 22:10:29.620324 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:29.620226 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea3e06d6-9ee4-4ecf-89fc-ef3451524f87-istio-podinfo\") pod \"ea3e06d6-9ee4-4ecf-89fc-ef3451524f87\" (UID: \"ea3e06d6-9ee4-4ecf-89fc-ef3451524f87\") " Jul 02 22:10:29.620324 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:29.620269 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-smqrx\" (UniqueName: \"kubernetes.io/projected/ea3e06d6-9ee4-4ecf-89fc-ef3451524f87-kube-api-access-smqrx\") pod \"ea3e06d6-9ee4-4ecf-89fc-ef3451524f87\" (UID: \"ea3e06d6-9ee4-4ecf-89fc-ef3451524f87\") " Jul 02 22:10:29.620324 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:29.620294 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea3e06d6-9ee4-4ecf-89fc-ef3451524f87-credential-socket\") pod \"ea3e06d6-9ee4-4ecf-89fc-ef3451524f87\" (UID: \"ea3e06d6-9ee4-4ecf-89fc-ef3451524f87\") " Jul 02 22:10:29.620572 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:29.620332 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea3e06d6-9ee4-4ecf-89fc-ef3451524f87-istio-envoy\") pod \"ea3e06d6-9ee4-4ecf-89fc-ef3451524f87\" (UID: \"ea3e06d6-9ee4-4ecf-89fc-ef3451524f87\") " Jul 02 22:10:29.620572 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:29.620360 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea3e06d6-9ee4-4ecf-89fc-ef3451524f87-istiod-ca-cert\") pod \"ea3e06d6-9ee4-4ecf-89fc-ef3451524f87\" (UID: \"ea3e06d6-9ee4-4ecf-89fc-ef3451524f87\") " Jul 02 22:10:29.620572 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:29.620383 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea3e06d6-9ee4-4ecf-89fc-ef3451524f87-workload-socket\") pod \"ea3e06d6-9ee4-4ecf-89fc-ef3451524f87\" (UID: \"ea3e06d6-9ee4-4ecf-89fc-ef3451524f87\") " Jul 02 22:10:29.620572 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:29.620423 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea3e06d6-9ee4-4ecf-89fc-ef3451524f87-istio-token\") pod \"ea3e06d6-9ee4-4ecf-89fc-ef3451524f87\" (UID: \"ea3e06d6-9ee4-4ecf-89fc-ef3451524f87\") " Jul 02 22:10:29.620572 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:29.620483 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea3e06d6-9ee4-4ecf-89fc-ef3451524f87-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ea3e06d6-9ee4-4ecf-89fc-ef3451524f87" (UID: "ea3e06d6-9ee4-4ecf-89fc-ef3451524f87"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:10:29.620873 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:29.620712 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea3e06d6-9ee4-4ecf-89fc-ef3451524f87-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ea3e06d6-9ee4-4ecf-89fc-ef3451524f87" (UID: "ea3e06d6-9ee4-4ecf-89fc-ef3451524f87"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:10:29.620873 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:29.620773 2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea3e06d6-9ee4-4ecf-89fc-ef3451524f87-workload-certs\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:10:29.620991 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:29.620949 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea3e06d6-9ee4-4ecf-89fc-ef3451524f87-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ea3e06d6-9ee4-4ecf-89fc-ef3451524f87" (UID: "ea3e06d6-9ee4-4ecf-89fc-ef3451524f87"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:10:29.621086 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:29.621064 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea3e06d6-9ee4-4ecf-89fc-ef3451524f87-istio-data" (OuterVolumeSpecName: "istio-data") pod "ea3e06d6-9ee4-4ecf-89fc-ef3451524f87" (UID: "ea3e06d6-9ee4-4ecf-89fc-ef3451524f87"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:10:29.621317 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:29.621264 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea3e06d6-9ee4-4ecf-89fc-ef3451524f87-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ea3e06d6-9ee4-4ecf-89fc-ef3451524f87" (UID: "ea3e06d6-9ee4-4ecf-89fc-ef3451524f87"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 22:10:29.623040 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:29.623009 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea3e06d6-9ee4-4ecf-89fc-ef3451524f87-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ea3e06d6-9ee4-4ecf-89fc-ef3451524f87" (UID: "ea3e06d6-9ee4-4ecf-89fc-ef3451524f87"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:10:29.623171 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:29.623153 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea3e06d6-9ee4-4ecf-89fc-ef3451524f87-kube-api-access-smqrx" (OuterVolumeSpecName: "kube-api-access-smqrx") pod "ea3e06d6-9ee4-4ecf-89fc-ef3451524f87" (UID: "ea3e06d6-9ee4-4ecf-89fc-ef3451524f87"). InnerVolumeSpecName "kube-api-access-smqrx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:10:29.623235 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:29.623161 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ea3e06d6-9ee4-4ecf-89fc-ef3451524f87-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ea3e06d6-9ee4-4ecf-89fc-ef3451524f87" (UID: "ea3e06d6-9ee4-4ecf-89fc-ef3451524f87"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 22:10:29.623277 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:29.623243 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea3e06d6-9ee4-4ecf-89fc-ef3451524f87-istio-token" (OuterVolumeSpecName: "istio-token") pod "ea3e06d6-9ee4-4ecf-89fc-ef3451524f87" (UID: "ea3e06d6-9ee4-4ecf-89fc-ef3451524f87"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:10:29.720636 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:29.720603 2569 generic.go:356] "Generic (PLEG): container finished" podID="ea3e06d6-9ee4-4ecf-89fc-ef3451524f87" containerID="bb76892f8cb1b4622ed92ae5be4b0ef1db36ded5d92088f3d46dd39dd65d2992" exitCode=0 Jul 02 22:10:29.720782 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:29.720653 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc" Jul 02 22:10:29.720782 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:29.720684 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc" event={"ID":"ea3e06d6-9ee4-4ecf-89fc-ef3451524f87","Type":"ContainerDied","Data":"bb76892f8cb1b4622ed92ae5be4b0ef1db36ded5d92088f3d46dd39dd65d2992"} Jul 02 22:10:29.720782 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:29.720740 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc" event={"ID":"ea3e06d6-9ee4-4ecf-89fc-ef3451524f87","Type":"ContainerDied","Data":"86bda3cf075dc79d341621b4a5af9b6671b2ce2ea4e186f6a2f44337b7e5d0c0"} Jul 02 22:10:29.720782 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:29.720770 2569 scope.go:117] "RemoveContainer" containerID="bb76892f8cb1b4622ed92ae5be4b0ef1db36ded5d92088f3d46dd39dd65d2992" Jul 02 22:10:29.721314 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:29.721291 2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea3e06d6-9ee4-4ecf-89fc-ef3451524f87-istio-data\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:10:29.721314 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:29.721315 2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea3e06d6-9ee4-4ecf-89fc-ef3451524f87-istio-podinfo\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:10:29.721489 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:29.721330 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-smqrx\" (UniqueName: \"kubernetes.io/projected/ea3e06d6-9ee4-4ecf-89fc-ef3451524f87-kube-api-access-smqrx\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:10:29.721489 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:29.721345 2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea3e06d6-9ee4-4ecf-89fc-ef3451524f87-credential-socket\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:10:29.721489 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:29.721358 2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea3e06d6-9ee4-4ecf-89fc-ef3451524f87-istio-envoy\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:10:29.721489 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:29.721370 2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea3e06d6-9ee4-4ecf-89fc-ef3451524f87-istiod-ca-cert\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:10:29.721489 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:29.721382 2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea3e06d6-9ee4-4ecf-89fc-ef3451524f87-workload-socket\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:10:29.721489 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:29.721393 2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea3e06d6-9ee4-4ecf-89fc-ef3451524f87-istio-token\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:10:29.730197 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:29.730182 2569 scope.go:117] "RemoveContainer" containerID="d35575890a70807d2ede6b6c4462c618b2f3043e70e50a1d564570f733497d1f" Jul 02 22:10:29.738103 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:29.738085 2569 scope.go:117] "RemoveContainer" containerID="bb76892f8cb1b4622ed92ae5be4b0ef1db36ded5d92088f3d46dd39dd65d2992" Jul 02 22:10:29.738337 ip-10-0-139-75 kubenswrapper[2569]: E0702 22:10:29.738320 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb76892f8cb1b4622ed92ae5be4b0ef1db36ded5d92088f3d46dd39dd65d2992\": container with ID starting with bb76892f8cb1b4622ed92ae5be4b0ef1db36ded5d92088f3d46dd39dd65d2992 not found: ID does not exist" containerID="bb76892f8cb1b4622ed92ae5be4b0ef1db36ded5d92088f3d46dd39dd65d2992" Jul 02 22:10:29.738382 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:29.738347 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb76892f8cb1b4622ed92ae5be4b0ef1db36ded5d92088f3d46dd39dd65d2992"} err="failed to get container status \"bb76892f8cb1b4622ed92ae5be4b0ef1db36ded5d92088f3d46dd39dd65d2992\": rpc error: code = NotFound desc = could not find container \"bb76892f8cb1b4622ed92ae5be4b0ef1db36ded5d92088f3d46dd39dd65d2992\": container with ID starting with bb76892f8cb1b4622ed92ae5be4b0ef1db36ded5d92088f3d46dd39dd65d2992 not found: ID does not exist" Jul 02 22:10:29.738382 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:29.738364 2569 scope.go:117] "RemoveContainer" containerID="d35575890a70807d2ede6b6c4462c618b2f3043e70e50a1d564570f733497d1f" Jul 02 22:10:29.738601 ip-10-0-139-75 kubenswrapper[2569]: E0702 22:10:29.738580 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d35575890a70807d2ede6b6c4462c618b2f3043e70e50a1d564570f733497d1f\": container with ID starting with d35575890a70807d2ede6b6c4462c618b2f3043e70e50a1d564570f733497d1f not found: ID does not exist" containerID="d35575890a70807d2ede6b6c4462c618b2f3043e70e50a1d564570f733497d1f" Jul 02 22:10:29.738638 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:29.738607 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d35575890a70807d2ede6b6c4462c618b2f3043e70e50a1d564570f733497d1f"} err="failed to get container status \"d35575890a70807d2ede6b6c4462c618b2f3043e70e50a1d564570f733497d1f\": rpc error: code = NotFound desc = could not find container \"d35575890a70807d2ede6b6c4462c618b2f3043e70e50a1d564570f733497d1f\": container with ID starting with d35575890a70807d2ede6b6c4462c618b2f3043e70e50a1d564570f733497d1f not found: ID does not exist" Jul 02 22:10:29.742072 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:29.742052 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc"] Jul 02 22:10:29.751918 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:29.751899 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-906c71-openshift-default-7f68657dcc-jltxc"] Jul 02 22:10:30.957135 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:30.957104 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea3e06d6-9ee4-4ecf-89fc-ef3451524f87" path="/var/lib/kubelet/pods/ea3e06d6-9ee4-4ecf-89fc-ef3451524f87/volumes" Jul 02 22:10:33.572291 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:33.572263 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2"] Jul 02 22:10:33.572718 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:33.572705 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea3e06d6-9ee4-4ecf-89fc-ef3451524f87" containerName="istio-proxy" Jul 02 22:10:33.572762 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:33.572722 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea3e06d6-9ee4-4ecf-89fc-ef3451524f87" containerName="istio-proxy" Jul 02 22:10:33.572762 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:33.572747 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca911658-4c38-49bc-992f-d16fe85178d9" containerName="istio-proxy" Jul 02 22:10:33.572762 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:33.572753 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca911658-4c38-49bc-992f-d16fe85178d9" containerName="istio-proxy" Jul 02 22:10:33.572762 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:33.572759 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea3e06d6-9ee4-4ecf-89fc-ef3451524f87" containerName="istio-proxy" Jul 02 22:10:33.572966 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:33.572765 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea3e06d6-9ee4-4ecf-89fc-ef3451524f87" containerName="istio-proxy" Jul 02 22:10:33.572966 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:33.572851 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea3e06d6-9ee4-4ecf-89fc-ef3451524f87" containerName="istio-proxy" Jul 02 22:10:33.572966 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:33.572862 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca911658-4c38-49bc-992f-d16fe85178d9" containerName="istio-proxy" Jul 02 22:10:33.576321 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:33.576298 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2" Jul 02 22:10:33.582051 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:33.582027 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-f2a709-openshift-default-dockercfg-rbrzv\"" Jul 02 22:10:33.591717 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:33.591695 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2"] Jul 02 22:10:33.661330 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:33.661293 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7e5e7bc-71a3-4040-a657-ed17e8db15e3-workload-socket\") pod \"e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2\" (UID: \"e7e5e7bc-71a3-4040-a657-ed17e8db15e3\") " pod="openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2" Jul 02 22:10:33.661512 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:33.661346 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7e5e7bc-71a3-4040-a657-ed17e8db15e3-istiod-ca-cert\") pod \"e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2\" (UID: \"e7e5e7bc-71a3-4040-a657-ed17e8db15e3\") " pod="openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2" Jul 02 22:10:33.661512 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:33.661377 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7e5e7bc-71a3-4040-a657-ed17e8db15e3-workload-certs\") pod \"e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2\" (UID: \"e7e5e7bc-71a3-4040-a657-ed17e8db15e3\") " pod="openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2" Jul 02 22:10:33.661512 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:33.661437 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7e5e7bc-71a3-4040-a657-ed17e8db15e3-istio-podinfo\") pod \"e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2\" (UID: \"e7e5e7bc-71a3-4040-a657-ed17e8db15e3\") " pod="openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2" Jul 02 22:10:33.661512 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:33.661487 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7e5e7bc-71a3-4040-a657-ed17e8db15e3-istio-data\") pod \"e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2\" (UID: \"e7e5e7bc-71a3-4040-a657-ed17e8db15e3\") " pod="openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2" Jul 02 22:10:33.661737 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:33.661521 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s26bp\" (UniqueName: \"kubernetes.io/projected/e7e5e7bc-71a3-4040-a657-ed17e8db15e3-kube-api-access-s26bp\") pod \"e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2\" (UID: \"e7e5e7bc-71a3-4040-a657-ed17e8db15e3\") " pod="openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2" Jul 02 22:10:33.661737 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:33.661640 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7e5e7bc-71a3-4040-a657-ed17e8db15e3-istio-token\") pod \"e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2\" (UID: \"e7e5e7bc-71a3-4040-a657-ed17e8db15e3\") " pod="openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2" Jul 02 22:10:33.661737 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:33.661670 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7e5e7bc-71a3-4040-a657-ed17e8db15e3-credential-socket\") pod \"e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2\" (UID: \"e7e5e7bc-71a3-4040-a657-ed17e8db15e3\") " pod="openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2" Jul 02 22:10:33.661737 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:33.661723 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7e5e7bc-71a3-4040-a657-ed17e8db15e3-istio-envoy\") pod \"e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2\" (UID: \"e7e5e7bc-71a3-4040-a657-ed17e8db15e3\") " pod="openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2" Jul 02 22:10:33.762487 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:33.762450 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7e5e7bc-71a3-4040-a657-ed17e8db15e3-istiod-ca-cert\") pod \"e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2\" (UID: \"e7e5e7bc-71a3-4040-a657-ed17e8db15e3\") " pod="openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2" Jul 02 22:10:33.762487 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:33.762489 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7e5e7bc-71a3-4040-a657-ed17e8db15e3-workload-certs\") pod \"e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2\" (UID: \"e7e5e7bc-71a3-4040-a657-ed17e8db15e3\") " pod="openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2" Jul 02 22:10:33.762851 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:33.762511 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7e5e7bc-71a3-4040-a657-ed17e8db15e3-istio-podinfo\") pod \"e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2\" (UID: \"e7e5e7bc-71a3-4040-a657-ed17e8db15e3\") " pod="openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2" Jul 02 22:10:33.762851 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:33.762531 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7e5e7bc-71a3-4040-a657-ed17e8db15e3-istio-data\") pod \"e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2\" (UID: \"e7e5e7bc-71a3-4040-a657-ed17e8db15e3\") " pod="openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2" Jul 02 22:10:33.762851 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:33.762654 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s26bp\" (UniqueName: \"kubernetes.io/projected/e7e5e7bc-71a3-4040-a657-ed17e8db15e3-kube-api-access-s26bp\") pod \"e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2\" (UID: \"e7e5e7bc-71a3-4040-a657-ed17e8db15e3\") " pod="openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2" Jul 02 22:10:33.762851 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:33.762775 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7e5e7bc-71a3-4040-a657-ed17e8db15e3-istio-token\") pod \"e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2\" (UID: \"e7e5e7bc-71a3-4040-a657-ed17e8db15e3\") " pod="openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2" Jul 02 22:10:33.762851 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:33.762829 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7e5e7bc-71a3-4040-a657-ed17e8db15e3-credential-socket\") pod \"e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2\" (UID: \"e7e5e7bc-71a3-4040-a657-ed17e8db15e3\") " pod="openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2" Jul 02 22:10:33.763125 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:33.762869 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7e5e7bc-71a3-4040-a657-ed17e8db15e3-istio-envoy\") pod \"e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2\" (UID: \"e7e5e7bc-71a3-4040-a657-ed17e8db15e3\") " pod="openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2" Jul 02 22:10:33.763125 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:33.762932 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7e5e7bc-71a3-4040-a657-ed17e8db15e3-istio-data\") pod \"e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2\" (UID: \"e7e5e7bc-71a3-4040-a657-ed17e8db15e3\") " pod="openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2" Jul 02 22:10:33.763125 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:33.762951 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7e5e7bc-71a3-4040-a657-ed17e8db15e3-workload-socket\") pod \"e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2\" (UID: \"e7e5e7bc-71a3-4040-a657-ed17e8db15e3\") " pod="openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2" Jul 02 22:10:33.763125 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:33.762990 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7e5e7bc-71a3-4040-a657-ed17e8db15e3-workload-certs\") pod \"e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2\" (UID: \"e7e5e7bc-71a3-4040-a657-ed17e8db15e3\") " pod="openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2" Jul 02 22:10:33.763325 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:33.763175 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7e5e7bc-71a3-4040-a657-ed17e8db15e3-credential-socket\") pod \"e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2\" (UID: \"e7e5e7bc-71a3-4040-a657-ed17e8db15e3\") " pod="openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2" Jul 02 22:10:33.763325 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:33.763238 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7e5e7bc-71a3-4040-a657-ed17e8db15e3-workload-socket\") pod \"e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2\" (UID: \"e7e5e7bc-71a3-4040-a657-ed17e8db15e3\") " pod="openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2" Jul 02 22:10:33.763434 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:33.763319 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7e5e7bc-71a3-4040-a657-ed17e8db15e3-istiod-ca-cert\") pod \"e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2\" (UID: \"e7e5e7bc-71a3-4040-a657-ed17e8db15e3\") " pod="openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2" Jul 02 22:10:33.765104 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:33.765085 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7e5e7bc-71a3-4040-a657-ed17e8db15e3-istio-envoy\") pod \"e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2\" (UID: \"e7e5e7bc-71a3-4040-a657-ed17e8db15e3\") " pod="openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2" Jul 02 22:10:33.765429 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:33.765408 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7e5e7bc-71a3-4040-a657-ed17e8db15e3-istio-podinfo\") pod \"e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2\" (UID: \"e7e5e7bc-71a3-4040-a657-ed17e8db15e3\") " pod="openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2" Jul 02 22:10:33.771228 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:33.771206 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7e5e7bc-71a3-4040-a657-ed17e8db15e3-istio-token\") pod \"e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2\" (UID: \"e7e5e7bc-71a3-4040-a657-ed17e8db15e3\") " pod="openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2" Jul 02 22:10:33.771323 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:33.771262 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s26bp\" (UniqueName: \"kubernetes.io/projected/e7e5e7bc-71a3-4040-a657-ed17e8db15e3-kube-api-access-s26bp\") pod \"e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2\" (UID: \"e7e5e7bc-71a3-4040-a657-ed17e8db15e3\") " pod="openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2" Jul 02 22:10:33.887047 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:33.886964 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2" Jul 02 22:10:34.045289 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:34.045265 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2"] Jul 02 22:10:34.047452 ip-10-0-139-75 kubenswrapper[2569]: W0702 22:10:34.047427 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7e5e7bc_71a3_4040_a657_ed17e8db15e3.slice/crio-ae0e905e7b4c393549fb71aafcdb1db0c072cd29d54d8c7e65a724cbcdd1478b WatchSource:0}: Error finding container ae0e905e7b4c393549fb71aafcdb1db0c072cd29d54d8c7e65a724cbcdd1478b: Status 404 returned error can't find the container with id ae0e905e7b4c393549fb71aafcdb1db0c072cd29d54d8c7e65a724cbcdd1478b Jul 02 22:10:34.049959 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:34.049924 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Jul 02 22:10:34.050032 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:34.050001 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Jul 02 22:10:34.050073 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:34.050035 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Jul 02 22:10:34.750974 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:34.750939 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2" event={"ID":"e7e5e7bc-71a3-4040-a657-ed17e8db15e3","Type":"ContainerStarted","Data":"3d5d56648a1251bb9b1f109787e1a4b52a6f1c41651ed1845d96f030f957f02e"} Jul 02 22:10:34.751349 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:34.751000 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2" event={"ID":"e7e5e7bc-71a3-4040-a657-ed17e8db15e3","Type":"ContainerStarted","Data":"ae0e905e7b4c393549fb71aafcdb1db0c072cd29d54d8c7e65a724cbcdd1478b"} Jul 02 22:10:34.775319 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:34.775261 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2" podStartSLOduration=1.77524024 podStartE2EDuration="1.77524024s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:10:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:10:34.772521396 +0000 UTC m=+2074.111491832" watchObservedRunningTime="2026-07-02 22:10:34.77524024 +0000 UTC m=+2074.114210675" Jul 02 22:10:34.887981 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:34.887950 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2" Jul 02 22:10:35.892709 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:35.892682 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2" Jul 02 22:10:36.762056 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:36.762030 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2" Jul 02 22:10:36.763009 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:36.762986 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2" Jul 02 22:10:50.844133 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:50.844096 2569 generic.go:356] "Generic (PLEG): container finished" podID="e7e5e7bc-71a3-4040-a657-ed17e8db15e3" containerID="3d5d56648a1251bb9b1f109787e1a4b52a6f1c41651ed1845d96f030f957f02e" exitCode=0 Jul 02 22:10:50.844549 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:50.844171 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2" event={"ID":"e7e5e7bc-71a3-4040-a657-ed17e8db15e3","Type":"ContainerDied","Data":"3d5d56648a1251bb9b1f109787e1a4b52a6f1c41651ed1845d96f030f957f02e"} Jul 02 22:10:50.844591 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:50.844559 2569 scope.go:117] "RemoveContainer" containerID="3d5d56648a1251bb9b1f109787e1a4b52a6f1c41651ed1845d96f030f957f02e" Jul 02 22:10:50.845634 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:50.845602 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Jul 02 22:10:50.845748 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:50.845672 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Jul 02 22:10:50.845748 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:50.845703 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Jul 02 22:10:51.762056 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:51.762023 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2" Jul 02 22:10:51.851213 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:51.851174 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2" event={"ID":"e7e5e7bc-71a3-4040-a657-ed17e8db15e3","Type":"ContainerStarted","Data":"b8fe3d53935a84b05354f08e9911317f48b60179e25f7b63e6cbe98202c87263"} Jul 02 22:10:51.889596 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:51.889166 2569 patch_prober.go:28] interesting pod/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 02 22:10:51.889596 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:51.889234 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2" podUID="e7e5e7bc-71a3-4040-a657-ed17e8db15e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 02 22:10:51.889596 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:51.889530 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2" Jul 02 22:10:52.888076 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:52.888034 2569 patch_prober.go:28] interesting pod/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 02 22:10:52.888591 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:52.888112 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2" podUID="e7e5e7bc-71a3-4040-a657-ed17e8db15e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 02 22:10:53.887578 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:53.887528 2569 patch_prober.go:28] interesting pod/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 02 22:10:53.887817 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:53.887600 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2" podUID="e7e5e7bc-71a3-4040-a657-ed17e8db15e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 02 22:10:54.887584 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:54.887544 2569 patch_prober.go:28] interesting pod/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 02 22:10:54.888230 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:54.888197 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2" podUID="e7e5e7bc-71a3-4040-a657-ed17e8db15e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 02 22:10:55.887893 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:55.887848 2569 patch_prober.go:28] interesting pod/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 02 22:10:55.888416 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:55.887922 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2" podUID="e7e5e7bc-71a3-4040-a657-ed17e8db15e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 02 22:10:56.892380 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:56.892352 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2" Jul 02 22:10:56.892751 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:56.892668 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2" Jul 02 22:10:56.893409 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:10:56.893394 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2" Jul 02 22:17:09.087832 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:09.087779 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2"] Jul 02 22:17:09.088275 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:09.088021 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2" podUID="e7e5e7bc-71a3-4040-a657-ed17e8db15e3" containerName="istio-proxy" containerID="cri-o://b8fe3d53935a84b05354f08e9911317f48b60179e25f7b63e6cbe98202c87263" gracePeriod=30 Jul 02 22:17:11.894567 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:11.894529 2569 patch_prober.go:28] interesting pod/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 02 22:17:11.894951 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:11.894601 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2" podUID="e7e5e7bc-71a3-4040-a657-ed17e8db15e3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 22:17:14.352704 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:14.352644 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2" Jul 02 22:17:14.487494 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:14.487465 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-8hwkh_c970199b-3cae-49ac-a593-9b35bcad6095/manager/0.log" Jul 02 22:17:14.509064 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:14.509034 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7e5e7bc-71a3-4040-a657-ed17e8db15e3-istio-envoy\") pod \"e7e5e7bc-71a3-4040-a657-ed17e8db15e3\" (UID: \"e7e5e7bc-71a3-4040-a657-ed17e8db15e3\") " Jul 02 22:17:14.509199 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:14.509072 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7e5e7bc-71a3-4040-a657-ed17e8db15e3-istio-podinfo\") pod \"e7e5e7bc-71a3-4040-a657-ed17e8db15e3\" (UID: \"e7e5e7bc-71a3-4040-a657-ed17e8db15e3\") " Jul 02 22:17:14.509199 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:14.509109 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s26bp\" (UniqueName: \"kubernetes.io/projected/e7e5e7bc-71a3-4040-a657-ed17e8db15e3-kube-api-access-s26bp\") pod \"e7e5e7bc-71a3-4040-a657-ed17e8db15e3\" (UID: \"e7e5e7bc-71a3-4040-a657-ed17e8db15e3\") " Jul 02 22:17:14.509322 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:14.509227 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7e5e7bc-71a3-4040-a657-ed17e8db15e3-istio-token\") pod \"e7e5e7bc-71a3-4040-a657-ed17e8db15e3\" (UID: \"e7e5e7bc-71a3-4040-a657-ed17e8db15e3\") " Jul 02 22:17:14.509322 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:14.509267 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7e5e7bc-71a3-4040-a657-ed17e8db15e3-credential-socket\") pod \"e7e5e7bc-71a3-4040-a657-ed17e8db15e3\" (UID: \"e7e5e7bc-71a3-4040-a657-ed17e8db15e3\") " Jul 02 22:17:14.509322 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:14.509300 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7e5e7bc-71a3-4040-a657-ed17e8db15e3-istiod-ca-cert\") pod \"e7e5e7bc-71a3-4040-a657-ed17e8db15e3\" (UID: \"e7e5e7bc-71a3-4040-a657-ed17e8db15e3\") " Jul 02 22:17:14.509473 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:14.509387 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7e5e7bc-71a3-4040-a657-ed17e8db15e3-workload-certs\") pod \"e7e5e7bc-71a3-4040-a657-ed17e8db15e3\" (UID: \"e7e5e7bc-71a3-4040-a657-ed17e8db15e3\") " Jul 02 22:17:14.509473 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:14.509441 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7e5e7bc-71a3-4040-a657-ed17e8db15e3-workload-socket\") pod \"e7e5e7bc-71a3-4040-a657-ed17e8db15e3\" (UID: \"e7e5e7bc-71a3-4040-a657-ed17e8db15e3\") " Jul 02 22:17:14.509578 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:14.509483 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7e5e7bc-71a3-4040-a657-ed17e8db15e3-istio-data\") pod \"e7e5e7bc-71a3-4040-a657-ed17e8db15e3\" (UID: \"e7e5e7bc-71a3-4040-a657-ed17e8db15e3\") " Jul 02 22:17:14.509578 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:14.509481 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7e5e7bc-71a3-4040-a657-ed17e8db15e3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e7e5e7bc-71a3-4040-a657-ed17e8db15e3" (UID: "e7e5e7bc-71a3-4040-a657-ed17e8db15e3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:17:14.509872 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:14.509841 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7e5e7bc-71a3-4040-a657-ed17e8db15e3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e7e5e7bc-71a3-4040-a657-ed17e8db15e3" (UID: "e7e5e7bc-71a3-4040-a657-ed17e8db15e3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 22:17:14.509963 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:14.509948 2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7e5e7bc-71a3-4040-a657-ed17e8db15e3-credential-socket\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:17:14.510027 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:14.509968 2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7e5e7bc-71a3-4040-a657-ed17e8db15e3-istiod-ca-cert\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:17:14.510087 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:14.510027 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7e5e7bc-71a3-4040-a657-ed17e8db15e3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e7e5e7bc-71a3-4040-a657-ed17e8db15e3" (UID: "e7e5e7bc-71a3-4040-a657-ed17e8db15e3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:17:14.510156 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:14.510138 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7e5e7bc-71a3-4040-a657-ed17e8db15e3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e7e5e7bc-71a3-4040-a657-ed17e8db15e3" (UID: "e7e5e7bc-71a3-4040-a657-ed17e8db15e3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:17:14.510290 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:14.510271 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7e5e7bc-71a3-4040-a657-ed17e8db15e3-istio-data" (OuterVolumeSpecName: "istio-data") pod "e7e5e7bc-71a3-4040-a657-ed17e8db15e3" (UID: "e7e5e7bc-71a3-4040-a657-ed17e8db15e3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:17:14.511395 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:14.511369 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7e5e7bc-71a3-4040-a657-ed17e8db15e3-istio-token" (OuterVolumeSpecName: "istio-token") pod "e7e5e7bc-71a3-4040-a657-ed17e8db15e3" (UID: "e7e5e7bc-71a3-4040-a657-ed17e8db15e3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:17:14.511517 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:14.511416 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7e5e7bc-71a3-4040-a657-ed17e8db15e3-kube-api-access-s26bp" (OuterVolumeSpecName: "kube-api-access-s26bp") pod "e7e5e7bc-71a3-4040-a657-ed17e8db15e3" (UID: "e7e5e7bc-71a3-4040-a657-ed17e8db15e3"). InnerVolumeSpecName "kube-api-access-s26bp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:17:14.511650 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:14.511635 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7e5e7bc-71a3-4040-a657-ed17e8db15e3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e7e5e7bc-71a3-4040-a657-ed17e8db15e3" (UID: "e7e5e7bc-71a3-4040-a657-ed17e8db15e3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:17:14.511712 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:14.511639 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e7e5e7bc-71a3-4040-a657-ed17e8db15e3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e7e5e7bc-71a3-4040-a657-ed17e8db15e3" (UID: "e7e5e7bc-71a3-4040-a657-ed17e8db15e3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 22:17:14.610734 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:14.610662 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s26bp\" (UniqueName: \"kubernetes.io/projected/e7e5e7bc-71a3-4040-a657-ed17e8db15e3-kube-api-access-s26bp\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:17:14.610734 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:14.610687 2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7e5e7bc-71a3-4040-a657-ed17e8db15e3-istio-token\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:17:14.610734 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:14.610698 2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7e5e7bc-71a3-4040-a657-ed17e8db15e3-workload-certs\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:17:14.610734 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:14.610706 2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7e5e7bc-71a3-4040-a657-ed17e8db15e3-workload-socket\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:17:14.610734 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:14.610714 2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7e5e7bc-71a3-4040-a657-ed17e8db15e3-istio-data\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:17:14.610734 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:14.610724 2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7e5e7bc-71a3-4040-a657-ed17e8db15e3-istio-envoy\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:17:14.610734 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:14.610731 2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7e5e7bc-71a3-4040-a657-ed17e8db15e3-istio-podinfo\") on node \"ip-10-0-139-75.ec2.internal\" DevicePath \"\"" Jul 02 22:17:14.618521 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:14.618497 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-8gxdc_8e629e4a-cfed-4f2f-aeeb-10ffc9116325/manager/0.log" Jul 02 22:17:14.868486 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:14.868403 2569 generic.go:356] "Generic (PLEG): container finished" podID="e7e5e7bc-71a3-4040-a657-ed17e8db15e3" containerID="b8fe3d53935a84b05354f08e9911317f48b60179e25f7b63e6cbe98202c87263" exitCode=0 Jul 02 22:17:14.868486 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:14.868452 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2" Jul 02 22:17:14.868667 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:14.868488 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2" event={"ID":"e7e5e7bc-71a3-4040-a657-ed17e8db15e3","Type":"ContainerDied","Data":"b8fe3d53935a84b05354f08e9911317f48b60179e25f7b63e6cbe98202c87263"} Jul 02 22:17:14.868667 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:14.868539 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2" event={"ID":"e7e5e7bc-71a3-4040-a657-ed17e8db15e3","Type":"ContainerDied","Data":"ae0e905e7b4c393549fb71aafcdb1db0c072cd29d54d8c7e65a724cbcdd1478b"} Jul 02 22:17:14.868667 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:14.868564 2569 scope.go:117] "RemoveContainer" containerID="b8fe3d53935a84b05354f08e9911317f48b60179e25f7b63e6cbe98202c87263" Jul 02 22:17:14.878334 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:14.878312 2569 scope.go:117] "RemoveContainer" containerID="3d5d56648a1251bb9b1f109787e1a4b52a6f1c41651ed1845d96f030f957f02e" Jul 02 22:17:14.886617 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:14.886601 2569 scope.go:117] "RemoveContainer" containerID="b8fe3d53935a84b05354f08e9911317f48b60179e25f7b63e6cbe98202c87263" Jul 02 22:17:14.886897 ip-10-0-139-75 kubenswrapper[2569]: E0702 22:17:14.886879 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8fe3d53935a84b05354f08e9911317f48b60179e25f7b63e6cbe98202c87263\": container with ID starting with b8fe3d53935a84b05354f08e9911317f48b60179e25f7b63e6cbe98202c87263 not found: ID does not exist" containerID="b8fe3d53935a84b05354f08e9911317f48b60179e25f7b63e6cbe98202c87263" Jul 02 22:17:14.886964 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:14.886905 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8fe3d53935a84b05354f08e9911317f48b60179e25f7b63e6cbe98202c87263"} err="failed to get container status \"b8fe3d53935a84b05354f08e9911317f48b60179e25f7b63e6cbe98202c87263\": rpc error: code = NotFound desc = could not find container \"b8fe3d53935a84b05354f08e9911317f48b60179e25f7b63e6cbe98202c87263\": container with ID starting with b8fe3d53935a84b05354f08e9911317f48b60179e25f7b63e6cbe98202c87263 not found: ID does not exist" Jul 02 22:17:14.886964 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:14.886924 2569 scope.go:117] "RemoveContainer" containerID="3d5d56648a1251bb9b1f109787e1a4b52a6f1c41651ed1845d96f030f957f02e" Jul 02 22:17:14.887158 ip-10-0-139-75 kubenswrapper[2569]: E0702 22:17:14.887141 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d5d56648a1251bb9b1f109787e1a4b52a6f1c41651ed1845d96f030f957f02e\": container with ID starting with 3d5d56648a1251bb9b1f109787e1a4b52a6f1c41651ed1845d96f030f957f02e not found: ID does not exist" containerID="3d5d56648a1251bb9b1f109787e1a4b52a6f1c41651ed1845d96f030f957f02e" Jul 02 22:17:14.887199 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:14.887164 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d5d56648a1251bb9b1f109787e1a4b52a6f1c41651ed1845d96f030f957f02e"} err="failed to get container status \"3d5d56648a1251bb9b1f109787e1a4b52a6f1c41651ed1845d96f030f957f02e\": rpc error: code = NotFound desc = could not find container \"3d5d56648a1251bb9b1f109787e1a4b52a6f1c41651ed1845d96f030f957f02e\": container with ID starting with 3d5d56648a1251bb9b1f109787e1a4b52a6f1c41651ed1845d96f030f957f02e not found: ID does not exist" Jul 02 22:17:14.892361 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:14.892338 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2"] Jul 02 22:17:14.897294 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:14.897274 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-f2a709-openshift-default-5d6f8589d7-2b9q2"] Jul 02 22:17:14.956001 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:14.955978 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7e5e7bc-71a3-4040-a657-ed17e8db15e3" path="/var/lib/kubelet/pods/e7e5e7bc-71a3-4040-a657-ed17e8db15e3/volumes" Jul 02 22:17:15.187273 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:15.187187 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-94g9b_d61dd72b-1715-411c-affd-b4bff8c76012/server/0.log" Jul 02 22:17:15.304149 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:15.304110 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-qsvm7_9abe791f-664b-45a5-ab2f-b6aa86f63503/manager/0.log" Jul 02 22:17:15.710090 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:15.710061 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bcc4b86cc-8md8v_8fa1f2a1-ceb0-4696-928b-bc09a5a4b5b6/copy-manifests/0.log" Jul 02 22:17:15.717518 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:15.717476 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bcc4b86cc-8md8v_8fa1f2a1-ceb0-4696-928b-bc09a5a4b5b6/manager/0.log" Jul 02 22:17:15.826817 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:15.826772 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-6khkg_34d3904e-29e4-413f-9f5f-8a259c6d2279/postgres/0.log" Jul 02 22:17:17.281375 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:17.281317 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-2dl5k_d9dd6b60-ceea-4092-93b6-98845a1d3974/manager/0.log" Jul 02 22:17:17.403166 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:17.403110 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-fj4xx_8281e5ad-c64f-4203-826c-4f081a7a2267/manager/0.log" Jul 02 22:17:17.638557 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:17.638464 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-hqzzt_957557fc-7241-4c3d-8c56-88abb6b4139d/registry-server/0.log" Jul 02 22:17:17.766040 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:17.766009 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-rgb8z_13dad243-1dd3-4e64-a799-15e96c4caadf/manager/0.log" Jul 02 22:17:18.066479 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:18.066452 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-vsgpl_5315e1f1-17e9-4f07-a961-f04ef2d00e64/manager/0.log" Jul 02 22:17:18.414393 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:18.414271 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4h7mzx_2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7/istio-proxy/0.log" Jul 02 22:17:18.789480 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:18.789447 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8697fd7f55-c7p8v_5515c578-d7e1-4942-bc9a-722589fa6a30/kube-auth-proxy/0.log" Jul 02 22:17:18.905270 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:18.905241 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-5tjmq_2749fe4b-86bd-4342-863c-ab4e0bd8bd24/istio-proxy/1.log" Jul 02 22:17:19.143114 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:19.143027 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-jqgtg_0fe11bea-2cf2-4c63-943a-3edd673560af/payload-processing/0.log" Jul 02 22:17:19.624055 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:19.624030 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-dgchj_dd2bf3ef-4f49-46cf-87cf-ee102b6d8fe5/main/0.log" Jul 02 22:17:19.860558 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:19.860529 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-727mw_0ad662ca-f02d-4364-8486-b462220ed2dc/main/0.log" Jul 02 22:17:20.100316 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:20.100286 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-9clz6_4a374984-8c70-4e94-b998-7882bf32e372/main/0.log" Jul 02 22:17:27.365403 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:27.365373 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-j9flr_d95b88bb-30e2-421a-84a7-96de7cd5d1e1/global-pull-secret-syncer/0.log" Jul 02 22:17:27.451405 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:27.451374 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-mw4n5_dc5ec2c9-c5d2-4a0f-94ee-971c99c93d72/konnectivity-agent/0.log" Jul 02 22:17:27.540946 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:27.540922 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-75.ec2.internal_93302d7a5c815ce5fd5abf7c49c032a4/haproxy/0.log" Jul 02 22:17:32.939129 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:32.939089 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-2dl5k_d9dd6b60-ceea-4092-93b6-98845a1d3974/manager/0.log" Jul 02 22:17:32.964582 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:32.964555 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-fj4xx_8281e5ad-c64f-4203-826c-4f081a7a2267/manager/0.log" Jul 02 22:17:33.044439 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:33.044405 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-hqzzt_957557fc-7241-4c3d-8c56-88abb6b4139d/registry-server/0.log" Jul 02 22:17:33.166563 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:33.166532 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-rgb8z_13dad243-1dd3-4e64-a799-15e96c4caadf/manager/0.log" Jul 02 22:17:33.261169 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:33.261128 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-vsgpl_5315e1f1-17e9-4f07-a961-f04ef2d00e64/manager/0.log" Jul 02 22:17:35.453401 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:35.453322 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-z69cz_cc6da78f-0511-4162-831c-4a433aaef2a6/monitoring-plugin/0.log" Jul 02 22:17:35.620939 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:35.620907 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-c8jks_f9f5712b-72f4-4536-b1fe-2a4240d463f6/node-exporter/0.log" Jul 02 22:17:35.665651 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:35.665595 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-c8jks_f9f5712b-72f4-4536-b1fe-2a4240d463f6/kube-rbac-proxy/0.log" Jul 02 22:17:35.719532 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:35.719462 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-c8jks_f9f5712b-72f4-4536-b1fe-2a4240d463f6/init-textfile/0.log" Jul 02 22:17:35.948753 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:35.948725 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-g6qqq_739024f7-d272-4932-b23d-5d815ade3282/kube-rbac-proxy-main/0.log" Jul 02 22:17:35.971624 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:35.971547 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-g6qqq_739024f7-d272-4932-b23d-5d815ade3282/kube-rbac-proxy-self/0.log" Jul 02 22:17:35.996099 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:35.996079 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-g6qqq_739024f7-d272-4932-b23d-5d815ade3282/openshift-state-metrics/0.log" Jul 02 22:17:36.033464 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:36.033439 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_066af47b-7dcb-49e6-b963-4cf7576bce1f/prometheus/0.log" Jul 02 22:17:36.081610 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:36.081586 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_066af47b-7dcb-49e6-b963-4cf7576bce1f/config-reloader/0.log" Jul 02 22:17:36.131727 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:36.131704 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_066af47b-7dcb-49e6-b963-4cf7576bce1f/thanos-sidecar/0.log" Jul 02 22:17:36.183061 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:36.183015 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_066af47b-7dcb-49e6-b963-4cf7576bce1f/kube-rbac-proxy-web/0.log" Jul 02 22:17:36.212909 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:36.212890 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_066af47b-7dcb-49e6-b963-4cf7576bce1f/kube-rbac-proxy/0.log" Jul 02 22:17:36.235995 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:36.235934 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_066af47b-7dcb-49e6-b963-4cf7576bce1f/kube-rbac-proxy-thanos/0.log" Jul 02 22:17:36.257701 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:36.257684 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_066af47b-7dcb-49e6-b963-4cf7576bce1f/init-config-reloader/0.log" Jul 02 22:17:36.287223 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:36.287199 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-l9rhw_efe39bd5-6e24-463f-af81-2f9beab85fe6/prometheus-operator/0.log" Jul 02 22:17:36.308910 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:36.308891 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-l9rhw_efe39bd5-6e24-463f-af81-2f9beab85fe6/kube-rbac-proxy/0.log" Jul 02 22:17:36.337653 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:36.337631 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-rzs97_8742b41f-9660-4b06-be1b-957b839a327b/prometheus-operator-admission-webhook/0.log" Jul 02 22:17:36.462701 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:36.462675 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6fd7d9ffcd-njp9f_57579b3f-0f25-49af-b810-64b3b0e3d3d1/thanos-query/0.log" Jul 02 22:17:36.484899 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:36.484879 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6fd7d9ffcd-njp9f_57579b3f-0f25-49af-b810-64b3b0e3d3d1/kube-rbac-proxy-web/0.log" Jul 02 22:17:36.509569 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:36.509551 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6fd7d9ffcd-njp9f_57579b3f-0f25-49af-b810-64b3b0e3d3d1/kube-rbac-proxy/0.log" Jul 02 22:17:36.532591 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:36.532565 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6fd7d9ffcd-njp9f_57579b3f-0f25-49af-b810-64b3b0e3d3d1/prom-label-proxy/0.log" Jul 02 22:17:36.556824 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:36.556806 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6fd7d9ffcd-njp9f_57579b3f-0f25-49af-b810-64b3b0e3d3d1/kube-rbac-proxy-rules/0.log" Jul 02 22:17:36.579177 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:36.579156 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6fd7d9ffcd-njp9f_57579b3f-0f25-49af-b810-64b3b0e3d3d1/kube-rbac-proxy-metrics/0.log" Jul 02 22:17:37.080873 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:37.080843 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-h24vs/perf-node-gather-daemonset-krnhz"] Jul 02 22:17:37.081369 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:37.081348 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7e5e7bc-71a3-4040-a657-ed17e8db15e3" containerName="istio-proxy" Jul 02 22:17:37.081469 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:37.081371 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7e5e7bc-71a3-4040-a657-ed17e8db15e3" containerName="istio-proxy" Jul 02 22:17:37.081524 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:37.081478 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea3e06d6-9ee4-4ecf-89fc-ef3451524f87" containerName="istio-proxy" Jul 02 22:17:37.081524 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:37.081494 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7e5e7bc-71a3-4040-a657-ed17e8db15e3" containerName="istio-proxy" Jul 02 22:17:37.081524 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:37.081506 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7e5e7bc-71a3-4040-a657-ed17e8db15e3" containerName="istio-proxy" Jul 02 22:17:37.085218 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:37.085196 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-h24vs/perf-node-gather-daemonset-krnhz" Jul 02 22:17:37.087974 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:37.087957 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-h24vs\"/\"openshift-service-ca.crt\"" Jul 02 22:17:37.088072 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:37.087971 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-h24vs\"/\"default-dockercfg-6wgd7\"" Jul 02 22:17:37.088823 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:37.088808 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-h24vs\"/\"kube-root-ca.crt\"" Jul 02 22:17:37.095077 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:37.095053 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-h24vs/perf-node-gather-daemonset-krnhz"] Jul 02 22:17:37.124901 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:37.124869 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/15777683-f9c9-4b27-859e-feb3dc3c2f06-lib-modules\") pod \"perf-node-gather-daemonset-krnhz\" (UID: \"15777683-f9c9-4b27-859e-feb3dc3c2f06\") " pod="openshift-must-gather-h24vs/perf-node-gather-daemonset-krnhz" Jul 02 22:17:37.125063 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:37.124908 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/15777683-f9c9-4b27-859e-feb3dc3c2f06-sys\") pod \"perf-node-gather-daemonset-krnhz\" (UID: \"15777683-f9c9-4b27-859e-feb3dc3c2f06\") " pod="openshift-must-gather-h24vs/perf-node-gather-daemonset-krnhz" Jul 02 22:17:37.125063 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:37.125024 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpvzx\" (UniqueName: \"kubernetes.io/projected/15777683-f9c9-4b27-859e-feb3dc3c2f06-kube-api-access-hpvzx\") pod \"perf-node-gather-daemonset-krnhz\" (UID: \"15777683-f9c9-4b27-859e-feb3dc3c2f06\") " pod="openshift-must-gather-h24vs/perf-node-gather-daemonset-krnhz" Jul 02 22:17:37.125134 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:37.125091 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/15777683-f9c9-4b27-859e-feb3dc3c2f06-proc\") pod \"perf-node-gather-daemonset-krnhz\" (UID: \"15777683-f9c9-4b27-859e-feb3dc3c2f06\") " pod="openshift-must-gather-h24vs/perf-node-gather-daemonset-krnhz" Jul 02 22:17:37.125134 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:37.125130 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/15777683-f9c9-4b27-859e-feb3dc3c2f06-podres\") pod \"perf-node-gather-daemonset-krnhz\" (UID: \"15777683-f9c9-4b27-859e-feb3dc3c2f06\") " pod="openshift-must-gather-h24vs/perf-node-gather-daemonset-krnhz" Jul 02 22:17:37.226230 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:37.226178 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/15777683-f9c9-4b27-859e-feb3dc3c2f06-proc\") pod \"perf-node-gather-daemonset-krnhz\" (UID: \"15777683-f9c9-4b27-859e-feb3dc3c2f06\") " pod="openshift-must-gather-h24vs/perf-node-gather-daemonset-krnhz" Jul 02 22:17:37.226398 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:37.226259 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/15777683-f9c9-4b27-859e-feb3dc3c2f06-podres\") pod \"perf-node-gather-daemonset-krnhz\" (UID: \"15777683-f9c9-4b27-859e-feb3dc3c2f06\") " pod="openshift-must-gather-h24vs/perf-node-gather-daemonset-krnhz" Jul 02 22:17:37.226398 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:37.226294 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/15777683-f9c9-4b27-859e-feb3dc3c2f06-lib-modules\") pod \"perf-node-gather-daemonset-krnhz\" (UID: \"15777683-f9c9-4b27-859e-feb3dc3c2f06\") " pod="openshift-must-gather-h24vs/perf-node-gather-daemonset-krnhz" Jul 02 22:17:37.226398 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:37.226306 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/15777683-f9c9-4b27-859e-feb3dc3c2f06-proc\") pod \"perf-node-gather-daemonset-krnhz\" (UID: \"15777683-f9c9-4b27-859e-feb3dc3c2f06\") " pod="openshift-must-gather-h24vs/perf-node-gather-daemonset-krnhz" Jul 02 22:17:37.226398 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:37.226320 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/15777683-f9c9-4b27-859e-feb3dc3c2f06-sys\") pod \"perf-node-gather-daemonset-krnhz\" (UID: \"15777683-f9c9-4b27-859e-feb3dc3c2f06\") " pod="openshift-must-gather-h24vs/perf-node-gather-daemonset-krnhz" Jul 02 22:17:37.226398 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:37.226373 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/15777683-f9c9-4b27-859e-feb3dc3c2f06-sys\") pod \"perf-node-gather-daemonset-krnhz\" (UID: \"15777683-f9c9-4b27-859e-feb3dc3c2f06\") " pod="openshift-must-gather-h24vs/perf-node-gather-daemonset-krnhz" Jul 02 22:17:37.226603 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:37.226432 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpvzx\" (UniqueName: \"kubernetes.io/projected/15777683-f9c9-4b27-859e-feb3dc3c2f06-kube-api-access-hpvzx\") pod \"perf-node-gather-daemonset-krnhz\" (UID: \"15777683-f9c9-4b27-859e-feb3dc3c2f06\") " pod="openshift-must-gather-h24vs/perf-node-gather-daemonset-krnhz" Jul 02 22:17:37.226603 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:37.226443 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/15777683-f9c9-4b27-859e-feb3dc3c2f06-lib-modules\") pod \"perf-node-gather-daemonset-krnhz\" (UID: \"15777683-f9c9-4b27-859e-feb3dc3c2f06\") " pod="openshift-must-gather-h24vs/perf-node-gather-daemonset-krnhz" Jul 02 22:17:37.226603 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:37.226433 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/15777683-f9c9-4b27-859e-feb3dc3c2f06-podres\") pod \"perf-node-gather-daemonset-krnhz\" (UID: \"15777683-f9c9-4b27-859e-feb3dc3c2f06\") " pod="openshift-must-gather-h24vs/perf-node-gather-daemonset-krnhz" Jul 02 22:17:37.235339 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:37.235308 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpvzx\" (UniqueName: \"kubernetes.io/projected/15777683-f9c9-4b27-859e-feb3dc3c2f06-kube-api-access-hpvzx\") pod \"perf-node-gather-daemonset-krnhz\" (UID: \"15777683-f9c9-4b27-859e-feb3dc3c2f06\") " pod="openshift-must-gather-h24vs/perf-node-gather-daemonset-krnhz" Jul 02 22:17:37.396339 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:37.396245 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-h24vs/perf-node-gather-daemonset-krnhz" Jul 02 22:17:37.535254 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:37.535221 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-h24vs/perf-node-gather-daemonset-krnhz"] Jul 02 22:17:37.536704 ip-10-0-139-75 kubenswrapper[2569]: W0702 22:17:37.536668 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod15777683_f9c9_4b27_859e_feb3dc3c2f06.slice/crio-9d2a067a3550cacb59b98a7d4903820bf3382de25c077290e4bed7c4d4c5195c WatchSource:0}: Error finding container 9d2a067a3550cacb59b98a7d4903820bf3382de25c077290e4bed7c4d4c5195c: Status 404 returned error can't find the container with id 9d2a067a3550cacb59b98a7d4903820bf3382de25c077290e4bed7c4d4c5195c Jul 02 22:17:37.746764 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:37.746736 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-rzzw8_af3c0081-d6c4-418c-8cfc-84b0248778f1/networking-console-plugin/0.log" Jul 02 22:17:37.985021 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:37.984935 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-h24vs/perf-node-gather-daemonset-krnhz" event={"ID":"15777683-f9c9-4b27-859e-feb3dc3c2f06","Type":"ContainerStarted","Data":"e4d90fc6b89c2813e03233425f5c1537b8ea4b3346e8436f06ffacc9442dc6cf"} Jul 02 22:17:37.985021 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:37.985000 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-h24vs/perf-node-gather-daemonset-krnhz" Jul 02 22:17:37.985021 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:37.985018 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-h24vs/perf-node-gather-daemonset-krnhz" event={"ID":"15777683-f9c9-4b27-859e-feb3dc3c2f06","Type":"ContainerStarted","Data":"9d2a067a3550cacb59b98a7d4903820bf3382de25c077290e4bed7c4d4c5195c"} Jul 02 22:17:38.007495 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:38.007441 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-h24vs/perf-node-gather-daemonset-krnhz" podStartSLOduration=1.007425166 podStartE2EDuration="1.007425166s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 22:17:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:17:38.005542988 +0000 UTC m=+2497.344513436" watchObservedRunningTime="2026-07-02 22:17:38.007425166 +0000 UTC m=+2497.346395626" Jul 02 22:17:38.764837 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:38.764807 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-ftvjj_cc6ebdd7-4265-45cc-86d2-b36a76ffe2c0/download-server/0.log" Jul 02 22:17:40.430938 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:40.430911 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fqj8h_7337a38e-9618-4a34-b207-44dc712dd12b/dns/0.log" Jul 02 22:17:40.452328 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:40.452304 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fqj8h_7337a38e-9618-4a34-b207-44dc712dd12b/kube-rbac-proxy/0.log" Jul 02 22:17:40.600992 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:40.600968 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bf9dd_1ed8e72b-e89d-42d5-94f7-3c00d5bb1ca9/dns-node-resolver/0.log" Jul 02 22:17:41.190929 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:41.190898 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-f788c7bf6-4fwb6_1adcf74b-7adb-4c3e-859d-2481e729caf1/registry/0.log" Jul 02 22:17:41.227624 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:41.227599 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4rgsb_283e60d2-fc01-4709-afeb-73699b0d8134/node-ca/0.log" Jul 02 22:17:42.297533 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:42.297497 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4h7mzx_2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7/istio-proxy/0.log" Jul 02 22:17:42.461629 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:42.461599 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8697fd7f55-c7p8v_5515c578-d7e1-4942-bc9a-722589fa6a30/kube-auth-proxy/0.log" Jul 02 22:17:42.495281 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:42.495258 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-5tjmq_2749fe4b-86bd-4342-863c-ab4e0bd8bd24/istio-proxy/0.log" Jul 02 22:17:42.520142 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:42.520119 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-5tjmq_2749fe4b-86bd-4342-863c-ab4e0bd8bd24/istio-proxy/1.log" Jul 02 22:17:42.612194 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:42.612118 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-jqgtg_0fe11bea-2cf2-4c63-943a-3edd673560af/payload-processing/0.log" Jul 02 22:17:43.372231 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:43.372201 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rw2zb_3d916985-dd9a-4407-8ed4-e302a9670f50/serve-healthcheck-canary/0.log" Jul 02 22:17:44.000112 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:44.000087 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-h24vs/perf-node-gather-daemonset-krnhz" Jul 02 22:17:44.004267 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:44.004246 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fsdjr_f9e06bbd-441d-43a5-afdd-48eb709a5ecd/kube-rbac-proxy/0.log" Jul 02 22:17:44.029683 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:44.029664 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fsdjr_f9e06bbd-441d-43a5-afdd-48eb709a5ecd/exporter/0.log" Jul 02 22:17:44.053030 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:44.053009 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fsdjr_f9e06bbd-441d-43a5-afdd-48eb709a5ecd/extractor/0.log" Jul 02 22:17:46.599970 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:46.599935 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-8hwkh_c970199b-3cae-49ac-a593-9b35bcad6095/manager/0.log" Jul 02 22:17:46.730021 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:46.729991 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-8gxdc_8e629e4a-cfed-4f2f-aeeb-10ffc9116325/manager/0.log" Jul 02 22:17:46.990076 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:46.989995 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-94g9b_d61dd72b-1715-411c-affd-b4bff8c76012/server/0.log" Jul 02 22:17:47.056990 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:47.056961 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-qsvm7_9abe791f-664b-45a5-ab2f-b6aa86f63503/manager/0.log" Jul 02 22:17:47.255765 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:47.255742 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bcc4b86cc-8md8v_8fa1f2a1-ceb0-4696-928b-bc09a5a4b5b6/manager/0.log" Jul 02 22:17:47.280759 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:47.280730 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bcc4b86cc-8md8v_8fa1f2a1-ceb0-4696-928b-bc09a5a4b5b6/copy-manifests/0.log" Jul 02 22:17:47.305832 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:47.305806 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-6khkg_34d3904e-29e4-413f-9f5f-8a259c6d2279/postgres/0.log" Jul 02 22:17:47.490549 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:47.490512 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4h7mzx_2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7/istio-proxy/0.log" Jul 02 22:17:47.620190 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:47.620113 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8697fd7f55-c7p8v_5515c578-d7e1-4942-bc9a-722589fa6a30/kube-auth-proxy/0.log" Jul 02 22:17:47.639123 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:47.639098 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-5tjmq_2749fe4b-86bd-4342-863c-ab4e0bd8bd24/istio-proxy/0.log" Jul 02 22:17:47.666906 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:47.666874 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-5tjmq_2749fe4b-86bd-4342-863c-ab4e0bd8bd24/istio-proxy/1.log" Jul 02 22:17:47.738636 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:47.738610 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-jqgtg_0fe11bea-2cf2-4c63-943a-3edd673560af/payload-processing/0.log" Jul 02 22:17:48.805015 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:48.804984 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-jvmm7_c8efb66e-8222-4daf-9fc9-6b41de55c4b3/openshift-lws-operator/0.log" Jul 02 22:17:49.015497 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:49.015469 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4h7mzx_2c11f1ef-e705-438a-b5d2-0ac53d5aa4e7/istio-proxy/0.log" Jul 02 22:17:49.155201 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:49.155174 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8697fd7f55-c7p8v_5515c578-d7e1-4942-bc9a-722589fa6a30/kube-auth-proxy/0.log" Jul 02 22:17:49.178553 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:49.178525 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-5tjmq_2749fe4b-86bd-4342-863c-ab4e0bd8bd24/istio-proxy/0.log" Jul 02 22:17:49.204563 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:49.204535 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-5tjmq_2749fe4b-86bd-4342-863c-ab4e0bd8bd24/istio-proxy/1.log" Jul 02 22:17:49.283356 ip-10-0-139-75 kubenswrapper[2569]: I0702 22:17:49.283300 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-jqgtg_0fe11bea-2cf2-4c63-943a-3edd673560af/payload-processing/0.log"