Jul 19 16:36:22.909751 ip-10-0-134-203 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 19 16:36:22.909766 ip-10-0-134-203 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 19 16:36:22.909775 ip-10-0-134-203 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 19 16:36:22.910091 ip-10-0-134-203 systemd[1]: Failed to start Kubernetes Kubelet. Jul 19 16:36:33.081727 ip-10-0-134-203 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 19 16:36:33.081749 ip-10-0-134-203 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot dac34812851141cea80cd3ca5851425c -- Jul 19 16:38:24.894342 ip-10-0-134-203 systemd[1]: Starting Kubernetes Kubelet... Jul 19 16:38:25.331958 ip-10-0-134-203 kubenswrapper[2583]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 19 16:38:25.331958 ip-10-0-134-203 kubenswrapper[2583]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 19 16:38:25.331958 ip-10-0-134-203 kubenswrapper[2583]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 19 16:38:25.331958 ip-10-0-134-203 kubenswrapper[2583]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 19 16:38:25.331958 ip-10-0-134-203 kubenswrapper[2583]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 19 16:38:25.331958 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.294632 2583 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 19 16:38:25.331958 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299091 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 19 16:38:25.331958 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299103 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 19 16:38:25.331958 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299106 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 19 16:38:25.331958 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299109 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 19 16:38:25.331958 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299112 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 19 16:38:25.331958 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299115 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 19 16:38:25.331958 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299118 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 19 16:38:25.331958 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299121 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 19 16:38:25.332903 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299124 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 19 16:38:25.332903 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299127 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 19 16:38:25.332903 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299129 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 19 16:38:25.332903 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299132 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 19 16:38:25.332903 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299135 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 19 16:38:25.332903 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299137 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 19 16:38:25.332903 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299140 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 19 16:38:25.332903 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299143 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 19 16:38:25.332903 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299145 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 19 16:38:25.332903 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299148 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 19 16:38:25.332903 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299150 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 19 16:38:25.332903 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299163 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 19 16:38:25.332903 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299166 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 19 16:38:25.332903 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299168 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 19 16:38:25.332903 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299171 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 19 16:38:25.332903 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299174 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 19 16:38:25.332903 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299177 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 19 16:38:25.332903 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299180 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 19 16:38:25.332903 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299183 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 19 16:38:25.332903 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299185 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 19 16:38:25.333571 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299188 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 19 16:38:25.333571 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299190 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 19 16:38:25.333571 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299193 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 19 16:38:25.333571 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299196 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 19 16:38:25.333571 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299198 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 19 16:38:25.333571 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299201 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 19 16:38:25.333571 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299203 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 19 16:38:25.333571 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299206 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 19 16:38:25.333571 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299208 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 19 16:38:25.333571 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299211 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 19 16:38:25.333571 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299214 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 19 16:38:25.333571 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299216 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 19 16:38:25.333571 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299219 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 19 16:38:25.333571 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299222 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 19 16:38:25.333571 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299225 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 19 16:38:25.333571 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299227 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 19 16:38:25.333571 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299230 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 19 16:38:25.333571 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299232 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 19 16:38:25.333571 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299234 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 19 16:38:25.333571 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299237 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 19 16:38:25.334144 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299239 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 19 16:38:25.334144 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299244 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 19 16:38:25.334144 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299247 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 19 16:38:25.334144 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299251 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 19 16:38:25.334144 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299254 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 19 16:38:25.334144 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299258 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 19 16:38:25.334144 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299262 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 19 16:38:25.334144 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299265 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 19 16:38:25.334144 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299268 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 19 16:38:25.334144 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299271 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 19 16:38:25.334144 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299274 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 19 16:38:25.334144 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299277 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 19 16:38:25.334144 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299280 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 19 16:38:25.334144 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299282 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 19 16:38:25.334144 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299285 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 19 16:38:25.334144 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299287 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 19 16:38:25.334144 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299290 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 19 16:38:25.334144 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299292 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 19 16:38:25.334144 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299295 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 19 16:38:25.334671 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299297 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 19 16:38:25.334671 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299299 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 19 16:38:25.334671 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299302 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 19 16:38:25.334671 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299304 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 19 16:38:25.334671 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299307 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 19 16:38:25.334671 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299310 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 19 16:38:25.334671 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299314 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 19 16:38:25.334671 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299316 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 19 16:38:25.334671 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299319 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 19 16:38:25.334671 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299321 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 19 16:38:25.334671 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299324 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 19 16:38:25.334671 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299326 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 19 16:38:25.334671 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299329 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 19 16:38:25.334671 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299332 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 19 16:38:25.334671 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299335 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 19 16:38:25.334671 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299337 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 19 16:38:25.334671 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299341 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 19 16:38:25.334671 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299344 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 19 16:38:25.334671 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299347 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 19 16:38:25.334671 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299350 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 19 16:38:25.335241 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299353 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 19 16:38:25.335241 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299358 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 19 16:38:25.335241 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299361 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 19 16:38:25.335241 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299364 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 19 16:38:25.335241 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299367 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 19 16:38:25.335241 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299369 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 19 16:38:25.335241 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299372 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 19 16:38:25.335241 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299374 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 19 16:38:25.335241 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299377 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 19 16:38:25.335241 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299380 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 19 16:38:25.335241 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299383 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 19 16:38:25.335241 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.299385 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 19 16:38:25.335241 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300102 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 19 16:38:25.335241 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300111 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 19 16:38:25.335241 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300114 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 19 16:38:25.335241 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300117 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 19 16:38:25.335241 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300120 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 19 16:38:25.335241 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300123 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 19 16:38:25.335241 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300126 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 19 16:38:25.335724 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300128 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 19 16:38:25.335724 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300131 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 19 16:38:25.335724 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300134 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 19 16:38:25.335724 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300137 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 19 16:38:25.335724 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300139 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 19 16:38:25.335724 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300142 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 19 16:38:25.335724 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300144 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 19 16:38:25.335724 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300148 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 19 16:38:25.335724 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300150 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 19 16:38:25.335724 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300153 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 19 16:38:25.335724 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300155 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 19 16:38:25.335724 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300158 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 19 16:38:25.335724 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300161 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 19 16:38:25.335724 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300164 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 19 16:38:25.335724 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300166 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 19 16:38:25.335724 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300169 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 19 16:38:25.335724 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300172 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 19 16:38:25.335724 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300175 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 19 16:38:25.335724 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300177 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 19 16:38:25.336301 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300182 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 19 16:38:25.336301 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300186 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 19 16:38:25.336301 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300189 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 19 16:38:25.336301 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300192 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 19 16:38:25.336301 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300195 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 19 16:38:25.336301 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300197 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 19 16:38:25.336301 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300200 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 19 16:38:25.336301 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300203 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 19 16:38:25.336301 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300205 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 19 16:38:25.336301 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300207 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 19 16:38:25.336301 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300210 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 19 16:38:25.336301 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300212 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 19 16:38:25.336301 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300216 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 19 16:38:25.336301 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300218 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 19 16:38:25.336301 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300221 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 19 16:38:25.336301 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300223 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 19 16:38:25.336301 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300226 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 19 16:38:25.336301 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300228 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 19 16:38:25.336301 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300230 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 19 16:38:25.336301 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300233 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 19 16:38:25.336820 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300235 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 19 16:38:25.336820 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300238 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 19 16:38:25.336820 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300240 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 19 16:38:25.336820 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300243 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 19 16:38:25.336820 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300246 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 19 16:38:25.336820 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300249 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 19 16:38:25.336820 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300251 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 19 16:38:25.336820 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300255 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 19 16:38:25.336820 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300257 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 19 16:38:25.336820 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300260 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 19 16:38:25.336820 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300263 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 19 16:38:25.336820 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300266 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 19 16:38:25.336820 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300268 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 19 16:38:25.336820 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300271 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 19 16:38:25.336820 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300274 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 19 16:38:25.336820 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300276 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 19 16:38:25.336820 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300280 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 19 16:38:25.336820 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300283 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 19 16:38:25.336820 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300285 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 19 16:38:25.337392 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300288 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 19 16:38:25.337392 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300291 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 19 16:38:25.337392 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300294 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 19 16:38:25.337392 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300297 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 19 16:38:25.337392 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300300 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 19 16:38:25.337392 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300302 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 19 16:38:25.337392 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300306 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 19 16:38:25.337392 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300308 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 19 16:38:25.337392 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300311 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 19 16:38:25.337392 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300315 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 19 16:38:25.337392 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300318 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 19 16:38:25.337392 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300321 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 19 16:38:25.337392 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300324 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 19 16:38:25.337392 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300327 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 19 16:38:25.337392 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300330 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 19 16:38:25.337392 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300333 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 19 16:38:25.337392 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300335 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 19 16:38:25.337392 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300338 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 19 16:38:25.337392 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300341 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 19 16:38:25.337392 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300343 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 19 16:38:25.338259 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300346 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 19 16:38:25.338259 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300349 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 19 16:38:25.338259 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300351 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 19 16:38:25.338259 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300354 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 19 16:38:25.338259 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300356 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 19 16:38:25.338259 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300359 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 19 16:38:25.338259 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300361 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 19 16:38:25.338259 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300364 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 19 16:38:25.338259 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300367 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 19 16:38:25.338259 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300369 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 19 16:38:25.338259 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300372 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 19 16:38:25.338259 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300375 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 19 16:38:25.338259 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300377 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 19 16:38:25.338259 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300379 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 19 16:38:25.338259 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300736 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 19 16:38:25.338259 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300742 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 19 16:38:25.338259 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300746 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 19 16:38:25.338259 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300749 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 19 16:38:25.338259 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300753 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 19 16:38:25.339269 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300756 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 19 16:38:25.339269 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300759 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 19 16:38:25.339269 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300762 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 19 16:38:25.339269 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300765 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 19 16:38:25.339269 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300771 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 19 16:38:25.339269 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300773 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 19 16:38:25.339269 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300776 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 19 16:38:25.339269 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300779 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 19 16:38:25.339269 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300782 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 19 16:38:25.339269 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300784 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 19 16:38:25.339269 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300787 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 19 16:38:25.339269 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300791 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 19 16:38:25.339269 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300794 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 19 16:38:25.339269 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300797 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 19 16:38:25.339269 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300800 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 19 16:38:25.339269 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300803 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 19 16:38:25.339269 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300806 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 19 16:38:25.339269 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300808 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 19 16:38:25.339269 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300811 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 19 16:38:25.342307 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300814 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 19 16:38:25.342307 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300816 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 19 16:38:25.342307 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300819 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 19 16:38:25.342307 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300827 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 19 16:38:25.342307 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300830 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 19 16:38:25.342307 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300833 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 19 16:38:25.342307 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300835 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 19 16:38:25.342307 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300838 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 19 16:38:25.342307 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300840 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 19 16:38:25.342307 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300843 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 19 16:38:25.342307 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300845 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 19 16:38:25.342307 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300848 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 19 16:38:25.342307 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300850 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 19 16:38:25.342307 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300854 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 19 16:38:25.342307 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300858 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 19 16:38:25.342307 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300860 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 19 16:38:25.342307 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300863 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 19 16:38:25.342307 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300867 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 19 16:38:25.342307 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300870 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 19 16:38:25.342307 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300872 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 19 16:38:25.343199 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300875 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 19 16:38:25.343199 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300877 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 19 16:38:25.343199 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300880 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 19 16:38:25.343199 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300882 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 19 16:38:25.343199 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300885 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 19 16:38:25.343199 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300887 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 19 16:38:25.343199 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300890 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 19 16:38:25.343199 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300892 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 19 16:38:25.343199 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300895 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 19 16:38:25.343199 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300897 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 19 16:38:25.343199 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300900 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 19 16:38:25.343199 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300902 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 19 16:38:25.343199 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300905 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 19 16:38:25.343199 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300907 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 19 16:38:25.343199 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300909 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 19 16:38:25.343199 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300913 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 19 16:38:25.343199 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300916 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 19 16:38:25.343199 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300918 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 19 16:38:25.343199 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300921 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 19 16:38:25.343199 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300923 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 19 16:38:25.344081 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300926 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 19 16:38:25.344081 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300928 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 19 16:38:25.344081 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300931 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 19 16:38:25.344081 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300933 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 19 16:38:25.344081 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300936 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 19 16:38:25.344081 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300938 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 19 16:38:25.344081 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300941 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 19 16:38:25.344081 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300943 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 19 16:38:25.344081 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300946 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 19 16:38:25.344081 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300950 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 19 16:38:25.344081 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300952 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 19 16:38:25.344081 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300955 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 19 16:38:25.344081 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300957 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 19 16:38:25.344081 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300960 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 19 16:38:25.344081 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300962 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 19 16:38:25.344081 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300965 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 19 16:38:25.344081 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300967 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 19 16:38:25.344081 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300970 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 19 16:38:25.344081 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300972 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 19 16:38:25.344081 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300975 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 19 16:38:25.345580 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300978 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 19 16:38:25.345580 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300980 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 19 16:38:25.345580 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300983 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 19 16:38:25.345580 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300986 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 19 16:38:25.345580 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300988 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 19 16:38:25.345580 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300991 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 19 16:38:25.345580 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300993 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 19 16:38:25.345580 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.300998 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 19 16:38:25.345580 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301001 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 19 16:38:25.345580 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301003 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 19 16:38:25.345580 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301006 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 19 16:38:25.345580 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301009 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 19 16:38:25.345580 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301011 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 19 16:38:25.345580 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301014 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 19 16:38:25.345580 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301016 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 19 16:38:25.345580 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301105 2583 flags.go:64] FLAG: --address="0.0.0.0" Jul 19 16:38:25.345580 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301113 2583 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 19 16:38:25.345580 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301119 2583 flags.go:64] FLAG: --anonymous-auth="true" Jul 19 16:38:25.345580 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301123 2583 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 19 16:38:25.345580 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301128 2583 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 19 16:38:25.345580 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301132 2583 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 19 16:38:25.346776 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301137 2583 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 19 16:38:25.346776 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301141 2583 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 19 16:38:25.346776 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301144 2583 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 19 16:38:25.346776 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301147 2583 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 19 16:38:25.346776 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301151 2583 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 19 16:38:25.346776 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301154 2583 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 19 16:38:25.346776 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301157 2583 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 19 16:38:25.346776 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301160 2583 flags.go:64] FLAG: --cgroup-root="" Jul 19 16:38:25.346776 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301162 2583 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 19 16:38:25.346776 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301165 2583 flags.go:64] FLAG: --client-ca-file="" Jul 19 16:38:25.346776 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301168 2583 flags.go:64] FLAG: --cloud-provider="external" Jul 19 16:38:25.346776 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301171 2583 flags.go:64] FLAG: --cluster-dns="[]" Jul 19 16:38:25.346776 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301175 2583 flags.go:64] FLAG: --cluster-domain="" Jul 19 16:38:25.346776 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301179 2583 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 19 16:38:25.346776 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301182 2583 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 19 16:38:25.346776 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301185 2583 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 19 16:38:25.346776 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301188 2583 flags.go:64] FLAG: --container-log-max-files="5" Jul 19 16:38:25.346776 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301192 2583 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 19 16:38:25.346776 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301196 2583 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 19 16:38:25.346776 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301199 2583 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 19 16:38:25.346776 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301202 2583 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 19 16:38:25.346776 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301205 2583 flags.go:64] FLAG: --contention-profiling="false" Jul 19 16:38:25.346776 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301208 2583 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 19 16:38:25.346776 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301211 2583 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 19 16:38:25.347866 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301214 2583 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 19 16:38:25.347866 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301217 2583 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 19 16:38:25.347866 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301221 2583 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 19 16:38:25.347866 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301224 2583 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 19 16:38:25.347866 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301227 2583 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 19 16:38:25.347866 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301230 2583 flags.go:64] FLAG: --enable-load-reader="false" Jul 19 16:38:25.347866 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301233 2583 flags.go:64] FLAG: --enable-server="true" Jul 19 16:38:25.347866 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301236 2583 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 19 16:38:25.347866 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301242 2583 flags.go:64] FLAG: --event-burst="100" Jul 19 16:38:25.347866 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301245 2583 flags.go:64] FLAG: --event-qps="50" Jul 19 16:38:25.347866 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301248 2583 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 19 16:38:25.347866 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301251 2583 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 19 16:38:25.347866 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301254 2583 flags.go:64] FLAG: --eviction-hard="" Jul 19 16:38:25.347866 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301258 2583 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 19 16:38:25.347866 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301260 2583 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 19 16:38:25.347866 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301263 2583 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 19 16:38:25.347866 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301266 2583 flags.go:64] FLAG: --eviction-soft="" Jul 19 16:38:25.347866 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301269 2583 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 19 16:38:25.347866 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301272 2583 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 19 16:38:25.347866 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301275 2583 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 19 16:38:25.347866 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301278 2583 flags.go:64] FLAG: --experimental-mounter-path="" Jul 19 16:38:25.347866 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301280 2583 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 19 16:38:25.347866 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301283 2583 flags.go:64] FLAG: --fail-swap-on="true" Jul 19 16:38:25.347866 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301286 2583 flags.go:64] FLAG: --feature-gates="" Jul 19 16:38:25.347866 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301290 2583 flags.go:64] FLAG: --file-check-frequency="20s" Jul 19 16:38:25.349373 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301293 2583 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 19 16:38:25.349373 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301297 2583 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 19 16:38:25.349373 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301300 2583 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 19 16:38:25.349373 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301303 2583 flags.go:64] FLAG: --healthz-port="10248" Jul 19 16:38:25.349373 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301306 2583 flags.go:64] FLAG: --help="false" Jul 19 16:38:25.349373 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301309 2583 flags.go:64] FLAG: --hostname-override="ip-10-0-134-203.ec2.internal" Jul 19 16:38:25.349373 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301312 2583 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 19 16:38:25.349373 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301315 2583 flags.go:64] FLAG: --http-check-frequency="20s" Jul 19 16:38:25.349373 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301318 2583 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 19 16:38:25.349373 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301321 2583 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 19 16:38:25.349373 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301325 2583 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 19 16:38:25.349373 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301327 2583 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 19 16:38:25.349373 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301330 2583 flags.go:64] FLAG: --image-service-endpoint="" Jul 19 16:38:25.349373 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301333 2583 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 19 16:38:25.349373 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301336 2583 flags.go:64] FLAG: --kube-api-burst="100" Jul 19 16:38:25.349373 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301341 2583 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 19 16:38:25.349373 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301344 2583 flags.go:64] FLAG: --kube-api-qps="50" Jul 19 16:38:25.349373 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301347 2583 flags.go:64] FLAG: --kube-reserved="" Jul 19 16:38:25.349373 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301350 2583 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 19 16:38:25.349373 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301352 2583 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 19 16:38:25.349373 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301356 2583 flags.go:64] FLAG: --kubelet-cgroups="" Jul 19 16:38:25.349373 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301358 2583 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 19 16:38:25.349373 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301361 2583 flags.go:64] FLAG: --lock-file="" Jul 19 16:38:25.349373 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301364 2583 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 19 16:38:25.350442 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301367 2583 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 19 16:38:25.350442 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301370 2583 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 19 16:38:25.350442 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301375 2583 flags.go:64] FLAG: --log-json-split-stream="false" Jul 19 16:38:25.350442 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301378 2583 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 19 16:38:25.350442 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301381 2583 flags.go:64] FLAG: --log-text-split-stream="false" Jul 19 16:38:25.350442 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301384 2583 flags.go:64] FLAG: --logging-format="text" Jul 19 16:38:25.350442 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301386 2583 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 19 16:38:25.350442 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301390 2583 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 19 16:38:25.350442 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301393 2583 flags.go:64] FLAG: --manifest-url="" Jul 19 16:38:25.350442 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301396 2583 flags.go:64] FLAG: --manifest-url-header="" Jul 19 16:38:25.350442 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301401 2583 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 19 16:38:25.350442 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301403 2583 flags.go:64] FLAG: --max-open-files="1000000" Jul 19 16:38:25.350442 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301408 2583 flags.go:64] FLAG: --max-pods="110" Jul 19 16:38:25.350442 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301410 2583 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 19 16:38:25.350442 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301413 2583 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 19 16:38:25.350442 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301416 2583 flags.go:64] FLAG: --memory-manager-policy="None" Jul 19 16:38:25.350442 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301419 2583 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 19 16:38:25.350442 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301422 2583 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 19 16:38:25.350442 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301425 2583 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 19 16:38:25.350442 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301428 2583 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 19 16:38:25.350442 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301435 2583 flags.go:64] FLAG: --node-status-max-images="50" Jul 19 16:38:25.350442 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301438 2583 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 19 16:38:25.350442 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301441 2583 flags.go:64] FLAG: --oom-score-adj="-999" Jul 19 16:38:25.350442 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301445 2583 flags.go:64] FLAG: --pod-cidr="" Jul 19 16:38:25.351730 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301449 2583 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 19 16:38:25.351730 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301454 2583 flags.go:64] FLAG: --pod-manifest-path="" Jul 19 16:38:25.351730 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301457 2583 flags.go:64] FLAG: --pod-max-pids="-1" Jul 19 16:38:25.351730 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301460 2583 flags.go:64] FLAG: --pods-per-core="0" Jul 19 16:38:25.351730 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301463 2583 flags.go:64] FLAG: --port="10250" Jul 19 16:38:25.351730 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301466 2583 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 19 16:38:25.351730 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301469 2583 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-068034e1dcbbf651a" Jul 19 16:38:25.351730 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301472 2583 flags.go:64] FLAG: --qos-reserved="" Jul 19 16:38:25.351730 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301475 2583 flags.go:64] FLAG: --read-only-port="10255" Jul 19 16:38:25.351730 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301478 2583 flags.go:64] FLAG: --register-node="true" Jul 19 16:38:25.351730 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301481 2583 flags.go:64] FLAG: --register-with-taints="" Jul 19 16:38:25.351730 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301484 2583 flags.go:64] FLAG: --registry-burst="10" Jul 19 16:38:25.351730 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301487 2583 flags.go:64] FLAG: --registry-qps="5" Jul 19 16:38:25.351730 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301490 2583 flags.go:64] FLAG: --reserved-cpus="" Jul 19 16:38:25.351730 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301492 2583 flags.go:64] FLAG: --reserved-memory="" Jul 19 16:38:25.351730 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301496 2583 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 19 16:38:25.351730 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301499 2583 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 19 16:38:25.351730 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301503 2583 flags.go:64] FLAG: --rotate-certificates="false" Jul 19 16:38:25.351730 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301506 2583 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 19 16:38:25.351730 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301509 2583 flags.go:64] FLAG: --runonce="false" Jul 19 16:38:25.351730 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301511 2583 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 19 16:38:25.351730 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301514 2583 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 19 16:38:25.351730 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301517 2583 flags.go:64] FLAG: --seccomp-default="false" Jul 19 16:38:25.351730 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301520 2583 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 19 16:38:25.351730 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301523 2583 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 19 16:38:25.352817 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301526 2583 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 19 16:38:25.352817 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301529 2583 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 19 16:38:25.352817 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301531 2583 flags.go:64] FLAG: --storage-driver-password="root" Jul 19 16:38:25.352817 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301534 2583 flags.go:64] FLAG: --storage-driver-secure="false" Jul 19 16:38:25.352817 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301537 2583 flags.go:64] FLAG: --storage-driver-table="stats" Jul 19 16:38:25.352817 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301540 2583 flags.go:64] FLAG: --storage-driver-user="root" Jul 19 16:38:25.352817 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301543 2583 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 19 16:38:25.352817 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301548 2583 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 19 16:38:25.352817 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301553 2583 flags.go:64] FLAG: --system-cgroups="" Jul 19 16:38:25.352817 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301558 2583 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 19 16:38:25.352817 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301602 2583 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 19 16:38:25.352817 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301609 2583 flags.go:64] FLAG: --tls-cert-file="" Jul 19 16:38:25.352817 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301612 2583 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 19 16:38:25.352817 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301619 2583 flags.go:64] FLAG: --tls-min-version="" Jul 19 16:38:25.352817 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301622 2583 flags.go:64] FLAG: --tls-private-key-file="" Jul 19 16:38:25.352817 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301625 2583 flags.go:64] FLAG: --topology-manager-policy="none" Jul 19 16:38:25.352817 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301628 2583 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 19 16:38:25.352817 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301632 2583 flags.go:64] FLAG: --topology-manager-scope="container" Jul 19 16:38:25.352817 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301635 2583 flags.go:64] FLAG: --v="2" Jul 19 16:38:25.352817 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301647 2583 flags.go:64] FLAG: --version="false" Jul 19 16:38:25.352817 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301652 2583 flags.go:64] FLAG: --vmodule="" Jul 19 16:38:25.352817 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301656 2583 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 19 16:38:25.352817 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.301659 2583 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 19 16:38:25.352817 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301784 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 19 16:38:25.352817 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301788 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 19 16:38:25.354328 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301791 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 19 16:38:25.354328 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301795 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 19 16:38:25.354328 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301798 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 19 16:38:25.354328 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301801 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 19 16:38:25.354328 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301804 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 19 16:38:25.354328 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301807 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 19 16:38:25.354328 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301810 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 19 16:38:25.354328 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301813 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 19 16:38:25.354328 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301816 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 19 16:38:25.354328 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301820 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 19 16:38:25.354328 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301824 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 19 16:38:25.354328 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301827 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 19 16:38:25.354328 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301829 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 19 16:38:25.354328 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301832 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 19 16:38:25.354328 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301835 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 19 16:38:25.354328 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301837 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 19 16:38:25.354328 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301840 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 19 16:38:25.354328 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301843 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 19 16:38:25.354328 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301847 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 19 16:38:25.355222 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301849 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 19 16:38:25.355222 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301852 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 19 16:38:25.355222 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301854 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 19 16:38:25.355222 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301857 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 19 16:38:25.355222 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301859 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 19 16:38:25.355222 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301862 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 19 16:38:25.355222 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301864 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 19 16:38:25.355222 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301867 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 19 16:38:25.355222 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301869 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 19 16:38:25.355222 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301872 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 19 16:38:25.355222 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301874 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 19 16:38:25.355222 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301877 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 19 16:38:25.355222 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301879 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 19 16:38:25.355222 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301882 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 19 16:38:25.355222 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301884 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 19 16:38:25.355222 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301887 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 19 16:38:25.355222 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301889 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 19 16:38:25.355222 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301892 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 19 16:38:25.355222 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301894 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 19 16:38:25.355222 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301897 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 19 16:38:25.356747 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301902 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 19 16:38:25.356747 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301905 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 19 16:38:25.356747 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301907 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 19 16:38:25.356747 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301910 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 19 16:38:25.356747 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301913 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 19 16:38:25.356747 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301916 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 19 16:38:25.356747 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301918 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 19 16:38:25.356747 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301921 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 19 16:38:25.356747 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301924 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 19 16:38:25.356747 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301927 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 19 16:38:25.356747 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301929 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 19 16:38:25.356747 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301932 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 19 16:38:25.356747 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301935 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 19 16:38:25.356747 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301938 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 19 16:38:25.356747 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301941 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 19 16:38:25.356747 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301943 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 19 16:38:25.356747 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301946 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 19 16:38:25.356747 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301948 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 19 16:38:25.356747 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301950 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 19 16:38:25.356747 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301953 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 19 16:38:25.357633 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301955 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 19 16:38:25.357633 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301958 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 19 16:38:25.357633 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301960 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 19 16:38:25.357633 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301963 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 19 16:38:25.357633 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301965 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 19 16:38:25.357633 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301968 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 19 16:38:25.357633 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301970 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 19 16:38:25.357633 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301973 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 19 16:38:25.357633 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301977 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 19 16:38:25.357633 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301981 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 19 16:38:25.357633 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301984 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 19 16:38:25.357633 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301987 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 19 16:38:25.357633 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301991 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 19 16:38:25.357633 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301994 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 19 16:38:25.357633 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301996 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 19 16:38:25.357633 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.301999 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 19 16:38:25.357633 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.302002 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 19 16:38:25.357633 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.302005 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 19 16:38:25.357633 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.302008 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 19 16:38:25.357633 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.302011 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 19 16:38:25.358722 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.302015 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 19 16:38:25.358722 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.302018 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 19 16:38:25.358722 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.302021 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 19 16:38:25.358722 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.302024 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 19 16:38:25.358722 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.302027 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 19 16:38:25.358722 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.302030 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 19 16:38:25.358722 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.302033 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 19 16:38:25.358722 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.302035 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 19 16:38:25.358722 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.302038 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 19 16:38:25.358722 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.302040 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 19 16:38:25.358722 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.302043 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 19 16:38:25.358722 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.302045 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 19 16:38:25.358722 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.302048 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 19 16:38:25.358722 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.302050 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 19 16:38:25.358722 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.302052 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 19 16:38:25.358722 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.302071 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 19 16:38:25.358722 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.302074 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 19 16:38:25.358722 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.302076 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 19 16:38:25.359584 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.302082 2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 19 16:38:25.359584 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.315808 2583 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 19 16:38:25.359584 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.315824 2583 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 19 16:38:25.359584 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.315837 2583 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 19 16:38:25.359584 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.315845 2583 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 19 16:38:25.359584 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.315903 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 19 16:38:25.359584 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.315908 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 19 16:38:25.359584 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.315911 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 19 16:38:25.359584 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.315915 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 19 16:38:25.359584 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.315918 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 19 16:38:25.359584 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.315921 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 19 16:38:25.359584 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.315923 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 19 16:38:25.359584 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.315926 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 19 16:38:25.359584 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.315929 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 19 16:38:25.359584 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.315932 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 19 16:38:25.360446 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.315934 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 19 16:38:25.360446 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.315937 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 19 16:38:25.360446 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.315940 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 19 16:38:25.360446 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.315942 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 19 16:38:25.360446 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.315947 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 19 16:38:25.360446 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.315950 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 19 16:38:25.360446 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.315953 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 19 16:38:25.360446 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.315956 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 19 16:38:25.360446 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.315959 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 19 16:38:25.360446 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.315962 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 19 16:38:25.360446 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.315964 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 19 16:38:25.360446 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.315967 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 19 16:38:25.360446 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.315970 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 19 16:38:25.360446 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.315972 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 19 16:38:25.360446 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.315975 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 19 16:38:25.360446 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.315977 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 19 16:38:25.360446 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.315980 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 19 16:38:25.360446 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.315984 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 19 16:38:25.360446 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.315987 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 19 16:38:25.361318 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.315990 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 19 16:38:25.361318 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.315993 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 19 16:38:25.361318 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.315996 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 19 16:38:25.361318 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.315998 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 19 16:38:25.361318 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316001 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 19 16:38:25.361318 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316003 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 19 16:38:25.361318 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316006 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 19 16:38:25.361318 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316009 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 19 16:38:25.361318 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316011 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 19 16:38:25.361318 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316013 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 19 16:38:25.361318 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316016 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 19 16:38:25.361318 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316019 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 19 16:38:25.361318 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316021 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 19 16:38:25.361318 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316024 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 19 16:38:25.361318 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316027 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 19 16:38:25.361318 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316029 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 19 16:38:25.361318 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316032 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 19 16:38:25.361318 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316037 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 19 16:38:25.361318 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316040 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 19 16:38:25.362828 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316043 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 19 16:38:25.362828 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316045 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 19 16:38:25.362828 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316048 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 19 16:38:25.362828 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316051 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 19 16:38:25.362828 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316070 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 19 16:38:25.362828 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316073 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 19 16:38:25.362828 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316076 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 19 16:38:25.362828 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316079 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 19 16:38:25.362828 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316082 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 19 16:38:25.362828 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316084 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 19 16:38:25.362828 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316087 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 19 16:38:25.362828 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316090 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 19 16:38:25.362828 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316093 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 19 16:38:25.362828 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316096 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 19 16:38:25.362828 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316099 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 19 16:38:25.362828 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316101 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 19 16:38:25.362828 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316104 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 19 16:38:25.362828 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316107 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 19 16:38:25.362828 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316109 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 19 16:38:25.362828 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316112 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 19 16:38:25.363911 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316114 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 19 16:38:25.363911 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316117 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 19 16:38:25.363911 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316120 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 19 16:38:25.363911 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316122 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 19 16:38:25.363911 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316125 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 19 16:38:25.363911 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316127 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 19 16:38:25.363911 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316130 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 19 16:38:25.363911 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316132 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 19 16:38:25.363911 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316135 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 19 16:38:25.363911 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316140 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 19 16:38:25.363911 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316142 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 19 16:38:25.363911 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316146 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 19 16:38:25.363911 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316150 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 19 16:38:25.363911 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316153 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 19 16:38:25.363911 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316155 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 19 16:38:25.363911 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316158 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 19 16:38:25.363911 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316160 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 19 16:38:25.363911 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316163 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 19 16:38:25.363911 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316166 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 19 16:38:25.363911 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316168 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 19 16:38:25.365770 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316171 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 19 16:38:25.365770 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316173 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 19 16:38:25.365770 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316176 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 19 16:38:25.365770 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316179 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 19 16:38:25.365770 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316183 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 19 16:38:25.365770 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316186 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 19 16:38:25.365770 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316188 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 19 16:38:25.365770 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316191 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 19 16:38:25.365770 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316194 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 19 16:38:25.365770 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316196 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 19 16:38:25.365770 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316199 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 19 16:38:25.365770 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.316204 2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 19 16:38:25.365770 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316340 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 19 16:38:25.365770 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316345 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 19 16:38:25.365770 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316348 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 19 16:38:25.450657 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316351 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 19 16:38:25.450657 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316353 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 19 16:38:25.450657 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316356 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 19 16:38:25.450657 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316358 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 19 16:38:25.450657 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316361 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 19 16:38:25.450657 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316364 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 19 16:38:25.450657 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316378 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 19 16:38:25.450657 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316383 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 19 16:38:25.450657 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316386 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 19 16:38:25.450657 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316389 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 19 16:38:25.450657 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316392 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 19 16:38:25.450657 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316395 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 19 16:38:25.450657 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316398 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 19 16:38:25.450657 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316400 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 19 16:38:25.450657 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316404 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 19 16:38:25.450657 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316406 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 19 16:38:25.450657 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316409 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 19 16:38:25.450657 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316411 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 19 16:38:25.450657 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316414 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 19 16:38:25.450657 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316417 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 19 16:38:25.420332 ip-10-0-134-203 systemd[1]: Started Kubernetes Kubelet. Jul 19 16:38:25.452155 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316419 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 19 16:38:25.452155 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316423 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 19 16:38:25.452155 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316426 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 19 16:38:25.452155 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316429 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 19 16:38:25.452155 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316431 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 19 16:38:25.452155 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316434 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 19 16:38:25.452155 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316436 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 19 16:38:25.452155 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316439 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 19 16:38:25.452155 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316441 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 19 16:38:25.452155 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316444 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 19 16:38:25.452155 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316447 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 19 16:38:25.452155 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316449 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 19 16:38:25.452155 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316451 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 19 16:38:25.452155 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316454 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 19 16:38:25.452155 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316456 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 19 16:38:25.452155 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316459 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 19 16:38:25.452155 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316461 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 19 16:38:25.452155 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316464 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 19 16:38:25.452155 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316466 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 19 16:38:25.452155 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316470 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 19 16:38:25.453536 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316472 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 19 16:38:25.453536 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316475 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 19 16:38:25.453536 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316477 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 19 16:38:25.453536 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316480 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 19 16:38:25.453536 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316482 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 19 16:38:25.453536 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316485 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 19 16:38:25.453536 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316488 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 19 16:38:25.453536 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316491 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 19 16:38:25.453536 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316493 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 19 16:38:25.453536 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316495 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 19 16:38:25.453536 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316498 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 19 16:38:25.453536 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316500 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 19 16:38:25.453536 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316503 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 19 16:38:25.453536 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316507 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 19 16:38:25.453536 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316510 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 19 16:38:25.453536 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316513 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 19 16:38:25.453536 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316516 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 19 16:38:25.453536 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316518 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 19 16:38:25.453536 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316521 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 19 16:38:25.454380 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316524 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 19 16:38:25.454380 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316528 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 19 16:38:25.454380 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316531 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 19 16:38:25.454380 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316533 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 19 16:38:25.454380 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316537 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 19 16:38:25.454380 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316540 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 19 16:38:25.454380 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316543 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 19 16:38:25.454380 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316546 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 19 16:38:25.454380 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316549 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 19 16:38:25.454380 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316552 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 19 16:38:25.454380 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316555 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 19 16:38:25.454380 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316557 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 19 16:38:25.454380 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316561 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 19 16:38:25.454380 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316564 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 19 16:38:25.454380 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316566 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 19 16:38:25.454380 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316569 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 19 16:38:25.454380 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316571 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 19 16:38:25.454380 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316574 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 19 16:38:25.454380 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316576 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 19 16:38:25.455454 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316580 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 19 16:38:25.455454 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316583 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 19 16:38:25.455454 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316586 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 19 16:38:25.455454 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316588 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 19 16:38:25.455454 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316591 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 19 16:38:25.455454 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316593 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 19 16:38:25.455454 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316596 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 19 16:38:25.455454 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316599 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 19 16:38:25.455454 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316602 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 19 16:38:25.455454 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316604 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 19 16:38:25.455454 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316607 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 19 16:38:25.455454 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316610 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 19 16:38:25.455454 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316612 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 19 16:38:25.455454 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316614 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 19 16:38:25.455454 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316617 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 19 16:38:25.455454 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316619 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 19 16:38:25.455454 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316622 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 19 16:38:25.455454 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:25.316624 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 19 16:38:25.457808 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.316629 2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 19 16:38:25.457808 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.316734 2583 server.go:962] "Client rotation is on, will bootstrap in background" Jul 19 16:38:25.457808 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.320852 2583 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 19 16:38:25.457808 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.322229 2583 server.go:1019] "Starting client certificate rotation" Jul 19 16:38:25.457808 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.322331 2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 19 16:38:25.457808 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.322371 2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 19 16:38:25.457808 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.332177 2583 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 19 16:38:25.457808 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.333330 2583 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 19 16:38:25.457808 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.373823 2583 log.go:25] "Validated CRI v1 runtime API" Jul 19 16:38:25.457808 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.381661 2583 log.go:25] "Validated CRI v1 image API" Jul 19 16:38:25.457808 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.382522 2583 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 19 16:38:25.457808 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.383934 2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 19 16:38:25.458633 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.387509 2583 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 8095ea76-2041-4e95-b855-c69a47f6aab9:/dev/nvme0n1p3 dc652cef-10f4-4225-8895-52887b5392c5:/dev/nvme0n1p4] Jul 19 16:38:25.458633 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.387531 2583 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 19 16:38:25.459454 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.395469 2583 manager.go:216] Machine: {Timestamp:2026-07-19 16:38:25.393671112 +0000 UTC m=+0.380246588 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099555 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec257402ae8e5320a78e44acc4a94dda SystemUUID:ec257402-ae8e-5320-a78e-44acc4a94dda BootID:dac34812-8511-41ce-a80c-d3ca5851425c Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:cf:fc:d7:3d:81 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:cf:fc:d7:3d:81 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:96:b9:9d:1c:e2:e2 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 19 16:38:25.459454 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.395654 2583 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 19 16:38:25.459454 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.395761 2583 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 19 16:38:25.459454 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.398982 2583 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 19 16:38:25.459454 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.399016 2583 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-203.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 19 16:38:25.459454 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.399905 2583 topology_manager.go:138] "Creating topology manager with none policy" Jul 19 16:38:25.459454 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.399919 2583 container_manager_linux.go:309] "Creating device plugin manager" Jul 19 16:38:25.459454 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.399928 2583 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 19 16:38:25.459454 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.401666 2583 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 19 16:38:25.459454 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.402889 2583 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 19 16:38:25.459454 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.405469 2583 state_mem.go:36] "Initialized new in-memory state store" Jul 19 16:38:25.459454 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.405576 2583 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 19 16:38:25.459454 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.405603 2583 kubelet.go:486] "Attempting to sync node with API server" Jul 19 16:38:25.459454 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.405612 2583 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 19 16:38:25.459454 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.405630 2583 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 19 16:38:25.459454 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.405639 2583 kubelet.go:398] "Adding apiserver pod source" Jul 19 16:38:25.459454 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.405648 2583 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 19 16:38:25.459454 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.405954 2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 19 16:38:25.460914 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.405969 2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 19 16:38:25.460914 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.406273 2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5mhhp" Jul 19 16:38:25.460914 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.413599 2583 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 19 16:38:25.460914 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.413638 2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5mhhp" Jul 19 16:38:25.460914 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.415335 2583 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 19 16:38:25.460914 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.415363 2583 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 19 16:38:25.460914 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.416284 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 19 16:38:25.460914 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.416296 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 19 16:38:25.460914 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.416302 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 19 16:38:25.460914 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.416307 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 19 16:38:25.460914 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.416315 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 19 16:38:25.460914 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.416321 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 19 16:38:25.460914 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.416328 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 19 16:38:25.460914 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.416333 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 19 16:38:25.460914 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.416340 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 19 16:38:25.460914 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.416346 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 19 16:38:25.460914 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.416355 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 19 16:38:25.460914 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.416360 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 19 16:38:25.460914 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.416381 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 19 16:38:25.460914 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.416386 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 19 16:38:25.460914 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.418623 2583 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-203.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 19 16:38:25.460914 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:25.418723 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-203.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 19 16:38:25.460914 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:25.418777 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 19 16:38:25.460914 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.419544 2583 server.go:1268] "Started kubelet" Jul 19 16:38:25.460914 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.419605 2583 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 19 16:38:25.460914 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.419617 2583 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 19 16:38:25.460914 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.419704 2583 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 19 16:38:25.460914 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.420829 2583 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 19 16:38:25.460914 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.422262 2583 server.go:310] "Adding debug handlers to kubelet server" Jul 19 16:38:25.460914 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.430179 2583 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 19 16:38:25.460914 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.430200 2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 19 16:38:25.460914 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:25.430340 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-203.ec2.internal\" not found" Jul 19 16:38:25.460914 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:25.430674 2583 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 19 16:38:25.460914 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.430791 2583 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 19 16:38:25.460914 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.430795 2583 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 19 16:38:25.460914 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.430807 2583 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 19 16:38:25.460914 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.430874 2583 reconstruct.go:97] "Volume reconstruction finished" Jul 19 16:38:25.460914 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.430879 2583 reconciler.go:29] "Reconciler: start to sync state" Jul 19 16:38:25.460914 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.431322 2583 factory.go:153] Registering CRI-O factory Jul 19 16:38:25.460914 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.431337 2583 factory.go:223] Registration of the crio container factory successfully Jul 19 16:38:25.460914 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.431400 2583 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 19 16:38:25.460914 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.431410 2583 factory.go:55] Registering systemd factory Jul 19 16:38:25.460914 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.431417 2583 factory.go:223] Registration of the systemd container factory successfully Jul 19 16:38:25.460914 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.431437 2583 factory.go:103] Registering Raw factory Jul 19 16:38:25.460914 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.431449 2583 manager.go:1196] Started watching for new ooms in manager Jul 19 16:38:25.460914 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.431867 2583 manager.go:349] Starting recovery of all containers Jul 19 16:38:25.460914 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.432562 2583 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 19 16:38:25.462264 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:25.438240 2583 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-134-203.ec2.internal\" not found" node="ip-10-0-134-203.ec2.internal" Jul 19 16:38:25.462264 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.444900 2583 manager.go:354] Recovery completed Jul 19 16:38:25.462264 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.450529 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 19 16:38:25.462264 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.452267 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-203.ec2.internal" event="NodeHasSufficientMemory" Jul 19 16:38:25.462264 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.452289 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-203.ec2.internal" event="NodeHasNoDiskPressure" Jul 19 16:38:25.462264 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.452301 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-203.ec2.internal" event="NodeHasSufficientPID" Jul 19 16:38:25.462264 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.452836 2583 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 19 16:38:25.462264 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.452845 2583 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 19 16:38:25.462264 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.452865 2583 state_mem.go:36] "Initialized new in-memory state store" Jul 19 16:38:25.462264 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.454121 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 19 16:38:25.462264 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.454180 2583 policy_none.go:49] "None policy: Start" Jul 19 16:38:25.462264 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.454204 2583 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 19 16:38:25.462264 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.454216 2583 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 19 16:38:25.462264 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.454812 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-203.ec2.internal" event="NodeHasSufficientMemory" Jul 19 16:38:25.462264 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.454833 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-203.ec2.internal" event="NodeHasNoDiskPressure" Jul 19 16:38:25.462264 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.454847 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-203.ec2.internal" event="NodeHasSufficientPID" Jul 19 16:38:25.462264 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.455420 2583 policy_none.go:47] "Start" Jul 19 16:38:25.614562 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.481681 2583 manager.go:337] "Starting Device Plugin manager" Jul 19 16:38:25.614562 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:25.481737 2583 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 19 16:38:25.614562 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.481746 2583 server.go:87] "Starting device plugin registration server" Jul 19 16:38:25.614562 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.481963 2583 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 19 16:38:25.614562 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.481974 2583 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 19 16:38:25.614562 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.482050 2583 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 19 16:38:25.614562 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.482179 2583 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 19 16:38:25.614562 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.482189 2583 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 19 16:38:25.614562 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:25.482929 2583 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 19 16:38:25.614562 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:25.482977 2583 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-203.ec2.internal\" not found" Jul 19 16:38:25.614562 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.583482 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 19 16:38:25.614562 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.585672 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-203.ec2.internal" event="NodeHasSufficientMemory" Jul 19 16:38:25.614562 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.585699 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-203.ec2.internal" event="NodeHasNoDiskPressure" Jul 19 16:38:25.614562 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.585715 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-203.ec2.internal" event="NodeHasSufficientPID" Jul 19 16:38:25.614562 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.585745 2583 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-134-203.ec2.internal" Jul 19 16:38:25.614562 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.593381 2583 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-134-203.ec2.internal" Jul 19 16:38:25.614562 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:25.593403 2583 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-134-203.ec2.internal\": node \"ip-10-0-134-203.ec2.internal\" not found" Jul 19 16:38:25.614562 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.593848 2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 19 16:38:25.614562 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.595190 2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 19 16:38:25.614562 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.595233 2583 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 19 16:38:25.614562 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.595260 2583 kubelet.go:2448] "Starting kubelet main sync loop" Jul 19 16:38:25.614562 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:25.595301 2583 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 19 16:38:25.614562 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.597308 2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 19 16:38:25.738547 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:25.641391 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-203.ec2.internal\" not found" Jul 19 16:38:25.738547 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.699900 2583 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-203.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-134-203.ec2.internal"] Jul 19 16:38:25.738547 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.699994 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 19 16:38:25.738547 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.701850 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-203.ec2.internal" event="NodeHasSufficientMemory" Jul 19 16:38:25.738547 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.701881 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-203.ec2.internal" event="NodeHasNoDiskPressure" Jul 19 16:38:25.738547 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.701892 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-203.ec2.internal" event="NodeHasSufficientPID" Jul 19 16:38:25.738547 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.703216 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 19 16:38:25.738547 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.703345 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-203.ec2.internal" Jul 19 16:38:25.738547 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.703386 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 19 16:38:25.738547 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.704014 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-203.ec2.internal" event="NodeHasSufficientMemory" Jul 19 16:38:25.738547 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.704045 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-203.ec2.internal" event="NodeHasNoDiskPressure" Jul 19 16:38:25.738547 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.704060 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-203.ec2.internal" event="NodeHasSufficientPID" Jul 19 16:38:25.738547 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.704136 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-203.ec2.internal" event="NodeHasSufficientMemory" Jul 19 16:38:25.738547 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.704159 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-203.ec2.internal" event="NodeHasNoDiskPressure" Jul 19 16:38:25.738547 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.704174 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-203.ec2.internal" event="NodeHasSufficientPID" Jul 19 16:38:25.738547 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.705460 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-203.ec2.internal" Jul 19 16:38:25.738547 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.705487 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 19 16:38:25.738547 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.707371 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-203.ec2.internal" event="NodeHasSufficientMemory" Jul 19 16:38:25.738547 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.707400 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-203.ec2.internal" event="NodeHasNoDiskPressure" Jul 19 16:38:25.738547 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.707414 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-203.ec2.internal" event="NodeHasSufficientPID" Jul 19 16:38:25.738547 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.732499 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7ec2080c7ba1c35609ddd06eab60dece-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-203.ec2.internal\" (UID: \"7ec2080c7ba1c35609ddd06eab60dece\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-203.ec2.internal" Jul 19 16:38:25.738547 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.732532 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7ec2080c7ba1c35609ddd06eab60dece-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-203.ec2.internal\" (UID: \"7ec2080c7ba1c35609ddd06eab60dece\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-203.ec2.internal" Jul 19 16:38:25.738547 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.732566 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e73bc9e2542284e7faf575627ba3b0a4-config\") pod \"kube-apiserver-proxy-ip-10-0-134-203.ec2.internal\" (UID: \"e73bc9e2542284e7faf575627ba3b0a4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-203.ec2.internal" Jul 19 16:38:25.888207 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:25.742114 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-203.ec2.internal\" not found" Jul 19 16:38:25.888207 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:25.742850 2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-203.ec2.internal\" not found" node="ip-10-0-134-203.ec2.internal" Jul 19 16:38:25.888207 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:25.753839 2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-203.ec2.internal\" not found" node="ip-10-0-134-203.ec2.internal" Jul 19 16:38:25.888207 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.833308 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7ec2080c7ba1c35609ddd06eab60dece-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-203.ec2.internal\" (UID: \"7ec2080c7ba1c35609ddd06eab60dece\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-203.ec2.internal" Jul 19 16:38:25.888207 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.833350 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e73bc9e2542284e7faf575627ba3b0a4-config\") pod \"kube-apiserver-proxy-ip-10-0-134-203.ec2.internal\" (UID: \"e73bc9e2542284e7faf575627ba3b0a4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-203.ec2.internal" Jul 19 16:38:25.888207 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.833369 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7ec2080c7ba1c35609ddd06eab60dece-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-203.ec2.internal\" (UID: \"7ec2080c7ba1c35609ddd06eab60dece\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-203.ec2.internal" Jul 19 16:38:25.888207 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.833408 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7ec2080c7ba1c35609ddd06eab60dece-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-203.ec2.internal\" (UID: \"7ec2080c7ba1c35609ddd06eab60dece\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-203.ec2.internal" Jul 19 16:38:25.888207 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.833420 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7ec2080c7ba1c35609ddd06eab60dece-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-203.ec2.internal\" (UID: \"7ec2080c7ba1c35609ddd06eab60dece\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-203.ec2.internal" Jul 19 16:38:25.888207 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:25.834549 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e73bc9e2542284e7faf575627ba3b0a4-config\") pod \"kube-apiserver-proxy-ip-10-0-134-203.ec2.internal\" (UID: \"e73bc9e2542284e7faf575627ba3b0a4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-203.ec2.internal" Jul 19 16:38:25.888207 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:25.843176 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-203.ec2.internal\" not found" Jul 19 16:38:26.025830 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:25.943245 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-203.ec2.internal\" not found" Jul 19 16:38:26.139414 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:26.044284 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-203.ec2.internal\" not found" Jul 19 16:38:26.139414 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:26.046557 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-203.ec2.internal" Jul 19 16:38:26.139414 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:26.057523 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-203.ec2.internal" Jul 19 16:38:26.145320 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:26.145295 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-203.ec2.internal\" not found" Jul 19 16:38:26.246479 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:26.246430 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-203.ec2.internal\" not found" Jul 19 16:38:26.321820 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:26.321755 2583 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 19 16:38:26.321979 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:26.321951 2583 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 19 16:38:26.322019 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:26.321951 2583 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 19 16:38:26.347155 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:26.347126 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-203.ec2.internal\" not found" Jul 19 16:38:26.417683 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:26.417633 2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-18 16:33:25 +0000 UTC" deadline="2028-04-13 17:09:51.942324038 +0000 UTC" Jul 19 16:38:26.417683 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:26.417673 2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15216h31m25.524653833s" Jul 19 16:38:26.447347 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:26.447322 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-203.ec2.internal\" not found" Jul 19 16:38:26.547681 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:26.547647 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-203.ec2.internal\" not found" Jul 19 16:38:26.571467 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:26.571436 2583 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 19 16:38:26.573616 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:26.573564 2583 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 19 16:38:26.578306 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:26.578263 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ec2080c7ba1c35609ddd06eab60dece.slice/crio-65b25f54f58c7f49c83b60fdd2b6c5ceb49a0791818a48abaaf7c74ea4d2f6a1 WatchSource:0}: Error finding container 65b25f54f58c7f49c83b60fdd2b6c5ceb49a0791818a48abaaf7c74ea4d2f6a1: Status 404 returned error can't find the container with id 65b25f54f58c7f49c83b60fdd2b6c5ceb49a0791818a48abaaf7c74ea4d2f6a1 Jul 19 16:38:26.578658 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:26.578634 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode73bc9e2542284e7faf575627ba3b0a4.slice/crio-ff00dfb05597ca71f918c8864a6a0133bd66bc71d6301572be4148ced842980c WatchSource:0}: Error finding container ff00dfb05597ca71f918c8864a6a0133bd66bc71d6301572be4148ced842980c: Status 404 returned error can't find the container with id ff00dfb05597ca71f918c8864a6a0133bd66bc71d6301572be4148ced842980c Jul 19 16:38:26.583327 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:26.583314 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 19 16:38:26.598511 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:26.598464 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-203.ec2.internal" event={"ID":"7ec2080c7ba1c35609ddd06eab60dece","Type":"ContainerStarted","Data":"65b25f54f58c7f49c83b60fdd2b6c5ceb49a0791818a48abaaf7c74ea4d2f6a1"} Jul 19 16:38:26.599454 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:26.599434 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-203.ec2.internal" event={"ID":"e73bc9e2542284e7faf575627ba3b0a4","Type":"ContainerStarted","Data":"ff00dfb05597ca71f918c8864a6a0133bd66bc71d6301572be4148ced842980c"} Jul 19 16:38:26.631585 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:26.631556 2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-203.ec2.internal" Jul 19 16:38:26.642523 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:26.642503 2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 19 16:38:26.643674 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:26.643661 2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-203.ec2.internal" Jul 19 16:38:26.655520 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:26.655502 2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 19 16:38:27.407374 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.407338 2583 apiserver.go:52] "Watching apiserver" Jul 19 16:38:27.416211 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.416183 2583 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 19 16:38:27.417265 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.417228 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-j6dwh","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-203.ec2.internal","openshift-multus/multus-additional-cni-plugins-spjzx","openshift-multus/multus-nqtmf","openshift-multus/network-metrics-daemon-thqxw","kube-system/global-pull-secret-syncer-smdwh","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c77zn","openshift-cluster-node-tuning-operator/tuned-c6rxm","openshift-network-diagnostics/network-check-target-hrqmq","openshift-network-operator/iptables-alerter-86vh4","openshift-ovn-kubernetes/ovnkube-node-mt8d2","kube-system/konnectivity-agent-vm6wh","kube-system/kube-apiserver-proxy-ip-10-0-134-203.ec2.internal","openshift-dns/node-resolver-cv2rk"] Jul 19 16:38:27.418966 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.418946 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c77zn" Jul 19 16:38:27.420966 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.420941 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.422769 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.422740 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 19 16:38:27.422978 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.422958 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 19 16:38:27.423063 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.423043 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 19 16:38:27.423495 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.423476 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-79v6x\"" Jul 19 16:38:27.423705 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.423687 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 19 16:38:27.423767 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.423723 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 19 16:38:27.423869 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.423849 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 19 16:38:27.424017 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.423997 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 19 16:38:27.424070 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.424048 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-9td7v\"" Jul 19 16:38:27.424168 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.424003 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 19 16:38:27.424240 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.424223 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 19 16:38:27.424493 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.424474 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-spjzx" Jul 19 16:38:27.425424 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.425405 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nqtmf" Jul 19 16:38:27.426202 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.426180 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 19 16:38:27.426643 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.426625 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 19 16:38:27.426865 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.426849 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 19 16:38:27.427260 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.427238 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-thqxw" Jul 19 16:38:27.427369 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:27.427324 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-thqxw" podUID="9c487ff0-684c-41bd-aca1-74636fae5571" Jul 19 16:38:27.427816 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.427795 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-wbnzx\"" Jul 19 16:38:27.428079 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.428042 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 19 16:38:27.428483 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.428371 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 19 16:38:27.428483 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.428406 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-8lv9m\"" Jul 19 16:38:27.428483 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.428440 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 19 16:38:27.430795 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.430775 2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 19 16:38:27.431923 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.431901 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-smdwh" Jul 19 16:38:27.432845 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:27.432513 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-smdwh" podUID="ea859009-b3b3-4344-8d24-493c6a7547fa" Jul 19 16:38:27.434842 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.434776 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrqmq" Jul 19 16:38:27.434942 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:27.434889 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hrqmq" podUID="f85e47bc-bc7c-47cb-b107-69d6fefde837" Jul 19 16:38:27.436495 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.436420 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-cv2rk" Jul 19 16:38:27.439073 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.439048 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-c6rxm" Jul 19 16:38:27.439550 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.439529 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 19 16:38:27.439681 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.439652 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 19 16:38:27.439800 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.439755 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-4r9c4\"" Jul 19 16:38:27.441048 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.441023 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 19 16:38:27.441842 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.441799 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7cz4s\"" Jul 19 16:38:27.441929 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.441887 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 19 16:38:27.442046 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.442028 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-86vh4" Jul 19 16:38:27.443636 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.443612 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-j6dwh" Jul 19 16:38:27.445284 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.444906 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-mq5c9\"" Jul 19 16:38:27.445284 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.444965 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 19 16:38:27.445284 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.444998 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 19 16:38:27.445284 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.445154 2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 19 16:38:27.445284 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.445176 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 19 16:38:27.445619 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.445526 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 19 16:38:27.445619 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.445607 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 19 16:38:27.445619 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.445614 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 19 16:38:27.446003 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.445964 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-xmqbx\"" Jul 19 16:38:27.446550 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.446154 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vm6wh" Jul 19 16:38:27.448157 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.448016 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-host-slash\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.448157 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.448050 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-log-socket\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.448157 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.448098 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ab2a92d3-ad97-4dd4-9566-9b46f2f85de5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-spjzx\" (UID: \"ab2a92d3-ad97-4dd4-9566-9b46f2f85de5\") " pod="openshift-multus/multus-additional-cni-plugins-spjzx" Jul 19 16:38:27.448157 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.448128 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ab2a92d3-ad97-4dd4-9566-9b46f2f85de5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-spjzx\" (UID: \"ab2a92d3-ad97-4dd4-9566-9b46f2f85de5\") " pod="openshift-multus/multus-additional-cni-plugins-spjzx" Jul 19 16:38:27.448157 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.448154 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-host-run-netns\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.448472 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.448174 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-7hpm8\"" Jul 19 16:38:27.448472 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.448180 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-host-run-ovn-kubernetes\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.448472 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.448205 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ab2a92d3-ad97-4dd4-9566-9b46f2f85de5-cnibin\") pod \"multus-additional-cni-plugins-spjzx\" (UID: \"ab2a92d3-ad97-4dd4-9566-9b46f2f85de5\") " pod="openshift-multus/multus-additional-cni-plugins-spjzx" Jul 19 16:38:27.448472 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.448230 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e4c13454-e438-40ed-b2fd-03a0d684fc8c-host-var-lib-cni-bin\") pod \"multus-nqtmf\" (UID: \"e4c13454-e438-40ed-b2fd-03a0d684fc8c\") " pod="openshift-multus/multus-nqtmf" Jul 19 16:38:27.448472 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.448248 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 19 16:38:27.448472 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.448254 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e4c13454-e438-40ed-b2fd-03a0d684fc8c-hostroot\") pod \"multus-nqtmf\" (UID: \"e4c13454-e438-40ed-b2fd-03a0d684fc8c\") " pod="openshift-multus/multus-nqtmf" Jul 19 16:38:27.448472 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.448280 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ea859009-b3b3-4344-8d24-493c6a7547fa-original-pull-secret\") pod \"global-pull-secret-syncer-smdwh\" (UID: \"ea859009-b3b3-4344-8d24-493c6a7547fa\") " pod="kube-system/global-pull-secret-syncer-smdwh" Jul 19 16:38:27.448472 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.448305 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ab2a92d3-ad97-4dd4-9566-9b46f2f85de5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-spjzx\" (UID: \"ab2a92d3-ad97-4dd4-9566-9b46f2f85de5\") " pod="openshift-multus/multus-additional-cni-plugins-spjzx" Jul 19 16:38:27.448472 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.448337 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e4c13454-e438-40ed-b2fd-03a0d684fc8c-host-run-k8s-cni-cncf-io\") pod \"multus-nqtmf\" (UID: \"e4c13454-e438-40ed-b2fd-03a0d684fc8c\") " pod="openshift-multus/multus-nqtmf" Jul 19 16:38:27.448472 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.448362 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e4c13454-e438-40ed-b2fd-03a0d684fc8c-multus-conf-dir\") pod \"multus-nqtmf\" (UID: \"e4c13454-e438-40ed-b2fd-03a0d684fc8c\") " pod="openshift-multus/multus-nqtmf" Jul 19 16:38:27.448472 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.448392 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e4c13454-e438-40ed-b2fd-03a0d684fc8c-etc-kubernetes\") pod \"multus-nqtmf\" (UID: \"e4c13454-e438-40ed-b2fd-03a0d684fc8c\") " pod="openshift-multus/multus-nqtmf" Jul 19 16:38:27.448472 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.448417 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ea859009-b3b3-4344-8d24-493c6a7547fa-kubelet-config\") pod \"global-pull-secret-syncer-smdwh\" (UID: \"ea859009-b3b3-4344-8d24-493c6a7547fa\") " pod="kube-system/global-pull-secret-syncer-smdwh" Jul 19 16:38:27.448472 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.448433 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ea859009-b3b3-4344-8d24-493c6a7547fa-dbus\") pod \"global-pull-secret-syncer-smdwh\" (UID: \"ea859009-b3b3-4344-8d24-493c6a7547fa\") " pod="kube-system/global-pull-secret-syncer-smdwh" Jul 19 16:38:27.448472 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.448451 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cf9367af-33aa-4a8a-8c39-dfe2abe512b3-etc-selinux\") pod \"aws-ebs-csi-driver-node-c77zn\" (UID: \"cf9367af-33aa-4a8a-8c39-dfe2abe512b3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c77zn" Jul 19 16:38:27.448472 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.448451 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 19 16:38:27.448472 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.448468 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9n7h\" (UniqueName: \"kubernetes.io/projected/f85e47bc-bc7c-47cb-b107-69d6fefde837-kube-api-access-f9n7h\") pod \"network-check-target-hrqmq\" (UID: \"f85e47bc-bc7c-47cb-b107-69d6fefde837\") " pod="openshift-network-diagnostics/network-check-target-hrqmq" Jul 19 16:38:27.449246 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.448500 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e4c13454-e438-40ed-b2fd-03a0d684fc8c-multus-cni-dir\") pod \"multus-nqtmf\" (UID: \"e4c13454-e438-40ed-b2fd-03a0d684fc8c\") " pod="openshift-multus/multus-nqtmf" Jul 19 16:38:27.449246 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.448534 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-host-kubelet\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.449246 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.448563 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cf9367af-33aa-4a8a-8c39-dfe2abe512b3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c77zn\" (UID: \"cf9367af-33aa-4a8a-8c39-dfe2abe512b3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c77zn" Jul 19 16:38:27.449246 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.448596 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cf9367af-33aa-4a8a-8c39-dfe2abe512b3-device-dir\") pod \"aws-ebs-csi-driver-node-c77zn\" (UID: \"cf9367af-33aa-4a8a-8c39-dfe2abe512b3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c77zn" Jul 19 16:38:27.449246 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.448620 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-run-openvswitch\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.449246 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.448645 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-host-cni-netd\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.449246 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.448675 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ab2a92d3-ad97-4dd4-9566-9b46f2f85de5-cni-binary-copy\") pod \"multus-additional-cni-plugins-spjzx\" (UID: \"ab2a92d3-ad97-4dd4-9566-9b46f2f85de5\") " pod="openshift-multus/multus-additional-cni-plugins-spjzx" Jul 19 16:38:27.449246 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.448701 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e4c13454-e438-40ed-b2fd-03a0d684fc8c-host-var-lib-cni-multus\") pod \"multus-nqtmf\" (UID: \"e4c13454-e438-40ed-b2fd-03a0d684fc8c\") " pod="openshift-multus/multus-nqtmf" Jul 19 16:38:27.449246 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.448760 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cf9367af-33aa-4a8a-8c39-dfe2abe512b3-sys-fs\") pod \"aws-ebs-csi-driver-node-c77zn\" (UID: \"cf9367af-33aa-4a8a-8c39-dfe2abe512b3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c77zn" Jul 19 16:38:27.449246 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.448795 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-ovnkube-script-lib\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.449246 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.448824 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.449246 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.448852 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-ovnkube-config\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.449246 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.448877 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qn42t\" (UniqueName: \"kubernetes.io/projected/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-kube-api-access-qn42t\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.449246 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.448907 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gf6gf\" (UniqueName: \"kubernetes.io/projected/e4c13454-e438-40ed-b2fd-03a0d684fc8c-kube-api-access-gf6gf\") pod \"multus-nqtmf\" (UID: \"e4c13454-e438-40ed-b2fd-03a0d684fc8c\") " pod="openshift-multus/multus-nqtmf" Jul 19 16:38:27.449246 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.448947 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-systemd-units\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.449246 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.448989 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-etc-openvswitch\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.450088 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.449042 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e4c13454-e438-40ed-b2fd-03a0d684fc8c-host-var-lib-kubelet\") pod \"multus-nqtmf\" (UID: \"e4c13454-e438-40ed-b2fd-03a0d684fc8c\") " pod="openshift-multus/multus-nqtmf" Jul 19 16:38:27.450088 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.449106 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e4c13454-e438-40ed-b2fd-03a0d684fc8c-system-cni-dir\") pod \"multus-nqtmf\" (UID: \"e4c13454-e438-40ed-b2fd-03a0d684fc8c\") " pod="openshift-multus/multus-nqtmf" Jul 19 16:38:27.450088 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.449161 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cf9367af-33aa-4a8a-8c39-dfe2abe512b3-socket-dir\") pod \"aws-ebs-csi-driver-node-c77zn\" (UID: \"cf9367af-33aa-4a8a-8c39-dfe2abe512b3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c77zn" Jul 19 16:38:27.450088 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.449189 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cf9367af-33aa-4a8a-8c39-dfe2abe512b3-registration-dir\") pod \"aws-ebs-csi-driver-node-c77zn\" (UID: \"cf9367af-33aa-4a8a-8c39-dfe2abe512b3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c77zn" Jul 19 16:38:27.450088 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.449219 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-node-log\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.450088 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.449242 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e4c13454-e438-40ed-b2fd-03a0d684fc8c-cni-binary-copy\") pod \"multus-nqtmf\" (UID: \"e4c13454-e438-40ed-b2fd-03a0d684fc8c\") " pod="openshift-multus/multus-nqtmf" Jul 19 16:38:27.450088 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.449266 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e4c13454-e438-40ed-b2fd-03a0d684fc8c-multus-socket-dir-parent\") pod \"multus-nqtmf\" (UID: \"e4c13454-e438-40ed-b2fd-03a0d684fc8c\") " pod="openshift-multus/multus-nqtmf" Jul 19 16:38:27.450088 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.449290 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e4c13454-e438-40ed-b2fd-03a0d684fc8c-host-run-netns\") pod \"multus-nqtmf\" (UID: \"e4c13454-e438-40ed-b2fd-03a0d684fc8c\") " pod="openshift-multus/multus-nqtmf" Jul 19 16:38:27.450088 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.449332 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t58ff\" (UniqueName: \"kubernetes.io/projected/ab2a92d3-ad97-4dd4-9566-9b46f2f85de5-kube-api-access-t58ff\") pod \"multus-additional-cni-plugins-spjzx\" (UID: \"ab2a92d3-ad97-4dd4-9566-9b46f2f85de5\") " pod="openshift-multus/multus-additional-cni-plugins-spjzx" Jul 19 16:38:27.450088 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.449368 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e4c13454-e438-40ed-b2fd-03a0d684fc8c-os-release\") pod \"multus-nqtmf\" (UID: \"e4c13454-e438-40ed-b2fd-03a0d684fc8c\") " pod="openshift-multus/multus-nqtmf" Jul 19 16:38:27.450088 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.449391 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e4c13454-e438-40ed-b2fd-03a0d684fc8c-multus-daemon-config\") pod \"multus-nqtmf\" (UID: \"e4c13454-e438-40ed-b2fd-03a0d684fc8c\") " pod="openshift-multus/multus-nqtmf" Jul 19 16:38:27.450088 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.449418 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e4c13454-e438-40ed-b2fd-03a0d684fc8c-host-run-multus-certs\") pod \"multus-nqtmf\" (UID: \"e4c13454-e438-40ed-b2fd-03a0d684fc8c\") " pod="openshift-multus/multus-nqtmf" Jul 19 16:38:27.450088 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.449442 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkx7v\" (UniqueName: \"kubernetes.io/projected/cf9367af-33aa-4a8a-8c39-dfe2abe512b3-kube-api-access-nkx7v\") pod \"aws-ebs-csi-driver-node-c77zn\" (UID: \"cf9367af-33aa-4a8a-8c39-dfe2abe512b3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c77zn" Jul 19 16:38:27.450088 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.449464 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-run-systemd\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.450088 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.449489 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9c487ff0-684c-41bd-aca1-74636fae5571-metrics-certs\") pod \"network-metrics-daemon-thqxw\" (UID: \"9c487ff0-684c-41bd-aca1-74636fae5571\") " pod="openshift-multus/network-metrics-daemon-thqxw" Jul 19 16:38:27.450088 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.449522 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-env-overrides\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.450994 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.449548 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lbkz\" (UniqueName: \"kubernetes.io/projected/9c487ff0-684c-41bd-aca1-74636fae5571-kube-api-access-4lbkz\") pod \"network-metrics-daemon-thqxw\" (UID: \"9c487ff0-684c-41bd-aca1-74636fae5571\") " pod="openshift-multus/network-metrics-daemon-thqxw" Jul 19 16:38:27.450994 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.449574 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e4c13454-e438-40ed-b2fd-03a0d684fc8c-cnibin\") pod \"multus-nqtmf\" (UID: \"e4c13454-e438-40ed-b2fd-03a0d684fc8c\") " pod="openshift-multus/multus-nqtmf" Jul 19 16:38:27.450994 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.449604 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-run-ovn\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.450994 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.449655 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-ovn-node-metrics-cert\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.450994 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.449732 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ab2a92d3-ad97-4dd4-9566-9b46f2f85de5-system-cni-dir\") pod \"multus-additional-cni-plugins-spjzx\" (UID: \"ab2a92d3-ad97-4dd4-9566-9b46f2f85de5\") " pod="openshift-multus/multus-additional-cni-plugins-spjzx" Jul 19 16:38:27.450994 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.449787 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ab2a92d3-ad97-4dd4-9566-9b46f2f85de5-os-release\") pod \"multus-additional-cni-plugins-spjzx\" (UID: \"ab2a92d3-ad97-4dd4-9566-9b46f2f85de5\") " pod="openshift-multus/multus-additional-cni-plugins-spjzx" Jul 19 16:38:27.450994 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.449817 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-var-lib-openvswitch\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.450994 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.449865 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-host-cni-bin\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.463705 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.463676 2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-twkq6" Jul 19 16:38:27.474847 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.474795 2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-twkq6" Jul 19 16:38:27.531889 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.531860 2583 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 19 16:38:27.550608 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.550576 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e4c13454-e438-40ed-b2fd-03a0d684fc8c-cni-binary-copy\") pod \"multus-nqtmf\" (UID: \"e4c13454-e438-40ed-b2fd-03a0d684fc8c\") " pod="openshift-multus/multus-nqtmf" Jul 19 16:38:27.550775 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.550623 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e4c13454-e438-40ed-b2fd-03a0d684fc8c-multus-socket-dir-parent\") pod \"multus-nqtmf\" (UID: \"e4c13454-e438-40ed-b2fd-03a0d684fc8c\") " pod="openshift-multus/multus-nqtmf" Jul 19 16:38:27.550775 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.550652 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e4c13454-e438-40ed-b2fd-03a0d684fc8c-host-run-netns\") pod \"multus-nqtmf\" (UID: \"e4c13454-e438-40ed-b2fd-03a0d684fc8c\") " pod="openshift-multus/multus-nqtmf" Jul 19 16:38:27.550775 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.550684 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9ww4\" (UniqueName: \"kubernetes.io/projected/ad28ffaa-398f-40b5-afd5-7b168b27f96a-kube-api-access-w9ww4\") pod \"tuned-c6rxm\" (UID: \"ad28ffaa-398f-40b5-afd5-7b168b27f96a\") " pod="openshift-cluster-node-tuning-operator/tuned-c6rxm" Jul 19 16:38:27.550775 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.550716 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t58ff\" (UniqueName: \"kubernetes.io/projected/ab2a92d3-ad97-4dd4-9566-9b46f2f85de5-kube-api-access-t58ff\") pod \"multus-additional-cni-plugins-spjzx\" (UID: \"ab2a92d3-ad97-4dd4-9566-9b46f2f85de5\") " pod="openshift-multus/multus-additional-cni-plugins-spjzx" Jul 19 16:38:27.550775 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.550742 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e4c13454-e438-40ed-b2fd-03a0d684fc8c-os-release\") pod \"multus-nqtmf\" (UID: \"e4c13454-e438-40ed-b2fd-03a0d684fc8c\") " pod="openshift-multus/multus-nqtmf" Jul 19 16:38:27.551050 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.550776 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e4c13454-e438-40ed-b2fd-03a0d684fc8c-multus-daemon-config\") pod \"multus-nqtmf\" (UID: \"e4c13454-e438-40ed-b2fd-03a0d684fc8c\") " pod="openshift-multus/multus-nqtmf" Jul 19 16:38:27.551050 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.550801 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e4c13454-e438-40ed-b2fd-03a0d684fc8c-host-run-multus-certs\") pod \"multus-nqtmf\" (UID: \"e4c13454-e438-40ed-b2fd-03a0d684fc8c\") " pod="openshift-multus/multus-nqtmf" Jul 19 16:38:27.551050 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.550826 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkx7v\" (UniqueName: \"kubernetes.io/projected/cf9367af-33aa-4a8a-8c39-dfe2abe512b3-kube-api-access-nkx7v\") pod \"aws-ebs-csi-driver-node-c77zn\" (UID: \"cf9367af-33aa-4a8a-8c39-dfe2abe512b3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c77zn" Jul 19 16:38:27.551050 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.550837 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e4c13454-e438-40ed-b2fd-03a0d684fc8c-multus-socket-dir-parent\") pod \"multus-nqtmf\" (UID: \"e4c13454-e438-40ed-b2fd-03a0d684fc8c\") " pod="openshift-multus/multus-nqtmf" Jul 19 16:38:27.551050 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.550854 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-run-systemd\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.551050 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.550894 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9c487ff0-684c-41bd-aca1-74636fae5571-metrics-certs\") pod \"network-metrics-daemon-thqxw\" (UID: \"9c487ff0-684c-41bd-aca1-74636fae5571\") " pod="openshift-multus/network-metrics-daemon-thqxw" Jul 19 16:38:27.551050 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.550921 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/afaa8a6b-a7e4-4e3e-b75e-edc55cd09956-konnectivity-ca\") pod \"konnectivity-agent-vm6wh\" (UID: \"afaa8a6b-a7e4-4e3e-b75e-edc55cd09956\") " pod="kube-system/konnectivity-agent-vm6wh" Jul 19 16:38:27.551050 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.550944 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e4c13454-e438-40ed-b2fd-03a0d684fc8c-os-release\") pod \"multus-nqtmf\" (UID: \"e4c13454-e438-40ed-b2fd-03a0d684fc8c\") " pod="openshift-multus/multus-nqtmf" Jul 19 16:38:27.551050 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.550958 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-env-overrides\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.551050 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.550964 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e4c13454-e438-40ed-b2fd-03a0d684fc8c-host-run-multus-certs\") pod \"multus-nqtmf\" (UID: \"e4c13454-e438-40ed-b2fd-03a0d684fc8c\") " pod="openshift-multus/multus-nqtmf" Jul 19 16:38:27.551050 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.550986 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lbkz\" (UniqueName: \"kubernetes.io/projected/9c487ff0-684c-41bd-aca1-74636fae5571-kube-api-access-4lbkz\") pod \"network-metrics-daemon-thqxw\" (UID: \"9c487ff0-684c-41bd-aca1-74636fae5571\") " pod="openshift-multus/network-metrics-daemon-thqxw" Jul 19 16:38:27.551529 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:27.551218 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 19 16:38:27.551529 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:27.551294 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c487ff0-684c-41bd-aca1-74636fae5571-metrics-certs podName:9c487ff0-684c-41bd-aca1-74636fae5571 nodeName:}" failed. No retries permitted until 2026-07-19 16:38:28.051272302 +0000 UTC m=+3.037847784 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9c487ff0-684c-41bd-aca1-74636fae5571-metrics-certs") pod "network-metrics-daemon-thqxw" (UID: "9c487ff0-684c-41bd-aca1-74636fae5571") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 19 16:38:27.551529 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.551336 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e4c13454-e438-40ed-b2fd-03a0d684fc8c-cni-binary-copy\") pod \"multus-nqtmf\" (UID: \"e4c13454-e438-40ed-b2fd-03a0d684fc8c\") " pod="openshift-multus/multus-nqtmf" Jul 19 16:38:27.551529 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.551352 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-run-systemd\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.551529 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.551361 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e4c13454-e438-40ed-b2fd-03a0d684fc8c-host-run-netns\") pod \"multus-nqtmf\" (UID: \"e4c13454-e438-40ed-b2fd-03a0d684fc8c\") " pod="openshift-multus/multus-nqtmf" Jul 19 16:38:27.551529 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.551408 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ad28ffaa-398f-40b5-afd5-7b168b27f96a-etc-sysctl-conf\") pod \"tuned-c6rxm\" (UID: \"ad28ffaa-398f-40b5-afd5-7b168b27f96a\") " pod="openshift-cluster-node-tuning-operator/tuned-c6rxm" Jul 19 16:38:27.551529 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.551437 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ad28ffaa-398f-40b5-afd5-7b168b27f96a-lib-modules\") pod \"tuned-c6rxm\" (UID: \"ad28ffaa-398f-40b5-afd5-7b168b27f96a\") " pod="openshift-cluster-node-tuning-operator/tuned-c6rxm" Jul 19 16:38:27.551529 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.551465 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e4c13454-e438-40ed-b2fd-03a0d684fc8c-cnibin\") pod \"multus-nqtmf\" (UID: \"e4c13454-e438-40ed-b2fd-03a0d684fc8c\") " pod="openshift-multus/multus-nqtmf" Jul 19 16:38:27.551529 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.551519 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-run-ovn\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.551924 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.551533 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e4c13454-e438-40ed-b2fd-03a0d684fc8c-multus-daemon-config\") pod \"multus-nqtmf\" (UID: \"e4c13454-e438-40ed-b2fd-03a0d684fc8c\") " pod="openshift-multus/multus-nqtmf" Jul 19 16:38:27.551924 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.551544 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-ovn-node-metrics-cert\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.551924 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.551596 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-run-ovn\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.551924 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.551599 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e4c13454-e438-40ed-b2fd-03a0d684fc8c-cnibin\") pod \"multus-nqtmf\" (UID: \"e4c13454-e438-40ed-b2fd-03a0d684fc8c\") " pod="openshift-multus/multus-nqtmf" Jul 19 16:38:27.551924 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.551628 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ab2a92d3-ad97-4dd4-9566-9b46f2f85de5-system-cni-dir\") pod \"multus-additional-cni-plugins-spjzx\" (UID: \"ab2a92d3-ad97-4dd4-9566-9b46f2f85de5\") " pod="openshift-multus/multus-additional-cni-plugins-spjzx" Jul 19 16:38:27.551924 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.551651 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ab2a92d3-ad97-4dd4-9566-9b46f2f85de5-system-cni-dir\") pod \"multus-additional-cni-plugins-spjzx\" (UID: \"ab2a92d3-ad97-4dd4-9566-9b46f2f85de5\") " pod="openshift-multus/multus-additional-cni-plugins-spjzx" Jul 19 16:38:27.551924 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.551663 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ab2a92d3-ad97-4dd4-9566-9b46f2f85de5-os-release\") pod \"multus-additional-cni-plugins-spjzx\" (UID: \"ab2a92d3-ad97-4dd4-9566-9b46f2f85de5\") " pod="openshift-multus/multus-additional-cni-plugins-spjzx" Jul 19 16:38:27.551924 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.551690 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-var-lib-openvswitch\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.551924 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.551718 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-host-cni-bin\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.551924 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.551748 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ad28ffaa-398f-40b5-afd5-7b168b27f96a-etc-modprobe-d\") pod \"tuned-c6rxm\" (UID: \"ad28ffaa-398f-40b5-afd5-7b168b27f96a\") " pod="openshift-cluster-node-tuning-operator/tuned-c6rxm" Jul 19 16:38:27.551924 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.551771 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-env-overrides\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.551924 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.551776 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-var-lib-openvswitch\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.551924 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.551778 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ab2a92d3-ad97-4dd4-9566-9b46f2f85de5-os-release\") pod \"multus-additional-cni-plugins-spjzx\" (UID: \"ab2a92d3-ad97-4dd4-9566-9b46f2f85de5\") " pod="openshift-multus/multus-additional-cni-plugins-spjzx" Jul 19 16:38:27.551924 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.551774 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ad28ffaa-398f-40b5-afd5-7b168b27f96a-etc-sysconfig\") pod \"tuned-c6rxm\" (UID: \"ad28ffaa-398f-40b5-afd5-7b168b27f96a\") " pod="openshift-cluster-node-tuning-operator/tuned-c6rxm" Jul 19 16:38:27.551924 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.551824 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-host-cni-bin\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.551924 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.551874 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-host-slash\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.551924 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.551892 2583 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 19 16:38:27.551924 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.551909 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-host-slash\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.552692 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.551912 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-log-socket\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.552692 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.551952 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ad28ffaa-398f-40b5-afd5-7b168b27f96a-etc-sysctl-d\") pod \"tuned-c6rxm\" (UID: \"ad28ffaa-398f-40b5-afd5-7b168b27f96a\") " pod="openshift-cluster-node-tuning-operator/tuned-c6rxm" Jul 19 16:38:27.552692 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.551980 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-log-socket\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.552692 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.552025 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ad28ffaa-398f-40b5-afd5-7b168b27f96a-sys\") pod \"tuned-c6rxm\" (UID: \"ad28ffaa-398f-40b5-afd5-7b168b27f96a\") " pod="openshift-cluster-node-tuning-operator/tuned-c6rxm" Jul 19 16:38:27.552692 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.552054 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ad28ffaa-398f-40b5-afd5-7b168b27f96a-var-lib-kubelet\") pod \"tuned-c6rxm\" (UID: \"ad28ffaa-398f-40b5-afd5-7b168b27f96a\") " pod="openshift-cluster-node-tuning-operator/tuned-c6rxm" Jul 19 16:38:27.552692 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.552110 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ab2a92d3-ad97-4dd4-9566-9b46f2f85de5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-spjzx\" (UID: \"ab2a92d3-ad97-4dd4-9566-9b46f2f85de5\") " pod="openshift-multus/multus-additional-cni-plugins-spjzx" Jul 19 16:38:27.552692 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.552141 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ab2a92d3-ad97-4dd4-9566-9b46f2f85de5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-spjzx\" (UID: \"ab2a92d3-ad97-4dd4-9566-9b46f2f85de5\") " pod="openshift-multus/multus-additional-cni-plugins-spjzx" Jul 19 16:38:27.552692 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.552169 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-host-run-netns\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.552692 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.552196 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-host-run-ovn-kubernetes\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.552692 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.552220 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1a2fb73-9db3-436a-8b9b-f4e5e7355889-tmp-dir\") pod \"node-resolver-cv2rk\" (UID: \"c1a2fb73-9db3-436a-8b9b-f4e5e7355889\") " pod="openshift-dns/node-resolver-cv2rk" Jul 19 16:38:27.552692 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.552244 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4758d408-7ed8-465a-9d96-754c3c97336a-iptables-alerter-script\") pod \"iptables-alerter-86vh4\" (UID: \"4758d408-7ed8-465a-9d96-754c3c97336a\") " pod="openshift-network-operator/iptables-alerter-86vh4" Jul 19 16:38:27.552692 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.552271 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ab2a92d3-ad97-4dd4-9566-9b46f2f85de5-cnibin\") pod \"multus-additional-cni-plugins-spjzx\" (UID: \"ab2a92d3-ad97-4dd4-9566-9b46f2f85de5\") " pod="openshift-multus/multus-additional-cni-plugins-spjzx" Jul 19 16:38:27.552692 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.552296 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e4c13454-e438-40ed-b2fd-03a0d684fc8c-host-var-lib-cni-bin\") pod \"multus-nqtmf\" (UID: \"e4c13454-e438-40ed-b2fd-03a0d684fc8c\") " pod="openshift-multus/multus-nqtmf" Jul 19 16:38:27.552692 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.552320 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e4c13454-e438-40ed-b2fd-03a0d684fc8c-hostroot\") pod \"multus-nqtmf\" (UID: \"e4c13454-e438-40ed-b2fd-03a0d684fc8c\") " pod="openshift-multus/multus-nqtmf" Jul 19 16:38:27.552692 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.552346 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ea859009-b3b3-4344-8d24-493c6a7547fa-original-pull-secret\") pod \"global-pull-secret-syncer-smdwh\" (UID: \"ea859009-b3b3-4344-8d24-493c6a7547fa\") " pod="kube-system/global-pull-secret-syncer-smdwh" Jul 19 16:38:27.552692 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.552373 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/07d22c2c-bf9e-47af-bdf6-8e535bad46b2-serviceca\") pod \"node-ca-j6dwh\" (UID: \"07d22c2c-bf9e-47af-bdf6-8e535bad46b2\") " pod="openshift-image-registry/node-ca-j6dwh" Jul 19 16:38:27.552692 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.552399 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6wjf\" (UniqueName: \"kubernetes.io/projected/c1a2fb73-9db3-436a-8b9b-f4e5e7355889-kube-api-access-d6wjf\") pod \"node-resolver-cv2rk\" (UID: \"c1a2fb73-9db3-436a-8b9b-f4e5e7355889\") " pod="openshift-dns/node-resolver-cv2rk" Jul 19 16:38:27.553440 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.552428 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ab2a92d3-ad97-4dd4-9566-9b46f2f85de5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-spjzx\" (UID: \"ab2a92d3-ad97-4dd4-9566-9b46f2f85de5\") " pod="openshift-multus/multus-additional-cni-plugins-spjzx" Jul 19 16:38:27.553440 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.552454 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e4c13454-e438-40ed-b2fd-03a0d684fc8c-host-run-k8s-cni-cncf-io\") pod \"multus-nqtmf\" (UID: \"e4c13454-e438-40ed-b2fd-03a0d684fc8c\") " pod="openshift-multus/multus-nqtmf" Jul 19 16:38:27.553440 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.552479 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e4c13454-e438-40ed-b2fd-03a0d684fc8c-multus-conf-dir\") pod \"multus-nqtmf\" (UID: \"e4c13454-e438-40ed-b2fd-03a0d684fc8c\") " pod="openshift-multus/multus-nqtmf" Jul 19 16:38:27.553440 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.552506 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e4c13454-e438-40ed-b2fd-03a0d684fc8c-etc-kubernetes\") pod \"multus-nqtmf\" (UID: \"e4c13454-e438-40ed-b2fd-03a0d684fc8c\") " pod="openshift-multus/multus-nqtmf" Jul 19 16:38:27.553440 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.552542 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ea859009-b3b3-4344-8d24-493c6a7547fa-kubelet-config\") pod \"global-pull-secret-syncer-smdwh\" (UID: \"ea859009-b3b3-4344-8d24-493c6a7547fa\") " pod="kube-system/global-pull-secret-syncer-smdwh" Jul 19 16:38:27.553440 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.552566 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ea859009-b3b3-4344-8d24-493c6a7547fa-dbus\") pod \"global-pull-secret-syncer-smdwh\" (UID: \"ea859009-b3b3-4344-8d24-493c6a7547fa\") " pod="kube-system/global-pull-secret-syncer-smdwh" Jul 19 16:38:27.553440 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.552590 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cf9367af-33aa-4a8a-8c39-dfe2abe512b3-etc-selinux\") pod \"aws-ebs-csi-driver-node-c77zn\" (UID: \"cf9367af-33aa-4a8a-8c39-dfe2abe512b3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c77zn" Jul 19 16:38:27.553440 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.552617 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9n7h\" (UniqueName: \"kubernetes.io/projected/f85e47bc-bc7c-47cb-b107-69d6fefde837-kube-api-access-f9n7h\") pod \"network-check-target-hrqmq\" (UID: \"f85e47bc-bc7c-47cb-b107-69d6fefde837\") " pod="openshift-network-diagnostics/network-check-target-hrqmq" Jul 19 16:38:27.553440 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.552641 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e4c13454-e438-40ed-b2fd-03a0d684fc8c-multus-cni-dir\") pod \"multus-nqtmf\" (UID: \"e4c13454-e438-40ed-b2fd-03a0d684fc8c\") " pod="openshift-multus/multus-nqtmf" Jul 19 16:38:27.553440 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.552666 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-host-kubelet\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.553440 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.552690 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c1a2fb73-9db3-436a-8b9b-f4e5e7355889-hosts-file\") pod \"node-resolver-cv2rk\" (UID: \"c1a2fb73-9db3-436a-8b9b-f4e5e7355889\") " pod="openshift-dns/node-resolver-cv2rk" Jul 19 16:38:27.553440 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.552714 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ad28ffaa-398f-40b5-afd5-7b168b27f96a-run\") pod \"tuned-c6rxm\" (UID: \"ad28ffaa-398f-40b5-afd5-7b168b27f96a\") " pod="openshift-cluster-node-tuning-operator/tuned-c6rxm" Jul 19 16:38:27.553440 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.552743 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thc89\" (UniqueName: \"kubernetes.io/projected/4758d408-7ed8-465a-9d96-754c3c97336a-kube-api-access-thc89\") pod \"iptables-alerter-86vh4\" (UID: \"4758d408-7ed8-465a-9d96-754c3c97336a\") " pod="openshift-network-operator/iptables-alerter-86vh4" Jul 19 16:38:27.553440 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.552766 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e4c13454-e438-40ed-b2fd-03a0d684fc8c-multus-conf-dir\") pod \"multus-nqtmf\" (UID: \"e4c13454-e438-40ed-b2fd-03a0d684fc8c\") " pod="openshift-multus/multus-nqtmf" Jul 19 16:38:27.553440 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.552822 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e4c13454-e438-40ed-b2fd-03a0d684fc8c-etc-kubernetes\") pod \"multus-nqtmf\" (UID: \"e4c13454-e438-40ed-b2fd-03a0d684fc8c\") " pod="openshift-multus/multus-nqtmf" Jul 19 16:38:27.553440 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.552827 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e4c13454-e438-40ed-b2fd-03a0d684fc8c-host-var-lib-cni-bin\") pod \"multus-nqtmf\" (UID: \"e4c13454-e438-40ed-b2fd-03a0d684fc8c\") " pod="openshift-multus/multus-nqtmf" Jul 19 16:38:27.553440 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.552876 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e4c13454-e438-40ed-b2fd-03a0d684fc8c-hostroot\") pod \"multus-nqtmf\" (UID: \"e4c13454-e438-40ed-b2fd-03a0d684fc8c\") " pod="openshift-multus/multus-nqtmf" Jul 19 16:38:27.554288 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.552880 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cf9367af-33aa-4a8a-8c39-dfe2abe512b3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c77zn\" (UID: \"cf9367af-33aa-4a8a-8c39-dfe2abe512b3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c77zn" Jul 19 16:38:27.554288 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.552924 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-host-run-ovn-kubernetes\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.554288 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.552954 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ab2a92d3-ad97-4dd4-9566-9b46f2f85de5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-spjzx\" (UID: \"ab2a92d3-ad97-4dd4-9566-9b46f2f85de5\") " pod="openshift-multus/multus-additional-cni-plugins-spjzx" Jul 19 16:38:27.554288 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.553003 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-host-run-netns\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.554288 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:27.553011 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 19 16:38:27.554288 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.553028 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ea859009-b3b3-4344-8d24-493c6a7547fa-dbus\") pod \"global-pull-secret-syncer-smdwh\" (UID: \"ea859009-b3b3-4344-8d24-493c6a7547fa\") " pod="kube-system/global-pull-secret-syncer-smdwh" Jul 19 16:38:27.554288 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.553051 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-host-kubelet\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.554288 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.552713 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ab2a92d3-ad97-4dd4-9566-9b46f2f85de5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-spjzx\" (UID: \"ab2a92d3-ad97-4dd4-9566-9b46f2f85de5\") " pod="openshift-multus/multus-additional-cni-plugins-spjzx" Jul 19 16:38:27.554288 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:27.553065 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea859009-b3b3-4344-8d24-493c6a7547fa-original-pull-secret podName:ea859009-b3b3-4344-8d24-493c6a7547fa nodeName:}" failed. No retries permitted until 2026-07-19 16:38:28.053047369 +0000 UTC m=+3.039622840 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ea859009-b3b3-4344-8d24-493c6a7547fa-original-pull-secret") pod "global-pull-secret-syncer-smdwh" (UID: "ea859009-b3b3-4344-8d24-493c6a7547fa") : object "kube-system"/"original-pull-secret" not registered Jul 19 16:38:27.554288 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.553113 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e4c13454-e438-40ed-b2fd-03a0d684fc8c-host-run-k8s-cni-cncf-io\") pod \"multus-nqtmf\" (UID: \"e4c13454-e438-40ed-b2fd-03a0d684fc8c\") " pod="openshift-multus/multus-nqtmf" Jul 19 16:38:27.554288 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.553170 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ab2a92d3-ad97-4dd4-9566-9b46f2f85de5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-spjzx\" (UID: \"ab2a92d3-ad97-4dd4-9566-9b46f2f85de5\") " pod="openshift-multus/multus-additional-cni-plugins-spjzx" Jul 19 16:38:27.554288 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.553177 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cf9367af-33aa-4a8a-8c39-dfe2abe512b3-etc-selinux\") pod \"aws-ebs-csi-driver-node-c77zn\" (UID: \"cf9367af-33aa-4a8a-8c39-dfe2abe512b3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c77zn" Jul 19 16:38:27.554288 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.553182 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ab2a92d3-ad97-4dd4-9566-9b46f2f85de5-cnibin\") pod \"multus-additional-cni-plugins-spjzx\" (UID: \"ab2a92d3-ad97-4dd4-9566-9b46f2f85de5\") " pod="openshift-multus/multus-additional-cni-plugins-spjzx" Jul 19 16:38:27.554288 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.552769 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cf9367af-33aa-4a8a-8c39-dfe2abe512b3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c77zn\" (UID: \"cf9367af-33aa-4a8a-8c39-dfe2abe512b3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c77zn" Jul 19 16:38:27.554288 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.553241 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cf9367af-33aa-4a8a-8c39-dfe2abe512b3-device-dir\") pod \"aws-ebs-csi-driver-node-c77zn\" (UID: \"cf9367af-33aa-4a8a-8c39-dfe2abe512b3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c77zn" Jul 19 16:38:27.554288 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.553244 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ea859009-b3b3-4344-8d24-493c6a7547fa-kubelet-config\") pod \"global-pull-secret-syncer-smdwh\" (UID: \"ea859009-b3b3-4344-8d24-493c6a7547fa\") " pod="kube-system/global-pull-secret-syncer-smdwh" Jul 19 16:38:27.554288 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.553259 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-run-openvswitch\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.555173 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.553275 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-host-cni-netd\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.555173 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.553298 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ad28ffaa-398f-40b5-afd5-7b168b27f96a-etc-systemd\") pod \"tuned-c6rxm\" (UID: \"ad28ffaa-398f-40b5-afd5-7b168b27f96a\") " pod="openshift-cluster-node-tuning-operator/tuned-c6rxm" Jul 19 16:38:27.555173 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.553299 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e4c13454-e438-40ed-b2fd-03a0d684fc8c-multus-cni-dir\") pod \"multus-nqtmf\" (UID: \"e4c13454-e438-40ed-b2fd-03a0d684fc8c\") " pod="openshift-multus/multus-nqtmf" Jul 19 16:38:27.555173 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.553327 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-run-openvswitch\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.555173 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.553334 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-host-cni-netd\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.555173 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.553328 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cf9367af-33aa-4a8a-8c39-dfe2abe512b3-device-dir\") pod \"aws-ebs-csi-driver-node-c77zn\" (UID: \"cf9367af-33aa-4a8a-8c39-dfe2abe512b3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c77zn" Jul 19 16:38:27.555173 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.553349 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ad28ffaa-398f-40b5-afd5-7b168b27f96a-tmp\") pod \"tuned-c6rxm\" (UID: \"ad28ffaa-398f-40b5-afd5-7b168b27f96a\") " pod="openshift-cluster-node-tuning-operator/tuned-c6rxm" Jul 19 16:38:27.555173 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.553382 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ab2a92d3-ad97-4dd4-9566-9b46f2f85de5-cni-binary-copy\") pod \"multus-additional-cni-plugins-spjzx\" (UID: \"ab2a92d3-ad97-4dd4-9566-9b46f2f85de5\") " pod="openshift-multus/multus-additional-cni-plugins-spjzx" Jul 19 16:38:27.555173 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.553412 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e4c13454-e438-40ed-b2fd-03a0d684fc8c-host-var-lib-cni-multus\") pod \"multus-nqtmf\" (UID: \"e4c13454-e438-40ed-b2fd-03a0d684fc8c\") " pod="openshift-multus/multus-nqtmf" Jul 19 16:38:27.555173 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.553456 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e4c13454-e438-40ed-b2fd-03a0d684fc8c-host-var-lib-cni-multus\") pod \"multus-nqtmf\" (UID: \"e4c13454-e438-40ed-b2fd-03a0d684fc8c\") " pod="openshift-multus/multus-nqtmf" Jul 19 16:38:27.555173 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.553439 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cf9367af-33aa-4a8a-8c39-dfe2abe512b3-sys-fs\") pod \"aws-ebs-csi-driver-node-c77zn\" (UID: \"cf9367af-33aa-4a8a-8c39-dfe2abe512b3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c77zn" Jul 19 16:38:27.555173 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.553503 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-ovnkube-script-lib\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.555173 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.553503 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cf9367af-33aa-4a8a-8c39-dfe2abe512b3-sys-fs\") pod \"aws-ebs-csi-driver-node-c77zn\" (UID: \"cf9367af-33aa-4a8a-8c39-dfe2abe512b3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c77zn" Jul 19 16:38:27.555173 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.553531 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.555173 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.553576 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.555173 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.553602 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-ovnkube-config\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.555173 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.553621 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qn42t\" (UniqueName: \"kubernetes.io/projected/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-kube-api-access-qn42t\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.555988 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.553666 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/07d22c2c-bf9e-47af-bdf6-8e535bad46b2-host\") pod \"node-ca-j6dwh\" (UID: \"07d22c2c-bf9e-47af-bdf6-8e535bad46b2\") " pod="openshift-image-registry/node-ca-j6dwh" Jul 19 16:38:27.555988 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.553684 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/afaa8a6b-a7e4-4e3e-b75e-edc55cd09956-agent-certs\") pod \"konnectivity-agent-vm6wh\" (UID: \"afaa8a6b-a7e4-4e3e-b75e-edc55cd09956\") " pod="kube-system/konnectivity-agent-vm6wh" Jul 19 16:38:27.555988 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.553701 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gf6gf\" (UniqueName: \"kubernetes.io/projected/e4c13454-e438-40ed-b2fd-03a0d684fc8c-kube-api-access-gf6gf\") pod \"multus-nqtmf\" (UID: \"e4c13454-e438-40ed-b2fd-03a0d684fc8c\") " pod="openshift-multus/multus-nqtmf" Jul 19 16:38:27.555988 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.553720 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-systemd-units\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.555988 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.553746 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-etc-openvswitch\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.555988 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.553763 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ad28ffaa-398f-40b5-afd5-7b168b27f96a-etc-kubernetes\") pod \"tuned-c6rxm\" (UID: \"ad28ffaa-398f-40b5-afd5-7b168b27f96a\") " pod="openshift-cluster-node-tuning-operator/tuned-c6rxm" Jul 19 16:38:27.555988 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.553778 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ad28ffaa-398f-40b5-afd5-7b168b27f96a-etc-tuned\") pod \"tuned-c6rxm\" (UID: \"ad28ffaa-398f-40b5-afd5-7b168b27f96a\") " pod="openshift-cluster-node-tuning-operator/tuned-c6rxm" Jul 19 16:38:27.555988 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.553805 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4758d408-7ed8-465a-9d96-754c3c97336a-host-slash\") pod \"iptables-alerter-86vh4\" (UID: \"4758d408-7ed8-465a-9d96-754c3c97336a\") " pod="openshift-network-operator/iptables-alerter-86vh4" Jul 19 16:38:27.555988 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.553827 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e4c13454-e438-40ed-b2fd-03a0d684fc8c-host-var-lib-kubelet\") pod \"multus-nqtmf\" (UID: \"e4c13454-e438-40ed-b2fd-03a0d684fc8c\") " pod="openshift-multus/multus-nqtmf" Jul 19 16:38:27.555988 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.553850 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kjj2\" (UniqueName: \"kubernetes.io/projected/07d22c2c-bf9e-47af-bdf6-8e535bad46b2-kube-api-access-2kjj2\") pod \"node-ca-j6dwh\" (UID: \"07d22c2c-bf9e-47af-bdf6-8e535bad46b2\") " pod="openshift-image-registry/node-ca-j6dwh" Jul 19 16:38:27.555988 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.553864 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ad28ffaa-398f-40b5-afd5-7b168b27f96a-host\") pod \"tuned-c6rxm\" (UID: \"ad28ffaa-398f-40b5-afd5-7b168b27f96a\") " pod="openshift-cluster-node-tuning-operator/tuned-c6rxm" Jul 19 16:38:27.555988 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.553895 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e4c13454-e438-40ed-b2fd-03a0d684fc8c-system-cni-dir\") pod \"multus-nqtmf\" (UID: \"e4c13454-e438-40ed-b2fd-03a0d684fc8c\") " pod="openshift-multus/multus-nqtmf" Jul 19 16:38:27.555988 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.553911 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cf9367af-33aa-4a8a-8c39-dfe2abe512b3-socket-dir\") pod \"aws-ebs-csi-driver-node-c77zn\" (UID: \"cf9367af-33aa-4a8a-8c39-dfe2abe512b3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c77zn" Jul 19 16:38:27.555988 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.553920 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-systemd-units\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.555988 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.553933 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cf9367af-33aa-4a8a-8c39-dfe2abe512b3-registration-dir\") pod \"aws-ebs-csi-driver-node-c77zn\" (UID: \"cf9367af-33aa-4a8a-8c39-dfe2abe512b3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c77zn" Jul 19 16:38:27.555988 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.553953 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-node-log\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.555988 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.554010 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-ovnkube-script-lib\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.556891 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.554029 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-node-log\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.556891 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.553869 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ab2a92d3-ad97-4dd4-9566-9b46f2f85de5-cni-binary-copy\") pod \"multus-additional-cni-plugins-spjzx\" (UID: \"ab2a92d3-ad97-4dd4-9566-9b46f2f85de5\") " pod="openshift-multus/multus-additional-cni-plugins-spjzx" Jul 19 16:38:27.556891 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.554085 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e4c13454-e438-40ed-b2fd-03a0d684fc8c-host-var-lib-kubelet\") pod \"multus-nqtmf\" (UID: \"e4c13454-e438-40ed-b2fd-03a0d684fc8c\") " pod="openshift-multus/multus-nqtmf" Jul 19 16:38:27.556891 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.554114 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-etc-openvswitch\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.556891 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.554163 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e4c13454-e438-40ed-b2fd-03a0d684fc8c-system-cni-dir\") pod \"multus-nqtmf\" (UID: \"e4c13454-e438-40ed-b2fd-03a0d684fc8c\") " pod="openshift-multus/multus-nqtmf" Jul 19 16:38:27.556891 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.554268 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cf9367af-33aa-4a8a-8c39-dfe2abe512b3-socket-dir\") pod \"aws-ebs-csi-driver-node-c77zn\" (UID: \"cf9367af-33aa-4a8a-8c39-dfe2abe512b3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c77zn" Jul 19 16:38:27.556891 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.554298 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cf9367af-33aa-4a8a-8c39-dfe2abe512b3-registration-dir\") pod \"aws-ebs-csi-driver-node-c77zn\" (UID: \"cf9367af-33aa-4a8a-8c39-dfe2abe512b3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c77zn" Jul 19 16:38:27.556891 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.554424 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-ovnkube-config\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.556891 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.556696 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-ovn-node-metrics-cert\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.560551 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:27.560521 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 19 16:38:27.560551 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:27.560545 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 19 16:38:27.560551 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:27.560560 2583 projected.go:196] Error preparing data for projected volume kube-api-access-f9n7h for pod openshift-network-diagnostics/network-check-target-hrqmq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 19 16:38:27.560810 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:27.560618 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f85e47bc-bc7c-47cb-b107-69d6fefde837-kube-api-access-f9n7h podName:f85e47bc-bc7c-47cb-b107-69d6fefde837 nodeName:}" failed. No retries permitted until 2026-07-19 16:38:28.060601887 +0000 UTC m=+3.047177367 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-f9n7h" (UniqueName: "kubernetes.io/projected/f85e47bc-bc7c-47cb-b107-69d6fefde837-kube-api-access-f9n7h") pod "network-check-target-hrqmq" (UID: "f85e47bc-bc7c-47cb-b107-69d6fefde837") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 19 16:38:27.563216 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.563191 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkx7v\" (UniqueName: \"kubernetes.io/projected/cf9367af-33aa-4a8a-8c39-dfe2abe512b3-kube-api-access-nkx7v\") pod \"aws-ebs-csi-driver-node-c77zn\" (UID: \"cf9367af-33aa-4a8a-8c39-dfe2abe512b3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c77zn" Jul 19 16:38:27.563638 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.563431 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t58ff\" (UniqueName: \"kubernetes.io/projected/ab2a92d3-ad97-4dd4-9566-9b46f2f85de5-kube-api-access-t58ff\") pod \"multus-additional-cni-plugins-spjzx\" (UID: \"ab2a92d3-ad97-4dd4-9566-9b46f2f85de5\") " pod="openshift-multus/multus-additional-cni-plugins-spjzx" Jul 19 16:38:27.564014 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.563992 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gf6gf\" (UniqueName: \"kubernetes.io/projected/e4c13454-e438-40ed-b2fd-03a0d684fc8c-kube-api-access-gf6gf\") pod \"multus-nqtmf\" (UID: \"e4c13454-e438-40ed-b2fd-03a0d684fc8c\") " pod="openshift-multus/multus-nqtmf" Jul 19 16:38:27.564596 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.564573 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lbkz\" (UniqueName: \"kubernetes.io/projected/9c487ff0-684c-41bd-aca1-74636fae5571-kube-api-access-4lbkz\") pod \"network-metrics-daemon-thqxw\" (UID: \"9c487ff0-684c-41bd-aca1-74636fae5571\") " pod="openshift-multus/network-metrics-daemon-thqxw" Jul 19 16:38:27.565215 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.565194 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qn42t\" (UniqueName: \"kubernetes.io/projected/6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a-kube-api-access-qn42t\") pod \"ovnkube-node-mt8d2\" (UID: \"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.586069 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.586023 2583 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 19 16:38:27.654920 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.654884 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9ww4\" (UniqueName: \"kubernetes.io/projected/ad28ffaa-398f-40b5-afd5-7b168b27f96a-kube-api-access-w9ww4\") pod \"tuned-c6rxm\" (UID: \"ad28ffaa-398f-40b5-afd5-7b168b27f96a\") " pod="openshift-cluster-node-tuning-operator/tuned-c6rxm" Jul 19 16:38:27.655179 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.654945 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/afaa8a6b-a7e4-4e3e-b75e-edc55cd09956-konnectivity-ca\") pod \"konnectivity-agent-vm6wh\" (UID: \"afaa8a6b-a7e4-4e3e-b75e-edc55cd09956\") " pod="kube-system/konnectivity-agent-vm6wh" Jul 19 16:38:27.655179 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.654980 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ad28ffaa-398f-40b5-afd5-7b168b27f96a-etc-sysctl-conf\") pod \"tuned-c6rxm\" (UID: \"ad28ffaa-398f-40b5-afd5-7b168b27f96a\") " pod="openshift-cluster-node-tuning-operator/tuned-c6rxm" Jul 19 16:38:27.655179 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.655007 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ad28ffaa-398f-40b5-afd5-7b168b27f96a-lib-modules\") pod \"tuned-c6rxm\" (UID: \"ad28ffaa-398f-40b5-afd5-7b168b27f96a\") " pod="openshift-cluster-node-tuning-operator/tuned-c6rxm" Jul 19 16:38:27.655179 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.655034 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ad28ffaa-398f-40b5-afd5-7b168b27f96a-etc-modprobe-d\") pod \"tuned-c6rxm\" (UID: \"ad28ffaa-398f-40b5-afd5-7b168b27f96a\") " pod="openshift-cluster-node-tuning-operator/tuned-c6rxm" Jul 19 16:38:27.655179 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.655053 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ad28ffaa-398f-40b5-afd5-7b168b27f96a-etc-sysconfig\") pod \"tuned-c6rxm\" (UID: \"ad28ffaa-398f-40b5-afd5-7b168b27f96a\") " pod="openshift-cluster-node-tuning-operator/tuned-c6rxm" Jul 19 16:38:27.655179 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.655081 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ad28ffaa-398f-40b5-afd5-7b168b27f96a-etc-sysctl-d\") pod \"tuned-c6rxm\" (UID: \"ad28ffaa-398f-40b5-afd5-7b168b27f96a\") " pod="openshift-cluster-node-tuning-operator/tuned-c6rxm" Jul 19 16:38:27.655179 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.655124 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ad28ffaa-398f-40b5-afd5-7b168b27f96a-sys\") pod \"tuned-c6rxm\" (UID: \"ad28ffaa-398f-40b5-afd5-7b168b27f96a\") " pod="openshift-cluster-node-tuning-operator/tuned-c6rxm" Jul 19 16:38:27.655179 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.655142 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ad28ffaa-398f-40b5-afd5-7b168b27f96a-var-lib-kubelet\") pod \"tuned-c6rxm\" (UID: \"ad28ffaa-398f-40b5-afd5-7b168b27f96a\") " pod="openshift-cluster-node-tuning-operator/tuned-c6rxm" Jul 19 16:38:27.655179 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.655171 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1a2fb73-9db3-436a-8b9b-f4e5e7355889-tmp-dir\") pod \"node-resolver-cv2rk\" (UID: \"c1a2fb73-9db3-436a-8b9b-f4e5e7355889\") " pod="openshift-dns/node-resolver-cv2rk" Jul 19 16:38:27.655635 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.655198 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4758d408-7ed8-465a-9d96-754c3c97336a-iptables-alerter-script\") pod \"iptables-alerter-86vh4\" (UID: \"4758d408-7ed8-465a-9d96-754c3c97336a\") " pod="openshift-network-operator/iptables-alerter-86vh4" Jul 19 16:38:27.655635 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.655243 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/07d22c2c-bf9e-47af-bdf6-8e535bad46b2-serviceca\") pod \"node-ca-j6dwh\" (UID: \"07d22c2c-bf9e-47af-bdf6-8e535bad46b2\") " pod="openshift-image-registry/node-ca-j6dwh" Jul 19 16:38:27.655635 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.655288 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ad28ffaa-398f-40b5-afd5-7b168b27f96a-etc-sysconfig\") pod \"tuned-c6rxm\" (UID: \"ad28ffaa-398f-40b5-afd5-7b168b27f96a\") " pod="openshift-cluster-node-tuning-operator/tuned-c6rxm" Jul 19 16:38:27.655635 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.655320 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ad28ffaa-398f-40b5-afd5-7b168b27f96a-var-lib-kubelet\") pod \"tuned-c6rxm\" (UID: \"ad28ffaa-398f-40b5-afd5-7b168b27f96a\") " pod="openshift-cluster-node-tuning-operator/tuned-c6rxm" Jul 19 16:38:27.655635 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.655375 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ad28ffaa-398f-40b5-afd5-7b168b27f96a-sys\") pod \"tuned-c6rxm\" (UID: \"ad28ffaa-398f-40b5-afd5-7b168b27f96a\") " pod="openshift-cluster-node-tuning-operator/tuned-c6rxm" Jul 19 16:38:27.655635 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.655419 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6wjf\" (UniqueName: \"kubernetes.io/projected/c1a2fb73-9db3-436a-8b9b-f4e5e7355889-kube-api-access-d6wjf\") pod \"node-resolver-cv2rk\" (UID: \"c1a2fb73-9db3-436a-8b9b-f4e5e7355889\") " pod="openshift-dns/node-resolver-cv2rk" Jul 19 16:38:27.655635 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.655440 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ad28ffaa-398f-40b5-afd5-7b168b27f96a-etc-modprobe-d\") pod \"tuned-c6rxm\" (UID: \"ad28ffaa-398f-40b5-afd5-7b168b27f96a\") " pod="openshift-cluster-node-tuning-operator/tuned-c6rxm" Jul 19 16:38:27.655635 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.655465 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ad28ffaa-398f-40b5-afd5-7b168b27f96a-etc-sysctl-d\") pod \"tuned-c6rxm\" (UID: \"ad28ffaa-398f-40b5-afd5-7b168b27f96a\") " pod="openshift-cluster-node-tuning-operator/tuned-c6rxm" Jul 19 16:38:27.655635 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.655486 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c1a2fb73-9db3-436a-8b9b-f4e5e7355889-hosts-file\") pod \"node-resolver-cv2rk\" (UID: \"c1a2fb73-9db3-436a-8b9b-f4e5e7355889\") " pod="openshift-dns/node-resolver-cv2rk" Jul 19 16:38:27.655635 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.655513 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ad28ffaa-398f-40b5-afd5-7b168b27f96a-run\") pod \"tuned-c6rxm\" (UID: \"ad28ffaa-398f-40b5-afd5-7b168b27f96a\") " pod="openshift-cluster-node-tuning-operator/tuned-c6rxm" Jul 19 16:38:27.655635 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.655540 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thc89\" (UniqueName: \"kubernetes.io/projected/4758d408-7ed8-465a-9d96-754c3c97336a-kube-api-access-thc89\") pod \"iptables-alerter-86vh4\" (UID: \"4758d408-7ed8-465a-9d96-754c3c97336a\") " pod="openshift-network-operator/iptables-alerter-86vh4" Jul 19 16:38:27.655635 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.655571 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ad28ffaa-398f-40b5-afd5-7b168b27f96a-etc-systemd\") pod \"tuned-c6rxm\" (UID: \"ad28ffaa-398f-40b5-afd5-7b168b27f96a\") " pod="openshift-cluster-node-tuning-operator/tuned-c6rxm" Jul 19 16:38:27.655635 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.655597 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ad28ffaa-398f-40b5-afd5-7b168b27f96a-tmp\") pod \"tuned-c6rxm\" (UID: \"ad28ffaa-398f-40b5-afd5-7b168b27f96a\") " pod="openshift-cluster-node-tuning-operator/tuned-c6rxm" Jul 19 16:38:27.655635 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.655611 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c1a2fb73-9db3-436a-8b9b-f4e5e7355889-hosts-file\") pod \"node-resolver-cv2rk\" (UID: \"c1a2fb73-9db3-436a-8b9b-f4e5e7355889\") " pod="openshift-dns/node-resolver-cv2rk" Jul 19 16:38:27.655635 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.655632 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/07d22c2c-bf9e-47af-bdf6-8e535bad46b2-host\") pod \"node-ca-j6dwh\" (UID: \"07d22c2c-bf9e-47af-bdf6-8e535bad46b2\") " pod="openshift-image-registry/node-ca-j6dwh" Jul 19 16:38:27.656455 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.655657 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/afaa8a6b-a7e4-4e3e-b75e-edc55cd09956-agent-certs\") pod \"konnectivity-agent-vm6wh\" (UID: \"afaa8a6b-a7e4-4e3e-b75e-edc55cd09956\") " pod="kube-system/konnectivity-agent-vm6wh" Jul 19 16:38:27.656455 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.655664 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ad28ffaa-398f-40b5-afd5-7b168b27f96a-etc-systemd\") pod \"tuned-c6rxm\" (UID: \"ad28ffaa-398f-40b5-afd5-7b168b27f96a\") " pod="openshift-cluster-node-tuning-operator/tuned-c6rxm" Jul 19 16:38:27.656455 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.655689 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ad28ffaa-398f-40b5-afd5-7b168b27f96a-etc-kubernetes\") pod \"tuned-c6rxm\" (UID: \"ad28ffaa-398f-40b5-afd5-7b168b27f96a\") " pod="openshift-cluster-node-tuning-operator/tuned-c6rxm" Jul 19 16:38:27.656455 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.655715 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ad28ffaa-398f-40b5-afd5-7b168b27f96a-etc-tuned\") pod \"tuned-c6rxm\" (UID: \"ad28ffaa-398f-40b5-afd5-7b168b27f96a\") " pod="openshift-cluster-node-tuning-operator/tuned-c6rxm" Jul 19 16:38:27.656455 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.655741 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4758d408-7ed8-465a-9d96-754c3c97336a-host-slash\") pod \"iptables-alerter-86vh4\" (UID: \"4758d408-7ed8-465a-9d96-754c3c97336a\") " pod="openshift-network-operator/iptables-alerter-86vh4" Jul 19 16:38:27.656455 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.655767 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/afaa8a6b-a7e4-4e3e-b75e-edc55cd09956-konnectivity-ca\") pod \"konnectivity-agent-vm6wh\" (UID: \"afaa8a6b-a7e4-4e3e-b75e-edc55cd09956\") " pod="kube-system/konnectivity-agent-vm6wh" Jul 19 16:38:27.656455 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.655779 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ad28ffaa-398f-40b5-afd5-7b168b27f96a-lib-modules\") pod \"tuned-c6rxm\" (UID: \"ad28ffaa-398f-40b5-afd5-7b168b27f96a\") " pod="openshift-cluster-node-tuning-operator/tuned-c6rxm" Jul 19 16:38:27.656455 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.655571 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ad28ffaa-398f-40b5-afd5-7b168b27f96a-etc-sysctl-conf\") pod \"tuned-c6rxm\" (UID: \"ad28ffaa-398f-40b5-afd5-7b168b27f96a\") " pod="openshift-cluster-node-tuning-operator/tuned-c6rxm" Jul 19 16:38:27.656455 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.655780 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1a2fb73-9db3-436a-8b9b-f4e5e7355889-tmp-dir\") pod \"node-resolver-cv2rk\" (UID: \"c1a2fb73-9db3-436a-8b9b-f4e5e7355889\") " pod="openshift-dns/node-resolver-cv2rk" Jul 19 16:38:27.656455 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.655809 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/07d22c2c-bf9e-47af-bdf6-8e535bad46b2-serviceca\") pod \"node-ca-j6dwh\" (UID: \"07d22c2c-bf9e-47af-bdf6-8e535bad46b2\") " pod="openshift-image-registry/node-ca-j6dwh" Jul 19 16:38:27.656455 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.655798 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kjj2\" (UniqueName: \"kubernetes.io/projected/07d22c2c-bf9e-47af-bdf6-8e535bad46b2-kube-api-access-2kjj2\") pod \"node-ca-j6dwh\" (UID: \"07d22c2c-bf9e-47af-bdf6-8e535bad46b2\") " pod="openshift-image-registry/node-ca-j6dwh" Jul 19 16:38:27.656455 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.655845 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/07d22c2c-bf9e-47af-bdf6-8e535bad46b2-host\") pod \"node-ca-j6dwh\" (UID: \"07d22c2c-bf9e-47af-bdf6-8e535bad46b2\") " pod="openshift-image-registry/node-ca-j6dwh" Jul 19 16:38:27.656455 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.655866 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ad28ffaa-398f-40b5-afd5-7b168b27f96a-host\") pod \"tuned-c6rxm\" (UID: \"ad28ffaa-398f-40b5-afd5-7b168b27f96a\") " pod="openshift-cluster-node-tuning-operator/tuned-c6rxm" Jul 19 16:38:27.656455 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.655876 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ad28ffaa-398f-40b5-afd5-7b168b27f96a-run\") pod \"tuned-c6rxm\" (UID: \"ad28ffaa-398f-40b5-afd5-7b168b27f96a\") " pod="openshift-cluster-node-tuning-operator/tuned-c6rxm" Jul 19 16:38:27.656455 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.655928 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4758d408-7ed8-465a-9d96-754c3c97336a-host-slash\") pod \"iptables-alerter-86vh4\" (UID: \"4758d408-7ed8-465a-9d96-754c3c97336a\") " pod="openshift-network-operator/iptables-alerter-86vh4" Jul 19 16:38:27.656455 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.655973 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ad28ffaa-398f-40b5-afd5-7b168b27f96a-etc-kubernetes\") pod \"tuned-c6rxm\" (UID: \"ad28ffaa-398f-40b5-afd5-7b168b27f96a\") " pod="openshift-cluster-node-tuning-operator/tuned-c6rxm" Jul 19 16:38:27.656455 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.655983 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ad28ffaa-398f-40b5-afd5-7b168b27f96a-host\") pod \"tuned-c6rxm\" (UID: \"ad28ffaa-398f-40b5-afd5-7b168b27f96a\") " pod="openshift-cluster-node-tuning-operator/tuned-c6rxm" Jul 19 16:38:27.657324 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.656864 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4758d408-7ed8-465a-9d96-754c3c97336a-iptables-alerter-script\") pod \"iptables-alerter-86vh4\" (UID: \"4758d408-7ed8-465a-9d96-754c3c97336a\") " pod="openshift-network-operator/iptables-alerter-86vh4" Jul 19 16:38:27.658887 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.658821 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ad28ffaa-398f-40b5-afd5-7b168b27f96a-etc-tuned\") pod \"tuned-c6rxm\" (UID: \"ad28ffaa-398f-40b5-afd5-7b168b27f96a\") " pod="openshift-cluster-node-tuning-operator/tuned-c6rxm" Jul 19 16:38:27.659001 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.658956 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/afaa8a6b-a7e4-4e3e-b75e-edc55cd09956-agent-certs\") pod \"konnectivity-agent-vm6wh\" (UID: \"afaa8a6b-a7e4-4e3e-b75e-edc55cd09956\") " pod="kube-system/konnectivity-agent-vm6wh" Jul 19 16:38:27.659456 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.659432 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ad28ffaa-398f-40b5-afd5-7b168b27f96a-tmp\") pod \"tuned-c6rxm\" (UID: \"ad28ffaa-398f-40b5-afd5-7b168b27f96a\") " pod="openshift-cluster-node-tuning-operator/tuned-c6rxm" Jul 19 16:38:27.665030 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.665006 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-thc89\" (UniqueName: \"kubernetes.io/projected/4758d408-7ed8-465a-9d96-754c3c97336a-kube-api-access-thc89\") pod \"iptables-alerter-86vh4\" (UID: \"4758d408-7ed8-465a-9d96-754c3c97336a\") " pod="openshift-network-operator/iptables-alerter-86vh4" Jul 19 16:38:27.665172 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.665111 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kjj2\" (UniqueName: \"kubernetes.io/projected/07d22c2c-bf9e-47af-bdf6-8e535bad46b2-kube-api-access-2kjj2\") pod \"node-ca-j6dwh\" (UID: \"07d22c2c-bf9e-47af-bdf6-8e535bad46b2\") " pod="openshift-image-registry/node-ca-j6dwh" Jul 19 16:38:27.665521 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.665499 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9ww4\" (UniqueName: \"kubernetes.io/projected/ad28ffaa-398f-40b5-afd5-7b168b27f96a-kube-api-access-w9ww4\") pod \"tuned-c6rxm\" (UID: \"ad28ffaa-398f-40b5-afd5-7b168b27f96a\") " pod="openshift-cluster-node-tuning-operator/tuned-c6rxm" Jul 19 16:38:27.665598 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.665499 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6wjf\" (UniqueName: \"kubernetes.io/projected/c1a2fb73-9db3-436a-8b9b-f4e5e7355889-kube-api-access-d6wjf\") pod \"node-resolver-cv2rk\" (UID: \"c1a2fb73-9db3-436a-8b9b-f4e5e7355889\") " pod="openshift-dns/node-resolver-cv2rk" Jul 19 16:38:27.735938 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.735905 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c77zn" Jul 19 16:38:27.745996 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.745969 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nqtmf" Jul 19 16:38:27.762336 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.762306 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:27.770304 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.770283 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-spjzx" Jul 19 16:38:27.777954 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.777930 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-cv2rk" Jul 19 16:38:27.783642 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.783621 2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 19 16:38:27.785444 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.785423 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-c6rxm" Jul 19 16:38:27.795215 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.795194 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-86vh4" Jul 19 16:38:27.803210 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.803174 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-j6dwh" Jul 19 16:38:27.808121 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:27.808102 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vm6wh" Jul 19 16:38:28.059879 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:28.059770 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9c487ff0-684c-41bd-aca1-74636fae5571-metrics-certs\") pod \"network-metrics-daemon-thqxw\" (UID: \"9c487ff0-684c-41bd-aca1-74636fae5571\") " pod="openshift-multus/network-metrics-daemon-thqxw" Jul 19 16:38:28.059879 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:28.059852 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ea859009-b3b3-4344-8d24-493c6a7547fa-original-pull-secret\") pod \"global-pull-secret-syncer-smdwh\" (UID: \"ea859009-b3b3-4344-8d24-493c6a7547fa\") " pod="kube-system/global-pull-secret-syncer-smdwh" Jul 19 16:38:28.060147 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:28.059965 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 19 16:38:28.060147 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:28.059992 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 19 16:38:28.060147 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:28.060042 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c487ff0-684c-41bd-aca1-74636fae5571-metrics-certs podName:9c487ff0-684c-41bd-aca1-74636fae5571 nodeName:}" failed. No retries permitted until 2026-07-19 16:38:29.060024699 +0000 UTC m=+4.046600167 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9c487ff0-684c-41bd-aca1-74636fae5571-metrics-certs") pod "network-metrics-daemon-thqxw" (UID: "9c487ff0-684c-41bd-aca1-74636fae5571") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 19 16:38:28.060147 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:28.060075 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea859009-b3b3-4344-8d24-493c6a7547fa-original-pull-secret podName:ea859009-b3b3-4344-8d24-493c6a7547fa nodeName:}" failed. No retries permitted until 2026-07-19 16:38:29.060057252 +0000 UTC m=+4.046632732 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ea859009-b3b3-4344-8d24-493c6a7547fa-original-pull-secret") pod "global-pull-secret-syncer-smdwh" (UID: "ea859009-b3b3-4344-8d24-493c6a7547fa") : object "kube-system"/"original-pull-secret" not registered Jul 19 16:38:28.160924 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:28.160871 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9n7h\" (UniqueName: \"kubernetes.io/projected/f85e47bc-bc7c-47cb-b107-69d6fefde837-kube-api-access-f9n7h\") pod \"network-check-target-hrqmq\" (UID: \"f85e47bc-bc7c-47cb-b107-69d6fefde837\") " pod="openshift-network-diagnostics/network-check-target-hrqmq" Jul 19 16:38:28.161136 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:28.161093 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 19 16:38:28.161136 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:28.161120 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 19 16:38:28.161136 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:28.161134 2583 projected.go:196] Error preparing data for projected volume kube-api-access-f9n7h for pod openshift-network-diagnostics/network-check-target-hrqmq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 19 16:38:28.161280 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:28.161207 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f85e47bc-bc7c-47cb-b107-69d6fefde837-kube-api-access-f9n7h podName:f85e47bc-bc7c-47cb-b107-69d6fefde837 nodeName:}" failed. No retries permitted until 2026-07-19 16:38:29.161185531 +0000 UTC m=+4.147760999 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-f9n7h" (UniqueName: "kubernetes.io/projected/f85e47bc-bc7c-47cb-b107-69d6fefde837-kube-api-access-f9n7h") pod "network-check-target-hrqmq" (UID: "f85e47bc-bc7c-47cb-b107-69d6fefde837") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 19 16:38:28.477960 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:28.477854 2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-18 16:33:27 +0000 UTC" deadline="2028-01-29 00:20:09.161738124 +0000 UTC" Jul 19 16:38:28.477960 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:28.477907 2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13399h41m40.683836315s" Jul 19 16:38:28.596086 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:28.596041 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-smdwh" Jul 19 16:38:28.596216 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:28.596192 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-smdwh" podUID="ea859009-b3b3-4344-8d24-493c6a7547fa" Jul 19 16:38:28.596748 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:28.596720 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrqmq" Jul 19 16:38:28.596876 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:28.596853 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hrqmq" podUID="f85e47bc-bc7c-47cb-b107-69d6fefde837" Jul 19 16:38:28.600156 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:28.600128 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab2a92d3_ad97_4dd4_9566_9b46f2f85de5.slice/crio-a5eea1492cc1b05d55b10abbb824a64af6c0e7c51be6184111414070a043218f WatchSource:0}: Error finding container a5eea1492cc1b05d55b10abbb824a64af6c0e7c51be6184111414070a043218f: Status 404 returned error can't find the container with id a5eea1492cc1b05d55b10abbb824a64af6c0e7c51be6184111414070a043218f Jul 19 16:38:28.601883 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:28.601843 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e0fb240_17f5_46e2_aee6_3f8ef4a9b18a.slice/crio-07f04cd07b9a75ab9d03aa71844053852d3b612d909a3e43886915a42bb8a5c0 WatchSource:0}: Error finding container 07f04cd07b9a75ab9d03aa71844053852d3b612d909a3e43886915a42bb8a5c0: Status 404 returned error can't find the container with id 07f04cd07b9a75ab9d03aa71844053852d3b612d909a3e43886915a42bb8a5c0 Jul 19 16:38:28.604621 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:28.604477 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-spjzx" event={"ID":"ab2a92d3-ad97-4dd4-9566-9b46f2f85de5","Type":"ContainerStarted","Data":"a5eea1492cc1b05d55b10abbb824a64af6c0e7c51be6184111414070a043218f"} Jul 19 16:38:28.606162 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:28.606112 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4758d408_7ed8_465a_9d96_754c3c97336a.slice/crio-dd297e68a75062c4d8286e01c8729e5a34ede810bdcc3a5800e9f24548fc44d9 WatchSource:0}: Error finding container dd297e68a75062c4d8286e01c8729e5a34ede810bdcc3a5800e9f24548fc44d9: Status 404 returned error can't find the container with id dd297e68a75062c4d8286e01c8729e5a34ede810bdcc3a5800e9f24548fc44d9 Jul 19 16:38:28.607297 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:28.607273 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1a2fb73_9db3_436a_8b9b_f4e5e7355889.slice/crio-6065ee65e38ecf8a1dab5de744bf810793ae9dbc8711ad2b739bb63e999f5f2b WatchSource:0}: Error finding container 6065ee65e38ecf8a1dab5de744bf810793ae9dbc8711ad2b739bb63e999f5f2b: Status 404 returned error can't find the container with id 6065ee65e38ecf8a1dab5de744bf810793ae9dbc8711ad2b739bb63e999f5f2b Jul 19 16:38:28.607752 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:28.607731 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad28ffaa_398f_40b5_afd5_7b168b27f96a.slice/crio-34f70f15d0c72f695ebef701d5a3d97cdc0552c7320b643d5c3b3859134ace64 WatchSource:0}: Error finding container 34f70f15d0c72f695ebef701d5a3d97cdc0552c7320b643d5c3b3859134ace64: Status 404 returned error can't find the container with id 34f70f15d0c72f695ebef701d5a3d97cdc0552c7320b643d5c3b3859134ace64 Jul 19 16:38:28.609658 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:28.609634 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafaa8a6b_a7e4_4e3e_b75e_edc55cd09956.slice/crio-27275fad2b70982cadb3e848faea5ce28b16507ba64b3b0ae03f7fca7d49b73d WatchSource:0}: Error finding container 27275fad2b70982cadb3e848faea5ce28b16507ba64b3b0ae03f7fca7d49b73d: Status 404 returned error can't find the container with id 27275fad2b70982cadb3e848faea5ce28b16507ba64b3b0ae03f7fca7d49b73d Jul 19 16:38:28.610977 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:28.610950 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07d22c2c_bf9e_47af_bdf6_8e535bad46b2.slice/crio-3df0e21167a4571cbcb6c931103a26fcb09ae5ac992e3be057324639642f482b WatchSource:0}: Error finding container 3df0e21167a4571cbcb6c931103a26fcb09ae5ac992e3be057324639642f482b: Status 404 returned error can't find the container with id 3df0e21167a4571cbcb6c931103a26fcb09ae5ac992e3be057324639642f482b Jul 19 16:38:28.614751 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:28.612237 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4c13454_e438_40ed_b2fd_03a0d684fc8c.slice/crio-be85725d25b4bdb8df67128ca1c4277372590c04826e90ce512e923bfde7f06a WatchSource:0}: Error finding container be85725d25b4bdb8df67128ca1c4277372590c04826e90ce512e923bfde7f06a: Status 404 returned error can't find the container with id be85725d25b4bdb8df67128ca1c4277372590c04826e90ce512e923bfde7f06a Jul 19 16:38:28.614751 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:38:28.612953 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf9367af_33aa_4a8a_8c39_dfe2abe512b3.slice/crio-64ab28f171683ed5cd5f2f22a394c8bdee77678a60044e295f212b0112036fa4 WatchSource:0}: Error finding container 64ab28f171683ed5cd5f2f22a394c8bdee77678a60044e295f212b0112036fa4: Status 404 returned error can't find the container with id 64ab28f171683ed5cd5f2f22a394c8bdee77678a60044e295f212b0112036fa4 Jul 19 16:38:29.071194 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:29.069996 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9c487ff0-684c-41bd-aca1-74636fae5571-metrics-certs\") pod \"network-metrics-daemon-thqxw\" (UID: \"9c487ff0-684c-41bd-aca1-74636fae5571\") " pod="openshift-multus/network-metrics-daemon-thqxw" Jul 19 16:38:29.071194 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:29.070332 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ea859009-b3b3-4344-8d24-493c6a7547fa-original-pull-secret\") pod \"global-pull-secret-syncer-smdwh\" (UID: \"ea859009-b3b3-4344-8d24-493c6a7547fa\") " pod="kube-system/global-pull-secret-syncer-smdwh" Jul 19 16:38:29.071194 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:29.070485 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 19 16:38:29.071194 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:29.070554 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea859009-b3b3-4344-8d24-493c6a7547fa-original-pull-secret podName:ea859009-b3b3-4344-8d24-493c6a7547fa nodeName:}" failed. No retries permitted until 2026-07-19 16:38:31.070533693 +0000 UTC m=+6.057109164 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ea859009-b3b3-4344-8d24-493c6a7547fa-original-pull-secret") pod "global-pull-secret-syncer-smdwh" (UID: "ea859009-b3b3-4344-8d24-493c6a7547fa") : object "kube-system"/"original-pull-secret" not registered Jul 19 16:38:29.071194 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:29.071012 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 19 16:38:29.071194 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:29.071064 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c487ff0-684c-41bd-aca1-74636fae5571-metrics-certs podName:9c487ff0-684c-41bd-aca1-74636fae5571 nodeName:}" failed. No retries permitted until 2026-07-19 16:38:31.071046321 +0000 UTC m=+6.057621788 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9c487ff0-684c-41bd-aca1-74636fae5571-metrics-certs") pod "network-metrics-daemon-thqxw" (UID: "9c487ff0-684c-41bd-aca1-74636fae5571") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 19 16:38:29.171203 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:29.171158 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9n7h\" (UniqueName: \"kubernetes.io/projected/f85e47bc-bc7c-47cb-b107-69d6fefde837-kube-api-access-f9n7h\") pod \"network-check-target-hrqmq\" (UID: \"f85e47bc-bc7c-47cb-b107-69d6fefde837\") " pod="openshift-network-diagnostics/network-check-target-hrqmq" Jul 19 16:38:29.171414 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:29.171396 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 19 16:38:29.171510 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:29.171435 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 19 16:38:29.171510 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:29.171451 2583 projected.go:196] Error preparing data for projected volume kube-api-access-f9n7h for pod openshift-network-diagnostics/network-check-target-hrqmq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 19 16:38:29.171627 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:29.171517 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f85e47bc-bc7c-47cb-b107-69d6fefde837-kube-api-access-f9n7h podName:f85e47bc-bc7c-47cb-b107-69d6fefde837 nodeName:}" failed. No retries permitted until 2026-07-19 16:38:31.171495439 +0000 UTC m=+6.158070921 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-f9n7h" (UniqueName: "kubernetes.io/projected/f85e47bc-bc7c-47cb-b107-69d6fefde837-kube-api-access-f9n7h") pod "network-check-target-hrqmq" (UID: "f85e47bc-bc7c-47cb-b107-69d6fefde837") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 19 16:38:29.478430 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:29.478329 2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-18 16:33:27 +0000 UTC" deadline="2028-01-04 15:17:01.964227192 +0000 UTC" Jul 19 16:38:29.478430 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:29.478373 2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12814h38m32.485859266s" Jul 19 16:38:29.598842 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:29.598802 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-thqxw" Jul 19 16:38:29.599101 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:29.598969 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-thqxw" podUID="9c487ff0-684c-41bd-aca1-74636fae5571" Jul 19 16:38:29.615546 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:29.615509 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c77zn" event={"ID":"cf9367af-33aa-4a8a-8c39-dfe2abe512b3","Type":"ContainerStarted","Data":"64ab28f171683ed5cd5f2f22a394c8bdee77678a60044e295f212b0112036fa4"} Jul 19 16:38:29.621987 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:29.621864 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-cv2rk" event={"ID":"c1a2fb73-9db3-436a-8b9b-f4e5e7355889","Type":"ContainerStarted","Data":"6065ee65e38ecf8a1dab5de744bf810793ae9dbc8711ad2b739bb63e999f5f2b"} Jul 19 16:38:29.629712 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:29.629639 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-86vh4" event={"ID":"4758d408-7ed8-465a-9d96-754c3c97336a","Type":"ContainerStarted","Data":"dd297e68a75062c4d8286e01c8729e5a34ede810bdcc3a5800e9f24548fc44d9"} Jul 19 16:38:29.638161 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:29.637311 2583 generic.go:356] "Generic (PLEG): container finished" podID="7ec2080c7ba1c35609ddd06eab60dece" containerID="5fb269ee01bba8b7fe0b965205a8b20e81286a1fe56645fee2abcacaaa56bf7f" exitCode=0 Jul 19 16:38:29.638161 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:29.637397 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-203.ec2.internal" event={"ID":"7ec2080c7ba1c35609ddd06eab60dece","Type":"ContainerDied","Data":"5fb269ee01bba8b7fe0b965205a8b20e81286a1fe56645fee2abcacaaa56bf7f"} Jul 19 16:38:29.655544 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:29.655460 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-203.ec2.internal" event={"ID":"e73bc9e2542284e7faf575627ba3b0a4","Type":"ContainerStarted","Data":"9c8d6ffa61a8531b26642973fcce81034c8150a274113651d6a1eb8fc78cb34e"} Jul 19 16:38:29.670451 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:29.670340 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nqtmf" event={"ID":"e4c13454-e438-40ed-b2fd-03a0d684fc8c","Type":"ContainerStarted","Data":"be85725d25b4bdb8df67128ca1c4277372590c04826e90ce512e923bfde7f06a"} Jul 19 16:38:29.671873 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:29.671782 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-j6dwh" event={"ID":"07d22c2c-bf9e-47af-bdf6-8e535bad46b2","Type":"ContainerStarted","Data":"3df0e21167a4571cbcb6c931103a26fcb09ae5ac992e3be057324639642f482b"} Jul 19 16:38:29.696676 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:29.696633 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vm6wh" event={"ID":"afaa8a6b-a7e4-4e3e-b75e-edc55cd09956","Type":"ContainerStarted","Data":"27275fad2b70982cadb3e848faea5ce28b16507ba64b3b0ae03f7fca7d49b73d"} Jul 19 16:38:29.705049 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:29.705012 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-c6rxm" event={"ID":"ad28ffaa-398f-40b5-afd5-7b168b27f96a","Type":"ContainerStarted","Data":"34f70f15d0c72f695ebef701d5a3d97cdc0552c7320b643d5c3b3859134ace64"} Jul 19 16:38:29.717718 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:29.717618 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" event={"ID":"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a","Type":"ContainerStarted","Data":"07f04cd07b9a75ab9d03aa71844053852d3b612d909a3e43886915a42bb8a5c0"} Jul 19 16:38:30.595637 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:30.595603 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-smdwh" Jul 19 16:38:30.596202 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:30.595736 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-smdwh" podUID="ea859009-b3b3-4344-8d24-493c6a7547fa" Jul 19 16:38:30.596202 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:30.595873 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrqmq" Jul 19 16:38:30.596202 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:30.596012 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hrqmq" podUID="f85e47bc-bc7c-47cb-b107-69d6fefde837" Jul 19 16:38:30.746334 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:30.746196 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-203.ec2.internal" event={"ID":"7ec2080c7ba1c35609ddd06eab60dece","Type":"ContainerStarted","Data":"01adc67df6fb662193c12d78ab432a8cc724937821f55ccae7a5a131fe2156a5"} Jul 19 16:38:30.762808 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:30.762403 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-203.ec2.internal" podStartSLOduration=3.762381258 podStartE2EDuration="3.762381258s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 16:38:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 16:38:29.670880548 +0000 UTC m=+4.657456039" watchObservedRunningTime="2026-07-19 16:38:30.762381258 +0000 UTC m=+5.748956746" Jul 19 16:38:31.091972 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:31.091673 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9c487ff0-684c-41bd-aca1-74636fae5571-metrics-certs\") pod \"network-metrics-daemon-thqxw\" (UID: \"9c487ff0-684c-41bd-aca1-74636fae5571\") " pod="openshift-multus/network-metrics-daemon-thqxw" Jul 19 16:38:31.091972 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:31.091752 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ea859009-b3b3-4344-8d24-493c6a7547fa-original-pull-secret\") pod \"global-pull-secret-syncer-smdwh\" (UID: \"ea859009-b3b3-4344-8d24-493c6a7547fa\") " pod="kube-system/global-pull-secret-syncer-smdwh" Jul 19 16:38:31.091972 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:31.091880 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 19 16:38:31.091972 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:31.091966 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c487ff0-684c-41bd-aca1-74636fae5571-metrics-certs podName:9c487ff0-684c-41bd-aca1-74636fae5571 nodeName:}" failed. No retries permitted until 2026-07-19 16:38:35.09194312 +0000 UTC m=+10.078518593 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9c487ff0-684c-41bd-aca1-74636fae5571-metrics-certs") pod "network-metrics-daemon-thqxw" (UID: "9c487ff0-684c-41bd-aca1-74636fae5571") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 19 16:38:31.092345 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:31.091890 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 19 16:38:31.092401 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:31.092393 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea859009-b3b3-4344-8d24-493c6a7547fa-original-pull-secret podName:ea859009-b3b3-4344-8d24-493c6a7547fa nodeName:}" failed. No retries permitted until 2026-07-19 16:38:35.09237487 +0000 UTC m=+10.078950349 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ea859009-b3b3-4344-8d24-493c6a7547fa-original-pull-secret") pod "global-pull-secret-syncer-smdwh" (UID: "ea859009-b3b3-4344-8d24-493c6a7547fa") : object "kube-system"/"original-pull-secret" not registered Jul 19 16:38:31.192989 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:31.192334 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9n7h\" (UniqueName: \"kubernetes.io/projected/f85e47bc-bc7c-47cb-b107-69d6fefde837-kube-api-access-f9n7h\") pod \"network-check-target-hrqmq\" (UID: \"f85e47bc-bc7c-47cb-b107-69d6fefde837\") " pod="openshift-network-diagnostics/network-check-target-hrqmq" Jul 19 16:38:31.192989 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:31.192521 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 19 16:38:31.192989 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:31.192548 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 19 16:38:31.192989 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:31.192562 2583 projected.go:196] Error preparing data for projected volume kube-api-access-f9n7h for pod openshift-network-diagnostics/network-check-target-hrqmq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 19 16:38:31.192989 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:31.192629 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f85e47bc-bc7c-47cb-b107-69d6fefde837-kube-api-access-f9n7h podName:f85e47bc-bc7c-47cb-b107-69d6fefde837 nodeName:}" failed. No retries permitted until 2026-07-19 16:38:35.192607972 +0000 UTC m=+10.179183440 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-f9n7h" (UniqueName: "kubernetes.io/projected/f85e47bc-bc7c-47cb-b107-69d6fefde837-kube-api-access-f9n7h") pod "network-check-target-hrqmq" (UID: "f85e47bc-bc7c-47cb-b107-69d6fefde837") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 19 16:38:31.598373 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:31.598278 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-thqxw" Jul 19 16:38:31.598861 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:31.598439 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-thqxw" podUID="9c487ff0-684c-41bd-aca1-74636fae5571" Jul 19 16:38:32.596838 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:32.596313 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrqmq" Jul 19 16:38:32.596838 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:32.596342 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-smdwh" Jul 19 16:38:32.596838 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:32.596448 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hrqmq" podUID="f85e47bc-bc7c-47cb-b107-69d6fefde837" Jul 19 16:38:32.596838 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:32.596532 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-smdwh" podUID="ea859009-b3b3-4344-8d24-493c6a7547fa" Jul 19 16:38:33.596805 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:33.596716 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-thqxw" Jul 19 16:38:33.597252 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:33.596888 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-thqxw" podUID="9c487ff0-684c-41bd-aca1-74636fae5571" Jul 19 16:38:34.596484 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:34.595790 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-smdwh" Jul 19 16:38:34.596484 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:34.595928 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-smdwh" podUID="ea859009-b3b3-4344-8d24-493c6a7547fa" Jul 19 16:38:34.596484 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:34.596354 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrqmq" Jul 19 16:38:34.596484 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:34.596444 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hrqmq" podUID="f85e47bc-bc7c-47cb-b107-69d6fefde837" Jul 19 16:38:35.134271 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:35.134052 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ea859009-b3b3-4344-8d24-493c6a7547fa-original-pull-secret\") pod \"global-pull-secret-syncer-smdwh\" (UID: \"ea859009-b3b3-4344-8d24-493c6a7547fa\") " pod="kube-system/global-pull-secret-syncer-smdwh" Jul 19 16:38:35.134271 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:35.134163 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9c487ff0-684c-41bd-aca1-74636fae5571-metrics-certs\") pod \"network-metrics-daemon-thqxw\" (UID: \"9c487ff0-684c-41bd-aca1-74636fae5571\") " pod="openshift-multus/network-metrics-daemon-thqxw" Jul 19 16:38:35.134271 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:35.134249 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 19 16:38:35.134850 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:35.134289 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 19 16:38:35.134850 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:35.134339 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea859009-b3b3-4344-8d24-493c6a7547fa-original-pull-secret podName:ea859009-b3b3-4344-8d24-493c6a7547fa nodeName:}" failed. No retries permitted until 2026-07-19 16:38:43.134314211 +0000 UTC m=+18.120889697 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ea859009-b3b3-4344-8d24-493c6a7547fa-original-pull-secret") pod "global-pull-secret-syncer-smdwh" (UID: "ea859009-b3b3-4344-8d24-493c6a7547fa") : object "kube-system"/"original-pull-secret" not registered Jul 19 16:38:35.134850 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:35.134363 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c487ff0-684c-41bd-aca1-74636fae5571-metrics-certs podName:9c487ff0-684c-41bd-aca1-74636fae5571 nodeName:}" failed. No retries permitted until 2026-07-19 16:38:43.134350654 +0000 UTC m=+18.120926118 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9c487ff0-684c-41bd-aca1-74636fae5571-metrics-certs") pod "network-metrics-daemon-thqxw" (UID: "9c487ff0-684c-41bd-aca1-74636fae5571") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 19 16:38:35.235019 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:35.234969 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9n7h\" (UniqueName: \"kubernetes.io/projected/f85e47bc-bc7c-47cb-b107-69d6fefde837-kube-api-access-f9n7h\") pod \"network-check-target-hrqmq\" (UID: \"f85e47bc-bc7c-47cb-b107-69d6fefde837\") " pod="openshift-network-diagnostics/network-check-target-hrqmq" Jul 19 16:38:35.235206 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:35.235174 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 19 16:38:35.235206 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:35.235200 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 19 16:38:35.235329 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:35.235213 2583 projected.go:196] Error preparing data for projected volume kube-api-access-f9n7h for pod openshift-network-diagnostics/network-check-target-hrqmq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 19 16:38:35.235329 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:35.235271 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f85e47bc-bc7c-47cb-b107-69d6fefde837-kube-api-access-f9n7h podName:f85e47bc-bc7c-47cb-b107-69d6fefde837 nodeName:}" failed. No retries permitted until 2026-07-19 16:38:43.235253465 +0000 UTC m=+18.221828930 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-f9n7h" (UniqueName: "kubernetes.io/projected/f85e47bc-bc7c-47cb-b107-69d6fefde837-kube-api-access-f9n7h") pod "network-check-target-hrqmq" (UID: "f85e47bc-bc7c-47cb-b107-69d6fefde837") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 19 16:38:35.597241 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:35.596941 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-thqxw" Jul 19 16:38:35.597241 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:35.597082 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-thqxw" podUID="9c487ff0-684c-41bd-aca1-74636fae5571" Jul 19 16:38:36.596794 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:36.596413 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrqmq" Jul 19 16:38:36.596794 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:36.596464 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-smdwh" Jul 19 16:38:36.596794 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:36.596556 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hrqmq" podUID="f85e47bc-bc7c-47cb-b107-69d6fefde837" Jul 19 16:38:36.597410 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:36.596973 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-smdwh" podUID="ea859009-b3b3-4344-8d24-493c6a7547fa" Jul 19 16:38:37.595886 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:37.595771 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-thqxw" Jul 19 16:38:37.596131 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:37.595895 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-thqxw" podUID="9c487ff0-684c-41bd-aca1-74636fae5571" Jul 19 16:38:38.595761 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:38.595621 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-smdwh" Jul 19 16:38:38.596216 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:38.595785 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-smdwh" podUID="ea859009-b3b3-4344-8d24-493c6a7547fa" Jul 19 16:38:38.596216 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:38.595827 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrqmq" Jul 19 16:38:38.596216 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:38.596002 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hrqmq" podUID="f85e47bc-bc7c-47cb-b107-69d6fefde837" Jul 19 16:38:39.595679 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:39.595594 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-thqxw" Jul 19 16:38:39.595839 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:39.595715 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-thqxw" podUID="9c487ff0-684c-41bd-aca1-74636fae5571" Jul 19 16:38:40.595758 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:40.595727 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-smdwh" Jul 19 16:38:40.595958 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:40.595726 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrqmq" Jul 19 16:38:40.595958 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:40.595871 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-smdwh" podUID="ea859009-b3b3-4344-8d24-493c6a7547fa" Jul 19 16:38:40.595958 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:40.595938 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hrqmq" podUID="f85e47bc-bc7c-47cb-b107-69d6fefde837" Jul 19 16:38:41.596621 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:41.596311 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-thqxw" Jul 19 16:38:41.596621 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:41.596466 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-thqxw" podUID="9c487ff0-684c-41bd-aca1-74636fae5571" Jul 19 16:38:42.595622 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:42.595583 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-smdwh" Jul 19 16:38:42.595851 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:42.595590 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrqmq" Jul 19 16:38:42.595851 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:42.595709 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-smdwh" podUID="ea859009-b3b3-4344-8d24-493c6a7547fa" Jul 19 16:38:42.595851 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:42.595791 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hrqmq" podUID="f85e47bc-bc7c-47cb-b107-69d6fefde837" Jul 19 16:38:43.202338 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:43.202299 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ea859009-b3b3-4344-8d24-493c6a7547fa-original-pull-secret\") pod \"global-pull-secret-syncer-smdwh\" (UID: \"ea859009-b3b3-4344-8d24-493c6a7547fa\") " pod="kube-system/global-pull-secret-syncer-smdwh" Jul 19 16:38:43.202786 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:43.202398 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9c487ff0-684c-41bd-aca1-74636fae5571-metrics-certs\") pod \"network-metrics-daemon-thqxw\" (UID: \"9c487ff0-684c-41bd-aca1-74636fae5571\") " pod="openshift-multus/network-metrics-daemon-thqxw" Jul 19 16:38:43.202786 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:43.202485 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 19 16:38:43.202786 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:43.202504 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 19 16:38:43.202786 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:43.202567 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c487ff0-684c-41bd-aca1-74636fae5571-metrics-certs podName:9c487ff0-684c-41bd-aca1-74636fae5571 nodeName:}" failed. No retries permitted until 2026-07-19 16:38:59.20254727 +0000 UTC m=+34.189122739 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9c487ff0-684c-41bd-aca1-74636fae5571-metrics-certs") pod "network-metrics-daemon-thqxw" (UID: "9c487ff0-684c-41bd-aca1-74636fae5571") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 19 16:38:43.202786 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:43.202585 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea859009-b3b3-4344-8d24-493c6a7547fa-original-pull-secret podName:ea859009-b3b3-4344-8d24-493c6a7547fa nodeName:}" failed. No retries permitted until 2026-07-19 16:38:59.202575893 +0000 UTC m=+34.189151358 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ea859009-b3b3-4344-8d24-493c6a7547fa-original-pull-secret") pod "global-pull-secret-syncer-smdwh" (UID: "ea859009-b3b3-4344-8d24-493c6a7547fa") : object "kube-system"/"original-pull-secret" not registered Jul 19 16:38:43.302751 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:43.302718 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9n7h\" (UniqueName: \"kubernetes.io/projected/f85e47bc-bc7c-47cb-b107-69d6fefde837-kube-api-access-f9n7h\") pod \"network-check-target-hrqmq\" (UID: \"f85e47bc-bc7c-47cb-b107-69d6fefde837\") " pod="openshift-network-diagnostics/network-check-target-hrqmq" Jul 19 16:38:43.302940 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:43.302871 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 19 16:38:43.302940 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:43.302886 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 19 16:38:43.302940 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:43.302896 2583 projected.go:196] Error preparing data for projected volume kube-api-access-f9n7h for pod openshift-network-diagnostics/network-check-target-hrqmq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 19 16:38:43.303120 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:43.302944 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f85e47bc-bc7c-47cb-b107-69d6fefde837-kube-api-access-f9n7h podName:f85e47bc-bc7c-47cb-b107-69d6fefde837 nodeName:}" failed. No retries permitted until 2026-07-19 16:38:59.30293031 +0000 UTC m=+34.289505772 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-f9n7h" (UniqueName: "kubernetes.io/projected/f85e47bc-bc7c-47cb-b107-69d6fefde837-kube-api-access-f9n7h") pod "network-check-target-hrqmq" (UID: "f85e47bc-bc7c-47cb-b107-69d6fefde837") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 19 16:38:43.596169 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:43.596094 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-thqxw" Jul 19 16:38:43.596318 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:43.596202 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-thqxw" podUID="9c487ff0-684c-41bd-aca1-74636fae5571" Jul 19 16:38:44.595814 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:44.595771 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrqmq" Jul 19 16:38:44.596289 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:44.595907 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hrqmq" podUID="f85e47bc-bc7c-47cb-b107-69d6fefde837" Jul 19 16:38:44.596289 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:44.595961 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-smdwh" Jul 19 16:38:44.596289 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:44.596086 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-smdwh" podUID="ea859009-b3b3-4344-8d24-493c6a7547fa" Jul 19 16:38:45.596805 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:45.596768 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-thqxw" Jul 19 16:38:45.597261 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:45.596876 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-thqxw" podUID="9c487ff0-684c-41bd-aca1-74636fae5571" Jul 19 16:38:46.596125 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:46.596087 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-smdwh" Jul 19 16:38:46.596289 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:46.596131 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrqmq" Jul 19 16:38:46.596289 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:46.596231 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-smdwh" podUID="ea859009-b3b3-4344-8d24-493c6a7547fa" Jul 19 16:38:46.596371 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:46.596349 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hrqmq" podUID="f85e47bc-bc7c-47cb-b107-69d6fefde837" Jul 19 16:38:47.597074 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:47.596474 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-thqxw" Jul 19 16:38:47.597074 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:47.596770 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-thqxw" podUID="9c487ff0-684c-41bd-aca1-74636fae5571" Jul 19 16:38:47.787440 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:47.787219 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" event={"ID":"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a","Type":"ContainerStarted","Data":"d36b776535d1952611c9ca9c750218ed9d637939f4b124636cce957c864ee84f"} Jul 19 16:38:47.787581 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:47.787488 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" event={"ID":"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a","Type":"ContainerStarted","Data":"2d9ce2af66e24e520a2aba212cff9ef56079cbc986673fa0206dc24b1f7b6836"} Jul 19 16:38:47.787581 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:47.787526 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" event={"ID":"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a","Type":"ContainerStarted","Data":"0f5d5831342163ae68f51c0fe8e041b8ca50f1272c23b9607f5e45f9e55e6916"} Jul 19 16:38:47.788891 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:47.788868 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c77zn" event={"ID":"cf9367af-33aa-4a8a-8c39-dfe2abe512b3","Type":"ContainerStarted","Data":"4aff6759f98440ef46bf83c81fba4e929aee3c65116b70059245bab2bde10a2d"} Jul 19 16:38:47.790183 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:47.790161 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-cv2rk" event={"ID":"c1a2fb73-9db3-436a-8b9b-f4e5e7355889","Type":"ContainerStarted","Data":"4a89c1a60bced51531ebe6975dea6686f0d8db35855a743e32baea03d031d488"} Jul 19 16:38:47.791640 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:47.791615 2583 generic.go:356] "Generic (PLEG): container finished" podID="ab2a92d3-ad97-4dd4-9566-9b46f2f85de5" containerID="8479417ab42f510bccc2682227809f4cbcf7e16f5c28ff95b2cb80e9b6a92f0b" exitCode=0 Jul 19 16:38:47.791724 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:47.791706 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-spjzx" event={"ID":"ab2a92d3-ad97-4dd4-9566-9b46f2f85de5","Type":"ContainerDied","Data":"8479417ab42f510bccc2682227809f4cbcf7e16f5c28ff95b2cb80e9b6a92f0b"} Jul 19 16:38:47.793258 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:47.793230 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nqtmf" event={"ID":"e4c13454-e438-40ed-b2fd-03a0d684fc8c","Type":"ContainerStarted","Data":"adc932217343dd573c34867d48c15b205ef23ae2c8610f39573b4b715521bb78"} Jul 19 16:38:47.794631 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:47.794608 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-j6dwh" event={"ID":"07d22c2c-bf9e-47af-bdf6-8e535bad46b2","Type":"ContainerStarted","Data":"77d904d793e861faea36d4688ec5148a65cbe8d23679ceb57cc25ac3c1427708"} Jul 19 16:38:47.795927 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:47.795899 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vm6wh" event={"ID":"afaa8a6b-a7e4-4e3e-b75e-edc55cd09956","Type":"ContainerStarted","Data":"f44f83a6fb27d30912427fc8c7b95e62d5ff7c515f3d78949d4df567135cd5ad"} Jul 19 16:38:47.797635 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:47.797617 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-c6rxm" event={"ID":"ad28ffaa-398f-40b5-afd5-7b168b27f96a","Type":"ContainerStarted","Data":"5ef12dc12078e718a8daaecc8a1a4187a2bdbee25f78edf694d98106b8f653ef"} Jul 19 16:38:47.802922 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:47.802887 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-cv2rk" podStartSLOduration=3.014741296 podStartE2EDuration="21.802876942s" totalImagesPullingTime="18.788135646s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 16:38:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 16:38:47.802794757 +0000 UTC m=+22.789370254" watchObservedRunningTime="2026-07-19 16:38:47.802876942 +0000 UTC m=+22.789452423" Jul 19 16:38:47.803027 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:47.803010 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-203.ec2.internal" podStartSLOduration=20.803003314 podStartE2EDuration="20.803003314s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 16:38:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 16:38:30.762725595 +0000 UTC m=+5.749301083" watchObservedRunningTime="2026-07-19 16:38:47.803003314 +0000 UTC m=+22.789578798" Jul 19 16:38:47.837663 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:47.837626 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-j6dwh" podStartSLOduration=3.056729741 podStartE2EDuration="21.837612502s" totalImagesPullingTime="18.780882761s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 16:38:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 16:38:47.837180233 +0000 UTC m=+22.823755716" watchObservedRunningTime="2026-07-19 16:38:47.837612502 +0000 UTC m=+22.824187986" Jul 19 16:38:47.850015 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:47.849980 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-vm6wh" podStartSLOduration=11.106338784 podStartE2EDuration="21.849965911s" totalImagesPullingTime="10.743627127s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 16:38:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 16:38:47.849787178 +0000 UTC m=+22.836362662" watchObservedRunningTime="2026-07-19 16:38:47.849965911 +0000 UTC m=+22.836541394" Jul 19 16:38:47.865518 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:47.865480 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-nqtmf" podStartSLOduration=3.031307832 podStartE2EDuration="21.865467559s" totalImagesPullingTime="18.834159727s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 16:38:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 16:38:47.865384016 +0000 UTC m=+22.851959499" watchObservedRunningTime="2026-07-19 16:38:47.865467559 +0000 UTC m=+22.852043043" Jul 19 16:38:48.596180 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:48.596140 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-smdwh" Jul 19 16:38:48.596461 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:48.596155 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrqmq" Jul 19 16:38:48.596461 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:48.596277 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-smdwh" podUID="ea859009-b3b3-4344-8d24-493c6a7547fa" Jul 19 16:38:48.596461 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:48.596345 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hrqmq" podUID="f85e47bc-bc7c-47cb-b107-69d6fefde837" Jul 19 16:38:48.805314 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:48.805285 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-86vh4" event={"ID":"4758d408-7ed8-465a-9d96-754c3c97336a","Type":"ContainerStarted","Data":"6d8fac153c213c1e6c280e015cede0204514521af057e3900b4d78a43579023d"} Jul 19 16:38:48.808685 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:48.808614 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" event={"ID":"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a","Type":"ContainerStarted","Data":"4c096e211f1f532def7ff4767bb4fbbaa62e9e89424d9f343ec5a28416d86e05"} Jul 19 16:38:48.808796 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:48.808699 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" event={"ID":"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a","Type":"ContainerStarted","Data":"e943a71b22523fe6202259746916b95ae4a74961ef291aee1f3ca046d7e6d32b"} Jul 19 16:38:48.808796 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:48.808750 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" event={"ID":"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a","Type":"ContainerStarted","Data":"27e524c1a3c70592576ef110b6673c9ecb09ae288b1a3d6c6eb48130a1fe0754"} Jul 19 16:38:48.819020 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:48.818964 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-c6rxm" podStartSLOduration=4.031651636 podStartE2EDuration="22.818944632s" totalImagesPullingTime="18.787292996s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 16:38:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 16:38:47.88584201 +0000 UTC m=+22.872417505" watchObservedRunningTime="2026-07-19 16:38:48.818944632 +0000 UTC m=+23.805520117" Jul 19 16:38:48.819228 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:48.819198 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-86vh4" podStartSLOduration=4.032455547 podStartE2EDuration="22.819187669s" totalImagesPullingTime="18.786732122s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 16:38:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 16:38:48.818639359 +0000 UTC m=+23.805214845" watchObservedRunningTime="2026-07-19 16:38:48.819187669 +0000 UTC m=+23.805763152" Jul 19 16:38:48.895105 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:48.895084 2583 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 19 16:38:49.501248 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:49.501155 2583 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-19T16:38:48.895100734Z","UUID":"0d08ac06-2451-484e-a90f-7d90ebb81539","Handler":null,"Name":"","Endpoint":""} Jul 19 16:38:49.502897 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:49.502868 2583 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 19 16:38:49.503041 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:49.502910 2583 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 19 16:38:49.525993 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:49.525955 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-vm6wh" Jul 19 16:38:49.526534 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:49.526482 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-vm6wh" Jul 19 16:38:49.595684 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:49.595636 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-thqxw" Jul 19 16:38:49.595860 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:49.595801 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-thqxw" podUID="9c487ff0-684c-41bd-aca1-74636fae5571" Jul 19 16:38:49.814544 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:49.814459 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c77zn" event={"ID":"cf9367af-33aa-4a8a-8c39-dfe2abe512b3","Type":"ContainerStarted","Data":"76510613aa163dcf4c2625c586157921ba07c0d94af5f7c0076667080d7936ee"} Jul 19 16:38:49.815105 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:49.814848 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-vm6wh" Jul 19 16:38:49.815373 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:49.815345 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-vm6wh" Jul 19 16:38:50.596369 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:50.596341 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-smdwh" Jul 19 16:38:50.596638 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:50.596441 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-smdwh" podUID="ea859009-b3b3-4344-8d24-493c6a7547fa" Jul 19 16:38:50.596638 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:50.596475 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrqmq" Jul 19 16:38:50.596638 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:50.596536 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hrqmq" podUID="f85e47bc-bc7c-47cb-b107-69d6fefde837" Jul 19 16:38:50.820900 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:50.820864 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" event={"ID":"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a","Type":"ContainerStarted","Data":"20508e83f581c1e63957bbefc8d47471e784ec821f72dbe8534517df5bb4b971"} Jul 19 16:38:50.823546 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:50.823468 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c77zn" event={"ID":"cf9367af-33aa-4a8a-8c39-dfe2abe512b3","Type":"ContainerStarted","Data":"cd96e0165e66e1a43947c3456d55d30efa44b326ab03885bfb36497b32ddbea1"} Jul 19 16:38:50.839724 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:50.839672 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c77zn" podStartSLOduration=3.682948301 podStartE2EDuration="24.839658511s" totalImagesPullingTime="21.15671021s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 16:38:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 16:38:50.839214701 +0000 UTC m=+25.825790185" watchObservedRunningTime="2026-07-19 16:38:50.839658511 +0000 UTC m=+25.826234004" Jul 19 16:38:51.596533 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:51.596501 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-thqxw" Jul 19 16:38:51.596732 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:51.596642 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-thqxw" podUID="9c487ff0-684c-41bd-aca1-74636fae5571" Jul 19 16:38:52.596740 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:52.596475 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrqmq" Jul 19 16:38:52.597595 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:52.596508 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-smdwh" Jul 19 16:38:52.597595 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:52.596800 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hrqmq" podUID="f85e47bc-bc7c-47cb-b107-69d6fefde837" Jul 19 16:38:52.597595 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:52.596853 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-smdwh" podUID="ea859009-b3b3-4344-8d24-493c6a7547fa" Jul 19 16:38:52.831192 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:52.831158 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" event={"ID":"6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a","Type":"ContainerStarted","Data":"27f53ba13d98f92fc98eb64cd2a9e2fa490d9602d88fdb95dc8b9666804d74ba"} Jul 19 16:38:52.831414 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:52.831400 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:52.831475 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:52.831454 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:52.833163 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:52.832988 2583 generic.go:356] "Generic (PLEG): container finished" podID="ab2a92d3-ad97-4dd4-9566-9b46f2f85de5" containerID="6afc6138954fba21df5f855852e0926911d86e8909b02933386a8e73e221b987" exitCode=0 Jul 19 16:38:52.833163 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:52.833027 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-spjzx" event={"ID":"ab2a92d3-ad97-4dd4-9566-9b46f2f85de5","Type":"ContainerDied","Data":"6afc6138954fba21df5f855852e0926911d86e8909b02933386a8e73e221b987"} Jul 19 16:38:52.846782 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:52.846731 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:52.854920 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:52.854883 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" podStartSLOduration=7.9960861659999996 podStartE2EDuration="26.854868318s" totalImagesPullingTime="18.858782152s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 16:38:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 16:38:52.853811076 +0000 UTC m=+27.840386560" watchObservedRunningTime="2026-07-19 16:38:52.854868318 +0000 UTC m=+27.841443796" Jul 19 16:38:53.596087 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:53.595972 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-thqxw" Jul 19 16:38:53.596427 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:53.596118 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-thqxw" podUID="9c487ff0-684c-41bd-aca1-74636fae5571" Jul 19 16:38:53.836214 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:53.836181 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:53.851159 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:53.851094 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:38:54.554763 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:54.554729 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-smdwh"] Jul 19 16:38:54.554995 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:54.554914 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-smdwh" Jul 19 16:38:54.555086 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:54.555023 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-smdwh" podUID="ea859009-b3b3-4344-8d24-493c6a7547fa" Jul 19 16:38:54.555439 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:54.555419 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-thqxw"] Jul 19 16:38:54.555511 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:54.555503 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-thqxw" Jul 19 16:38:54.555597 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:54.555581 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-thqxw" podUID="9c487ff0-684c-41bd-aca1-74636fae5571" Jul 19 16:38:54.557133 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:54.557084 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hrqmq"] Jul 19 16:38:54.557230 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:54.557176 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrqmq" Jul 19 16:38:54.557289 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:54.557267 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hrqmq" podUID="f85e47bc-bc7c-47cb-b107-69d6fefde837" Jul 19 16:38:54.840413 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:54.840382 2583 generic.go:356] "Generic (PLEG): container finished" podID="ab2a92d3-ad97-4dd4-9566-9b46f2f85de5" containerID="fa033d1a5e7d5adeb79a673523209e2913e2baf07e203f95bee03ca511cf8dab" exitCode=0 Jul 19 16:38:54.840953 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:54.840473 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-spjzx" event={"ID":"ab2a92d3-ad97-4dd4-9566-9b46f2f85de5","Type":"ContainerDied","Data":"fa033d1a5e7d5adeb79a673523209e2913e2baf07e203f95bee03ca511cf8dab"} Jul 19 16:38:56.596237 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:56.596004 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-smdwh" Jul 19 16:38:56.596600 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:56.596071 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-thqxw" Jul 19 16:38:56.596600 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:56.596261 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-smdwh" podUID="ea859009-b3b3-4344-8d24-493c6a7547fa" Jul 19 16:38:56.596600 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:56.596076 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrqmq" Jul 19 16:38:56.596600 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:56.596341 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-thqxw" podUID="9c487ff0-684c-41bd-aca1-74636fae5571" Jul 19 16:38:56.596600 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:56.596401 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hrqmq" podUID="f85e47bc-bc7c-47cb-b107-69d6fefde837" Jul 19 16:38:56.847298 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:56.847204 2583 generic.go:356] "Generic (PLEG): container finished" podID="ab2a92d3-ad97-4dd4-9566-9b46f2f85de5" containerID="2b5a0070388bf247cf2e1fb91c11d3e9bf408872da820664ccfaa6a38a468525" exitCode=0 Jul 19 16:38:56.847298 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:56.847249 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-spjzx" event={"ID":"ab2a92d3-ad97-4dd4-9566-9b46f2f85de5","Type":"ContainerDied","Data":"2b5a0070388bf247cf2e1fb91c11d3e9bf408872da820664ccfaa6a38a468525"} Jul 19 16:38:58.596367 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:58.596332 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrqmq" Jul 19 16:38:58.596968 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:58.596451 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-smdwh" Jul 19 16:38:58.596968 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:58.596466 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hrqmq" podUID="f85e47bc-bc7c-47cb-b107-69d6fefde837" Jul 19 16:38:58.596968 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:58.596535 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-smdwh" podUID="ea859009-b3b3-4344-8d24-493c6a7547fa" Jul 19 16:38:58.596968 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:58.596563 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-thqxw" Jul 19 16:38:58.596968 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:58.596666 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-thqxw" podUID="9c487ff0-684c-41bd-aca1-74636fae5571" Jul 19 16:38:59.225202 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:59.225173 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9c487ff0-684c-41bd-aca1-74636fae5571-metrics-certs\") pod \"network-metrics-daemon-thqxw\" (UID: \"9c487ff0-684c-41bd-aca1-74636fae5571\") " pod="openshift-multus/network-metrics-daemon-thqxw" Jul 19 16:38:59.225390 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:59.225215 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ea859009-b3b3-4344-8d24-493c6a7547fa-original-pull-secret\") pod \"global-pull-secret-syncer-smdwh\" (UID: \"ea859009-b3b3-4344-8d24-493c6a7547fa\") " pod="kube-system/global-pull-secret-syncer-smdwh" Jul 19 16:38:59.225390 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:59.225331 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 19 16:38:59.225390 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:59.225343 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 19 16:38:59.225390 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:59.225377 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea859009-b3b3-4344-8d24-493c6a7547fa-original-pull-secret podName:ea859009-b3b3-4344-8d24-493c6a7547fa nodeName:}" failed. No retries permitted until 2026-07-19 16:39:31.225365334 +0000 UTC m=+66.211940796 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ea859009-b3b3-4344-8d24-493c6a7547fa-original-pull-secret") pod "global-pull-secret-syncer-smdwh" (UID: "ea859009-b3b3-4344-8d24-493c6a7547fa") : object "kube-system"/"original-pull-secret" not registered Jul 19 16:38:59.225564 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:59.225405 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c487ff0-684c-41bd-aca1-74636fae5571-metrics-certs podName:9c487ff0-684c-41bd-aca1-74636fae5571 nodeName:}" failed. No retries permitted until 2026-07-19 16:39:31.225386003 +0000 UTC m=+66.211961469 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9c487ff0-684c-41bd-aca1-74636fae5571-metrics-certs") pod "network-metrics-daemon-thqxw" (UID: "9c487ff0-684c-41bd-aca1-74636fae5571") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 19 16:38:59.325518 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:38:59.325487 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9n7h\" (UniqueName: \"kubernetes.io/projected/f85e47bc-bc7c-47cb-b107-69d6fefde837-kube-api-access-f9n7h\") pod \"network-check-target-hrqmq\" (UID: \"f85e47bc-bc7c-47cb-b107-69d6fefde837\") " pod="openshift-network-diagnostics/network-check-target-hrqmq" Jul 19 16:38:59.325692 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:59.325623 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 19 16:38:59.325692 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:59.325649 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 19 16:38:59.325692 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:59.325681 2583 projected.go:196] Error preparing data for projected volume kube-api-access-f9n7h for pod openshift-network-diagnostics/network-check-target-hrqmq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 19 16:38:59.325808 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:38:59.325739 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f85e47bc-bc7c-47cb-b107-69d6fefde837-kube-api-access-f9n7h podName:f85e47bc-bc7c-47cb-b107-69d6fefde837 nodeName:}" failed. No retries permitted until 2026-07-19 16:39:31.325723695 +0000 UTC m=+66.312299159 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-f9n7h" (UniqueName: "kubernetes.io/projected/f85e47bc-bc7c-47cb-b107-69d6fefde837-kube-api-access-f9n7h") pod "network-check-target-hrqmq" (UID: "f85e47bc-bc7c-47cb-b107-69d6fefde837") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 19 16:39:00.341994 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:00.341918 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-203.ec2.internal" event="NodeReady" Jul 19 16:39:00.342532 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:00.342108 2583 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 19 16:39:00.390403 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:00.389038 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-znbhx"] Jul 19 16:39:00.416202 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:00.416173 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-txwnf"] Jul 19 16:39:00.416371 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:00.416295 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-znbhx" Jul 19 16:39:00.418679 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:00.418650 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 19 16:39:00.418816 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:00.418769 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-pk4kh\"" Jul 19 16:39:00.418950 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:00.418934 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 19 16:39:00.434841 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:00.434814 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e1a462c-d8cc-4635-9eac-2271aed8e352-tmp-dir\") pod \"dns-default-znbhx\" (UID: \"9e1a462c-d8cc-4635-9eac-2271aed8e352\") " pod="openshift-dns/dns-default-znbhx" Jul 19 16:39:00.434958 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:00.434864 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9e1a462c-d8cc-4635-9eac-2271aed8e352-config-volume\") pod \"dns-default-znbhx\" (UID: \"9e1a462c-d8cc-4635-9eac-2271aed8e352\") " pod="openshift-dns/dns-default-znbhx" Jul 19 16:39:00.434958 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:00.434922 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmpjx\" (UniqueName: \"kubernetes.io/projected/9e1a462c-d8cc-4635-9eac-2271aed8e352-kube-api-access-kmpjx\") pod \"dns-default-znbhx\" (UID: \"9e1a462c-d8cc-4635-9eac-2271aed8e352\") " pod="openshift-dns/dns-default-znbhx" Jul 19 16:39:00.434958 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:00.434948 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e1a462c-d8cc-4635-9eac-2271aed8e352-metrics-tls\") pod \"dns-default-znbhx\" (UID: \"9e1a462c-d8cc-4635-9eac-2271aed8e352\") " pod="openshift-dns/dns-default-znbhx" Jul 19 16:39:00.442391 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:00.442373 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-znbhx"] Jul 19 16:39:00.442391 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:00.442401 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-txwnf"] Jul 19 16:39:00.442545 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:00.442478 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-txwnf" Jul 19 16:39:00.447147 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:00.447126 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 19 16:39:00.447256 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:00.447159 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-wcrqr\"" Jul 19 16:39:00.447845 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:00.447711 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 19 16:39:00.447845 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:00.447711 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 19 16:39:00.535408 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:00.535369 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pb2kv\" (UniqueName: \"kubernetes.io/projected/f133de90-2db7-4454-9d67-26e266dec955-kube-api-access-pb2kv\") pod \"ingress-canary-txwnf\" (UID: \"f133de90-2db7-4454-9d67-26e266dec955\") " pod="openshift-ingress-canary/ingress-canary-txwnf" Jul 19 16:39:00.535575 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:00.535431 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9e1a462c-d8cc-4635-9eac-2271aed8e352-config-volume\") pod \"dns-default-znbhx\" (UID: \"9e1a462c-d8cc-4635-9eac-2271aed8e352\") " pod="openshift-dns/dns-default-znbhx" Jul 19 16:39:00.535575 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:00.535483 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmpjx\" (UniqueName: \"kubernetes.io/projected/9e1a462c-d8cc-4635-9eac-2271aed8e352-kube-api-access-kmpjx\") pod \"dns-default-znbhx\" (UID: \"9e1a462c-d8cc-4635-9eac-2271aed8e352\") " pod="openshift-dns/dns-default-znbhx" Jul 19 16:39:00.535575 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:00.535510 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e1a462c-d8cc-4635-9eac-2271aed8e352-metrics-tls\") pod \"dns-default-znbhx\" (UID: \"9e1a462c-d8cc-4635-9eac-2271aed8e352\") " pod="openshift-dns/dns-default-znbhx" Jul 19 16:39:00.535728 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:00.535575 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e1a462c-d8cc-4635-9eac-2271aed8e352-tmp-dir\") pod \"dns-default-znbhx\" (UID: \"9e1a462c-d8cc-4635-9eac-2271aed8e352\") " pod="openshift-dns/dns-default-znbhx" Jul 19 16:39:00.535728 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:00.535597 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f133de90-2db7-4454-9d67-26e266dec955-cert\") pod \"ingress-canary-txwnf\" (UID: \"f133de90-2db7-4454-9d67-26e266dec955\") " pod="openshift-ingress-canary/ingress-canary-txwnf" Jul 19 16:39:00.535728 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:39:00.535679 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 19 16:39:00.535835 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:39:00.535776 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e1a462c-d8cc-4635-9eac-2271aed8e352-metrics-tls podName:9e1a462c-d8cc-4635-9eac-2271aed8e352 nodeName:}" failed. No retries permitted until 2026-07-19 16:39:01.035751789 +0000 UTC m=+36.022327251 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9e1a462c-d8cc-4635-9eac-2271aed8e352-metrics-tls") pod "dns-default-znbhx" (UID: "9e1a462c-d8cc-4635-9eac-2271aed8e352") : secret "dns-default-metrics-tls" not found Jul 19 16:39:00.535934 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:00.535916 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e1a462c-d8cc-4635-9eac-2271aed8e352-tmp-dir\") pod \"dns-default-znbhx\" (UID: \"9e1a462c-d8cc-4635-9eac-2271aed8e352\") " pod="openshift-dns/dns-default-znbhx" Jul 19 16:39:00.536157 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:00.536134 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9e1a462c-d8cc-4635-9eac-2271aed8e352-config-volume\") pod \"dns-default-znbhx\" (UID: \"9e1a462c-d8cc-4635-9eac-2271aed8e352\") " pod="openshift-dns/dns-default-znbhx" Jul 19 16:39:00.546392 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:00.546365 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmpjx\" (UniqueName: \"kubernetes.io/projected/9e1a462c-d8cc-4635-9eac-2271aed8e352-kube-api-access-kmpjx\") pod \"dns-default-znbhx\" (UID: \"9e1a462c-d8cc-4635-9eac-2271aed8e352\") " pod="openshift-dns/dns-default-znbhx" Jul 19 16:39:00.595584 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:00.595498 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrqmq" Jul 19 16:39:00.595584 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:00.595521 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-smdwh" Jul 19 16:39:00.595584 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:00.595540 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-thqxw" Jul 19 16:39:00.597823 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:00.597787 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 19 16:39:00.597971 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:00.597791 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 19 16:39:00.598074 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:00.597983 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 19 16:39:00.598149 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:00.598116 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pkhnw\"" Jul 19 16:39:00.598249 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:00.598231 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-sh2cq\"" Jul 19 16:39:00.598291 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:00.598232 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 19 16:39:00.636286 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:00.636264 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f133de90-2db7-4454-9d67-26e266dec955-cert\") pod \"ingress-canary-txwnf\" (UID: \"f133de90-2db7-4454-9d67-26e266dec955\") " pod="openshift-ingress-canary/ingress-canary-txwnf" Jul 19 16:39:00.636420 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:00.636293 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pb2kv\" (UniqueName: \"kubernetes.io/projected/f133de90-2db7-4454-9d67-26e266dec955-kube-api-access-pb2kv\") pod \"ingress-canary-txwnf\" (UID: \"f133de90-2db7-4454-9d67-26e266dec955\") " pod="openshift-ingress-canary/ingress-canary-txwnf" Jul 19 16:39:00.638918 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:00.638898 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f133de90-2db7-4454-9d67-26e266dec955-cert\") pod \"ingress-canary-txwnf\" (UID: \"f133de90-2db7-4454-9d67-26e266dec955\") " pod="openshift-ingress-canary/ingress-canary-txwnf" Jul 19 16:39:00.644309 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:00.644282 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pb2kv\" (UniqueName: \"kubernetes.io/projected/f133de90-2db7-4454-9d67-26e266dec955-kube-api-access-pb2kv\") pod \"ingress-canary-txwnf\" (UID: \"f133de90-2db7-4454-9d67-26e266dec955\") " pod="openshift-ingress-canary/ingress-canary-txwnf" Jul 19 16:39:00.752437 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:00.752398 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-txwnf" Jul 19 16:39:01.039602 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:01.039572 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e1a462c-d8cc-4635-9eac-2271aed8e352-metrics-tls\") pod \"dns-default-znbhx\" (UID: \"9e1a462c-d8cc-4635-9eac-2271aed8e352\") " pod="openshift-dns/dns-default-znbhx" Jul 19 16:39:01.039716 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:39:01.039683 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 19 16:39:01.039750 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:39:01.039738 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e1a462c-d8cc-4635-9eac-2271aed8e352-metrics-tls podName:9e1a462c-d8cc-4635-9eac-2271aed8e352 nodeName:}" failed. No retries permitted until 2026-07-19 16:39:02.039723473 +0000 UTC m=+37.026298953 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9e1a462c-d8cc-4635-9eac-2271aed8e352-metrics-tls") pod "dns-default-znbhx" (UID: "9e1a462c-d8cc-4635-9eac-2271aed8e352") : secret "dns-default-metrics-tls" not found Jul 19 16:39:01.089442 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:01.089413 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-txwnf"] Jul 19 16:39:01.154336 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:39:01.154305 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf133de90_2db7_4454_9d67_26e266dec955.slice/crio-f2eef960a6ec1ff153967b456aa41d0e7d8e70e295cfdbf6c3561db59157e79a WatchSource:0}: Error finding container f2eef960a6ec1ff153967b456aa41d0e7d8e70e295cfdbf6c3561db59157e79a: Status 404 returned error can't find the container with id f2eef960a6ec1ff153967b456aa41d0e7d8e70e295cfdbf6c3561db59157e79a Jul 19 16:39:01.863030 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:01.862993 2583 generic.go:356] "Generic (PLEG): container finished" podID="ab2a92d3-ad97-4dd4-9566-9b46f2f85de5" containerID="de4c06ff4efa2261061753fe565ceef899dcbf9373d573339284834ca15af32a" exitCode=0 Jul 19 16:39:01.863685 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:01.863093 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-spjzx" event={"ID":"ab2a92d3-ad97-4dd4-9566-9b46f2f85de5","Type":"ContainerDied","Data":"de4c06ff4efa2261061753fe565ceef899dcbf9373d573339284834ca15af32a"} Jul 19 16:39:01.864487 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:01.864452 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-txwnf" event={"ID":"f133de90-2db7-4454-9d67-26e266dec955","Type":"ContainerStarted","Data":"f2eef960a6ec1ff153967b456aa41d0e7d8e70e295cfdbf6c3561db59157e79a"} Jul 19 16:39:02.045842 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:02.045814 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e1a462c-d8cc-4635-9eac-2271aed8e352-metrics-tls\") pod \"dns-default-znbhx\" (UID: \"9e1a462c-d8cc-4635-9eac-2271aed8e352\") " pod="openshift-dns/dns-default-znbhx" Jul 19 16:39:02.046009 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:39:02.045991 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 19 16:39:02.046075 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:39:02.046066 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e1a462c-d8cc-4635-9eac-2271aed8e352-metrics-tls podName:9e1a462c-d8cc-4635-9eac-2271aed8e352 nodeName:}" failed. No retries permitted until 2026-07-19 16:39:04.046039033 +0000 UTC m=+39.032614495 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9e1a462c-d8cc-4635-9eac-2271aed8e352-metrics-tls") pod "dns-default-znbhx" (UID: "9e1a462c-d8cc-4635-9eac-2271aed8e352") : secret "dns-default-metrics-tls" not found Jul 19 16:39:02.871546 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:02.871506 2583 generic.go:356] "Generic (PLEG): container finished" podID="ab2a92d3-ad97-4dd4-9566-9b46f2f85de5" containerID="01474358457ee23930fef7f4ddb00db5491829dcbf39699895c24e73de628460" exitCode=0 Jul 19 16:39:02.871999 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:02.871585 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-spjzx" event={"ID":"ab2a92d3-ad97-4dd4-9566-9b46f2f85de5","Type":"ContainerDied","Data":"01474358457ee23930fef7f4ddb00db5491829dcbf39699895c24e73de628460"} Jul 19 16:39:03.875140 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:03.874918 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-txwnf" event={"ID":"f133de90-2db7-4454-9d67-26e266dec955","Type":"ContainerStarted","Data":"65b5adf074148ef8da7fcf3eee12ce81432b4c7dd91d7b00f6cd4cf4c6cdf90a"} Jul 19 16:39:03.878239 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:03.878207 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-spjzx" event={"ID":"ab2a92d3-ad97-4dd4-9566-9b46f2f85de5","Type":"ContainerStarted","Data":"bd4e842f09fd5cd5ddad7aa7e4a6ab9228ad50e53b9bc68791716b1aa339898b"} Jul 19 16:39:03.890152 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:03.890109 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-txwnf" podStartSLOduration=1.693454105 podStartE2EDuration="3.890094891s" totalImagesPullingTime="2.196640786s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 16:39:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 16:39:03.888988456 +0000 UTC m=+38.875563951" watchObservedRunningTime="2026-07-19 16:39:03.890094891 +0000 UTC m=+38.876670374" Jul 19 16:39:03.911931 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:03.911870 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-spjzx" podStartSLOduration=8.086983706 podStartE2EDuration="37.911848093s" totalImagesPullingTime="29.824864387s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 16:38:26 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 16:39:03.910182507 +0000 UTC m=+38.896757992" watchObservedRunningTime="2026-07-19 16:39:03.911848093 +0000 UTC m=+38.898423578" Jul 19 16:39:04.058144 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:04.058100 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e1a462c-d8cc-4635-9eac-2271aed8e352-metrics-tls\") pod \"dns-default-znbhx\" (UID: \"9e1a462c-d8cc-4635-9eac-2271aed8e352\") " pod="openshift-dns/dns-default-znbhx" Jul 19 16:39:04.058316 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:39:04.058248 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 19 16:39:04.058316 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:39:04.058309 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e1a462c-d8cc-4635-9eac-2271aed8e352-metrics-tls podName:9e1a462c-d8cc-4635-9eac-2271aed8e352 nodeName:}" failed. No retries permitted until 2026-07-19 16:39:08.058293246 +0000 UTC m=+43.044868716 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9e1a462c-d8cc-4635-9eac-2271aed8e352-metrics-tls") pod "dns-default-znbhx" (UID: "9e1a462c-d8cc-4635-9eac-2271aed8e352") : secret "dns-default-metrics-tls" not found Jul 19 16:39:08.087458 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:08.087413 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e1a462c-d8cc-4635-9eac-2271aed8e352-metrics-tls\") pod \"dns-default-znbhx\" (UID: \"9e1a462c-d8cc-4635-9eac-2271aed8e352\") " pod="openshift-dns/dns-default-znbhx" Jul 19 16:39:08.087920 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:39:08.087572 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 19 16:39:08.087920 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:39:08.087644 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e1a462c-d8cc-4635-9eac-2271aed8e352-metrics-tls podName:9e1a462c-d8cc-4635-9eac-2271aed8e352 nodeName:}" failed. No retries permitted until 2026-07-19 16:39:16.087627136 +0000 UTC m=+51.074202599 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9e1a462c-d8cc-4635-9eac-2271aed8e352-metrics-tls") pod "dns-default-znbhx" (UID: "9e1a462c-d8cc-4635-9eac-2271aed8e352") : secret "dns-default-metrics-tls" not found Jul 19 16:39:16.141424 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:16.141379 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e1a462c-d8cc-4635-9eac-2271aed8e352-metrics-tls\") pod \"dns-default-znbhx\" (UID: \"9e1a462c-d8cc-4635-9eac-2271aed8e352\") " pod="openshift-dns/dns-default-znbhx" Jul 19 16:39:16.141901 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:39:16.141526 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 19 16:39:16.141901 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:39:16.141590 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e1a462c-d8cc-4635-9eac-2271aed8e352-metrics-tls podName:9e1a462c-d8cc-4635-9eac-2271aed8e352 nodeName:}" failed. No retries permitted until 2026-07-19 16:39:32.141574376 +0000 UTC m=+67.128149838 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9e1a462c-d8cc-4635-9eac-2271aed8e352-metrics-tls") pod "dns-default-znbhx" (UID: "9e1a462c-d8cc-4635-9eac-2271aed8e352") : secret "dns-default-metrics-tls" not found Jul 19 16:39:25.852765 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:25.852738 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mt8d2" Jul 19 16:39:31.247564 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:31.247524 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ea859009-b3b3-4344-8d24-493c6a7547fa-original-pull-secret\") pod \"global-pull-secret-syncer-smdwh\" (UID: \"ea859009-b3b3-4344-8d24-493c6a7547fa\") " pod="kube-system/global-pull-secret-syncer-smdwh" Jul 19 16:39:31.248051 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:31.247604 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9c487ff0-684c-41bd-aca1-74636fae5571-metrics-certs\") pod \"network-metrics-daemon-thqxw\" (UID: \"9c487ff0-684c-41bd-aca1-74636fae5571\") " pod="openshift-multus/network-metrics-daemon-thqxw" Jul 19 16:39:31.250558 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:31.250531 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 19 16:39:31.250675 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:31.250573 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 19 16:39:31.258113 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:39:31.258095 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 19 16:39:31.258193 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:39:31.258171 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c487ff0-684c-41bd-aca1-74636fae5571-metrics-certs podName:9c487ff0-684c-41bd-aca1-74636fae5571 nodeName:}" failed. No retries permitted until 2026-07-19 16:40:35.258153441 +0000 UTC m=+130.244728908 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9c487ff0-684c-41bd-aca1-74636fae5571-metrics-certs") pod "network-metrics-daemon-thqxw" (UID: "9c487ff0-684c-41bd-aca1-74636fae5571") : secret "metrics-daemon-secret" not found Jul 19 16:39:31.261758 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:31.261738 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ea859009-b3b3-4344-8d24-493c6a7547fa-original-pull-secret\") pod \"global-pull-secret-syncer-smdwh\" (UID: \"ea859009-b3b3-4344-8d24-493c6a7547fa\") " pod="kube-system/global-pull-secret-syncer-smdwh" Jul 19 16:39:31.348591 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:31.348552 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9n7h\" (UniqueName: \"kubernetes.io/projected/f85e47bc-bc7c-47cb-b107-69d6fefde837-kube-api-access-f9n7h\") pod \"network-check-target-hrqmq\" (UID: \"f85e47bc-bc7c-47cb-b107-69d6fefde837\") " pod="openshift-network-diagnostics/network-check-target-hrqmq" Jul 19 16:39:31.351132 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:31.351116 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 19 16:39:31.361565 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:31.361548 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 19 16:39:31.372613 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:31.372590 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9n7h\" (UniqueName: \"kubernetes.io/projected/f85e47bc-bc7c-47cb-b107-69d6fefde837-kube-api-access-f9n7h\") pod \"network-check-target-hrqmq\" (UID: \"f85e47bc-bc7c-47cb-b107-69d6fefde837\") " pod="openshift-network-diagnostics/network-check-target-hrqmq" Jul 19 16:39:31.511136 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:31.511041 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-sh2cq\"" Jul 19 16:39:31.515746 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:31.515729 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-smdwh" Jul 19 16:39:31.519709 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:31.519690 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrqmq" Jul 19 16:39:31.644734 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:31.644701 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hrqmq"] Jul 19 16:39:31.647441 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:39:31.647413 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf85e47bc_bc7c_47cb_b107_69d6fefde837.slice/crio-74817c88c62c6c69fadf799a7e349f22f773f95d5e76f423b95b7e28332ffcd7 WatchSource:0}: Error finding container 74817c88c62c6c69fadf799a7e349f22f773f95d5e76f423b95b7e28332ffcd7: Status 404 returned error can't find the container with id 74817c88c62c6c69fadf799a7e349f22f773f95d5e76f423b95b7e28332ffcd7 Jul 19 16:39:31.659153 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:31.659119 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-smdwh"] Jul 19 16:39:31.663176 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:39:31.663150 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea859009_b3b3_4344_8d24_493c6a7547fa.slice/crio-7f5e311c3f42cac79f5b0b2dc0fc0b09864d1c1b5ef5cf3fb68af34c73fa2220 WatchSource:0}: Error finding container 7f5e311c3f42cac79f5b0b2dc0fc0b09864d1c1b5ef5cf3fb68af34c73fa2220: Status 404 returned error can't find the container with id 7f5e311c3f42cac79f5b0b2dc0fc0b09864d1c1b5ef5cf3fb68af34c73fa2220 Jul 19 16:39:31.941257 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:31.941224 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hrqmq" event={"ID":"f85e47bc-bc7c-47cb-b107-69d6fefde837","Type":"ContainerStarted","Data":"74817c88c62c6c69fadf799a7e349f22f773f95d5e76f423b95b7e28332ffcd7"} Jul 19 16:39:31.942411 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:31.942374 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-smdwh" event={"ID":"ea859009-b3b3-4344-8d24-493c6a7547fa","Type":"ContainerStarted","Data":"7f5e311c3f42cac79f5b0b2dc0fc0b09864d1c1b5ef5cf3fb68af34c73fa2220"} Jul 19 16:39:32.155231 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:32.155184 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e1a462c-d8cc-4635-9eac-2271aed8e352-metrics-tls\") pod \"dns-default-znbhx\" (UID: \"9e1a462c-d8cc-4635-9eac-2271aed8e352\") " pod="openshift-dns/dns-default-znbhx" Jul 19 16:39:32.155409 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:39:32.155328 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 19 16:39:32.155409 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:39:32.155389 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e1a462c-d8cc-4635-9eac-2271aed8e352-metrics-tls podName:9e1a462c-d8cc-4635-9eac-2271aed8e352 nodeName:}" failed. No retries permitted until 2026-07-19 16:40:04.15537416 +0000 UTC m=+99.141949623 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9e1a462c-d8cc-4635-9eac-2271aed8e352-metrics-tls") pod "dns-default-znbhx" (UID: "9e1a462c-d8cc-4635-9eac-2271aed8e352") : secret "dns-default-metrics-tls" not found Jul 19 16:39:36.958210 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:36.958170 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hrqmq" event={"ID":"f85e47bc-bc7c-47cb-b107-69d6fefde837","Type":"ContainerStarted","Data":"d6b6ba0e615e8669f3d9c80b77e1c6dcad8cbaeb477c41ff914ea1ca958feec3"} Jul 19 16:39:36.958705 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:36.958306 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-hrqmq" Jul 19 16:39:36.959765 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:36.959735 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-smdwh" event={"ID":"ea859009-b3b3-4344-8d24-493c6a7547fa","Type":"ContainerStarted","Data":"654e2589cdfe4ca44f571268e9526066f83101a644e8496d3f50218b4a96775d"} Jul 19 16:39:36.972505 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:36.972465 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-hrqmq" podStartSLOduration=66.265771407 podStartE2EDuration="1m10.972452898s" totalImagesPullingTime="4.706681491s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 16:38:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 16:39:36.972235821 +0000 UTC m=+71.958811323" watchObservedRunningTime="2026-07-19 16:39:36.972452898 +0000 UTC m=+71.959028381" Jul 19 16:39:36.985964 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:39:36.985914 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-smdwh" podStartSLOduration=66.284320124 podStartE2EDuration="1m10.985896806s" totalImagesPullingTime="4.701576682s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 16:38:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 16:39:36.985530045 +0000 UTC m=+71.972105531" watchObservedRunningTime="2026-07-19 16:39:36.985896806 +0000 UTC m=+71.972472291" Jul 19 16:40:04.162307 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:40:04.162202 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e1a462c-d8cc-4635-9eac-2271aed8e352-metrics-tls\") pod \"dns-default-znbhx\" (UID: \"9e1a462c-d8cc-4635-9eac-2271aed8e352\") " pod="openshift-dns/dns-default-znbhx" Jul 19 16:40:04.162700 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:40:04.162379 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 19 16:40:04.162700 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:40:04.162482 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e1a462c-d8cc-4635-9eac-2271aed8e352-metrics-tls podName:9e1a462c-d8cc-4635-9eac-2271aed8e352 nodeName:}" failed. No retries permitted until 2026-07-19 16:41:08.162462312 +0000 UTC m=+163.149037775 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9e1a462c-d8cc-4635-9eac-2271aed8e352-metrics-tls") pod "dns-default-znbhx" (UID: "9e1a462c-d8cc-4635-9eac-2271aed8e352") : secret "dns-default-metrics-tls" not found Jul 19 16:40:05.671496 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:40:05.671467 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86d8cd66b8-g6v7k"] Jul 19 16:40:05.673938 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:40:05.673911 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86d8cd66b8-g6v7k" Jul 19 16:40:05.677076 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:40:05.677041 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-zqgzf\"" Jul 19 16:40:05.677210 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:40:05.677047 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 19 16:40:05.677210 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:40:05.677070 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 19 16:40:05.677210 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:40:05.677072 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 19 16:40:05.677210 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:40:05.677082 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 19 16:40:05.682253 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:40:05.682234 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86d8cd66b8-g6v7k"] Jul 19 16:40:05.774604 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:40:05.774565 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-th9nk\" (UniqueName: \"kubernetes.io/projected/2113bfe9-6335-4248-a8c7-31de65166555-kube-api-access-th9nk\") pod \"managed-serviceaccount-addon-agent-86d8cd66b8-g6v7k\" (UID: \"2113bfe9-6335-4248-a8c7-31de65166555\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86d8cd66b8-g6v7k" Jul 19 16:40:05.774604 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:40:05.774606 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2113bfe9-6335-4248-a8c7-31de65166555-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-86d8cd66b8-g6v7k\" (UID: \"2113bfe9-6335-4248-a8c7-31de65166555\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86d8cd66b8-g6v7k" Jul 19 16:40:05.875214 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:40:05.875177 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-th9nk\" (UniqueName: \"kubernetes.io/projected/2113bfe9-6335-4248-a8c7-31de65166555-kube-api-access-th9nk\") pod \"managed-serviceaccount-addon-agent-86d8cd66b8-g6v7k\" (UID: \"2113bfe9-6335-4248-a8c7-31de65166555\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86d8cd66b8-g6v7k" Jul 19 16:40:05.875299 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:40:05.875221 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2113bfe9-6335-4248-a8c7-31de65166555-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-86d8cd66b8-g6v7k\" (UID: \"2113bfe9-6335-4248-a8c7-31de65166555\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86d8cd66b8-g6v7k" Jul 19 16:40:05.877843 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:40:05.877809 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2113bfe9-6335-4248-a8c7-31de65166555-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-86d8cd66b8-g6v7k\" (UID: \"2113bfe9-6335-4248-a8c7-31de65166555\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86d8cd66b8-g6v7k" Jul 19 16:40:05.882605 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:40:05.882579 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-th9nk\" (UniqueName: \"kubernetes.io/projected/2113bfe9-6335-4248-a8c7-31de65166555-kube-api-access-th9nk\") pod \"managed-serviceaccount-addon-agent-86d8cd66b8-g6v7k\" (UID: \"2113bfe9-6335-4248-a8c7-31de65166555\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86d8cd66b8-g6v7k" Jul 19 16:40:05.991893 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:40:05.991788 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86d8cd66b8-g6v7k" Jul 19 16:40:06.113723 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:40:06.113691 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86d8cd66b8-g6v7k"] Jul 19 16:40:06.117114 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:40:06.117080 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2113bfe9_6335_4248_a8c7_31de65166555.slice/crio-578dd525f301e4bce62346d06948bea457293c04ecfeb17352a53faccf3e6a40 WatchSource:0}: Error finding container 578dd525f301e4bce62346d06948bea457293c04ecfeb17352a53faccf3e6a40: Status 404 returned error can't find the container with id 578dd525f301e4bce62346d06948bea457293c04ecfeb17352a53faccf3e6a40 Jul 19 16:40:07.032386 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:40:07.032245 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86d8cd66b8-g6v7k" event={"ID":"2113bfe9-6335-4248-a8c7-31de65166555","Type":"ContainerStarted","Data":"578dd525f301e4bce62346d06948bea457293c04ecfeb17352a53faccf3e6a40"} Jul 19 16:40:07.965035 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:40:07.964997 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-hrqmq" Jul 19 16:40:09.039446 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:40:09.039411 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86d8cd66b8-g6v7k" event={"ID":"2113bfe9-6335-4248-a8c7-31de65166555","Type":"ContainerStarted","Data":"1d797d21c4605da0ecf6988e395c398f01e680c129aa17c88a2ba46ac3d31ebc"} Jul 19 16:40:09.053848 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:40:09.053682 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86d8cd66b8-g6v7k" podStartSLOduration=1.78318085 podStartE2EDuration="4.053667028s" totalImagesPullingTime="2.270486178s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 16:40:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 16:40:09.053305775 +0000 UTC m=+104.039881259" watchObservedRunningTime="2026-07-19 16:40:09.053667028 +0000 UTC m=+104.040242511" Jul 19 16:40:35.275669 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:40:35.275613 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9c487ff0-684c-41bd-aca1-74636fae5571-metrics-certs\") pod \"network-metrics-daemon-thqxw\" (UID: \"9c487ff0-684c-41bd-aca1-74636fae5571\") " pod="openshift-multus/network-metrics-daemon-thqxw" Jul 19 16:40:35.276303 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:40:35.275785 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 19 16:40:35.276303 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:40:35.275914 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c487ff0-684c-41bd-aca1-74636fae5571-metrics-certs podName:9c487ff0-684c-41bd-aca1-74636fae5571 nodeName:}" failed. No retries permitted until 2026-07-19 16:42:37.275892037 +0000 UTC m=+252.262467499 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9c487ff0-684c-41bd-aca1-74636fae5571-metrics-certs") pod "network-metrics-daemon-thqxw" (UID: "9c487ff0-684c-41bd-aca1-74636fae5571") : secret "metrics-daemon-secret" not found Jul 19 16:41:03.427445 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:41:03.427378 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-znbhx" podUID="9e1a462c-d8cc-4635-9eac-2271aed8e352" Jul 19 16:41:03.620723 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:41:03.620666 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-thqxw" podUID="9c487ff0-684c-41bd-aca1-74636fae5571" Jul 19 16:41:04.163310 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:41:04.163231 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-znbhx" Jul 19 16:41:08.200724 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:41:08.200667 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e1a462c-d8cc-4635-9eac-2271aed8e352-metrics-tls\") pod \"dns-default-znbhx\" (UID: \"9e1a462c-d8cc-4635-9eac-2271aed8e352\") " pod="openshift-dns/dns-default-znbhx" Jul 19 16:41:08.201228 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:41:08.200811 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 19 16:41:08.201228 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:41:08.200884 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e1a462c-d8cc-4635-9eac-2271aed8e352-metrics-tls podName:9e1a462c-d8cc-4635-9eac-2271aed8e352 nodeName:}" failed. No retries permitted until 2026-07-19 16:43:10.200867416 +0000 UTC m=+285.187442879 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9e1a462c-d8cc-4635-9eac-2271aed8e352-metrics-tls") pod "dns-default-znbhx" (UID: "9e1a462c-d8cc-4635-9eac-2271aed8e352") : secret "dns-default-metrics-tls" not found Jul 19 16:41:16.595828 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:41:16.595791 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-thqxw" Jul 19 16:42:09.314273 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:09.314235 2583 generic.go:356] "Generic (PLEG): container finished" podID="2113bfe9-6335-4248-a8c7-31de65166555" containerID="1d797d21c4605da0ecf6988e395c398f01e680c129aa17c88a2ba46ac3d31ebc" exitCode=255 Jul 19 16:42:09.314780 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:09.314298 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86d8cd66b8-g6v7k" event={"ID":"2113bfe9-6335-4248-a8c7-31de65166555","Type":"ContainerDied","Data":"1d797d21c4605da0ecf6988e395c398f01e680c129aa17c88a2ba46ac3d31ebc"} Jul 19 16:42:09.319922 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:09.319903 2583 scope.go:117] "RemoveContainer" containerID="1d797d21c4605da0ecf6988e395c398f01e680c129aa17c88a2ba46ac3d31ebc" Jul 19 16:42:10.317831 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:10.317798 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86d8cd66b8-g6v7k" event={"ID":"2113bfe9-6335-4248-a8c7-31de65166555","Type":"ContainerStarted","Data":"3f475d812c37e69ce245c1102355e9522ef6682a1d9fc6699bbf2d50795e2fb5"} Jul 19 16:42:24.621493 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.621455 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-p9vzx"] Jul 19 16:42:24.624339 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.624317 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-p9vzx" Jul 19 16:42:24.630727 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.627532 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 19 16:42:24.630727 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.627817 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 19 16:42:24.630727 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.628344 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 19 16:42:24.630727 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.628789 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-qs9kw\"" Jul 19 16:42:24.634855 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.634834 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-p9vzx"] Jul 19 16:42:24.705690 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.705657 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-9w62k"] Jul 19 16:42:24.708525 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.708508 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-9w62k" Jul 19 16:42:24.710390 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.710371 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-g7x27\"" Jul 19 16:42:24.712368 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.712324 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4248ffdc-b491-4236-9d08-af8a40e27903-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-p9vzx\" (UID: \"4248ffdc-b491-4236-9d08-af8a40e27903\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-p9vzx" Jul 19 16:42:24.712488 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.712457 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqrhm\" (UniqueName: \"kubernetes.io/projected/4248ffdc-b491-4236-9d08-af8a40e27903-kube-api-access-wqrhm\") pod \"cluster-samples-operator-79b7c55f87-p9vzx\" (UID: \"4248ffdc-b491-4236-9d08-af8a40e27903\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-p9vzx" Jul 19 16:42:24.713210 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.713191 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-d2qfz"] Jul 19 16:42:24.716720 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.716703 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-755474c78d-xctf5"] Jul 19 16:42:24.716841 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.716825 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-d2qfz" Jul 19 16:42:24.718792 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.718770 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 19 16:42:24.719190 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.719169 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-qqp5c\"" Jul 19 16:42:24.719297 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.719263 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 19 16:42:24.719518 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.719498 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 19 16:42:24.719578 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.719566 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 19 16:42:24.719723 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.719704 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5c456d485-qvzgh"] Jul 19 16:42:24.719804 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.719789 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-755474c78d-xctf5" Jul 19 16:42:24.721660 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.721645 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 19 16:42:24.721909 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.721893 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 19 16:42:24.722018 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.722000 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 19 16:42:24.722112 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.722007 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 19 16:42:24.722220 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.722205 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 19 16:42:24.722360 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.722347 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 19 16:42:24.722467 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.722454 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-bpkmv\"" Jul 19 16:42:24.722506 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.722471 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-9w62k"] Jul 19 16:42:24.722561 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.722548 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5c456d485-qvzgh" Jul 19 16:42:24.724470 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.724448 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 19 16:42:24.724844 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.724826 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 19 16:42:24.724923 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.724841 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 19 16:42:24.724923 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.724852 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-qblxf\"" Jul 19 16:42:24.731201 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.730663 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 19 16:42:24.732930 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.732913 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-d2qfz"] Jul 19 16:42:24.733704 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.733682 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5c456d485-qvzgh"] Jul 19 16:42:24.734580 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.734560 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-755474c78d-xctf5"] Jul 19 16:42:24.813029 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.812990 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ea535ad5-b3c6-46f0-9444-424035178c81-ca-trust-extracted\") pod \"image-registry-5c456d485-qvzgh\" (UID: \"ea535ad5-b3c6-46f0-9444-424035178c81\") " pod="openshift-image-registry/image-registry-5c456d485-qvzgh" Jul 19 16:42:24.813239 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.813046 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kl42n\" (UniqueName: \"kubernetes.io/projected/c580fe52-93d3-46c6-bf10-b90f9af30ee5-kube-api-access-kl42n\") pod \"router-default-755474c78d-xctf5\" (UID: \"c580fe52-93d3-46c6-bf10-b90f9af30ee5\") " pod="openshift-ingress/router-default-755474c78d-xctf5" Jul 19 16:42:24.813239 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.813114 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqrhm\" (UniqueName: \"kubernetes.io/projected/4248ffdc-b491-4236-9d08-af8a40e27903-kube-api-access-wqrhm\") pod \"cluster-samples-operator-79b7c55f87-p9vzx\" (UID: \"4248ffdc-b491-4236-9d08-af8a40e27903\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-p9vzx" Jul 19 16:42:24.813239 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.813207 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c580fe52-93d3-46c6-bf10-b90f9af30ee5-default-certificate\") pod \"router-default-755474c78d-xctf5\" (UID: \"c580fe52-93d3-46c6-bf10-b90f9af30ee5\") " pod="openshift-ingress/router-default-755474c78d-xctf5" Jul 19 16:42:24.813385 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.813242 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ea535ad5-b3c6-46f0-9444-424035178c81-bound-sa-token\") pod \"image-registry-5c456d485-qvzgh\" (UID: \"ea535ad5-b3c6-46f0-9444-424035178c81\") " pod="openshift-image-registry/image-registry-5c456d485-qvzgh" Jul 19 16:42:24.813385 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.813282 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ea535ad5-b3c6-46f0-9444-424035178c81-image-registry-private-configuration\") pod \"image-registry-5c456d485-qvzgh\" (UID: \"ea535ad5-b3c6-46f0-9444-424035178c81\") " pod="openshift-image-registry/image-registry-5c456d485-qvzgh" Jul 19 16:42:24.813385 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.813310 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glcpj\" (UniqueName: \"kubernetes.io/projected/7b5796d8-6465-4c8b-835e-7bf00c972c9e-kube-api-access-glcpj\") pod \"service-ca-operator-5bcbc94c78-d2qfz\" (UID: \"7b5796d8-6465-4c8b-835e-7bf00c972c9e\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-d2qfz" Jul 19 16:42:24.813385 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.813345 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7b5796d8-6465-4c8b-835e-7bf00c972c9e-serving-cert\") pod \"service-ca-operator-5bcbc94c78-d2qfz\" (UID: \"7b5796d8-6465-4c8b-835e-7bf00c972c9e\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-d2qfz" Jul 19 16:42:24.813385 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.813375 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c580fe52-93d3-46c6-bf10-b90f9af30ee5-stats-auth\") pod \"router-default-755474c78d-xctf5\" (UID: \"c580fe52-93d3-46c6-bf10-b90f9af30ee5\") " pod="openshift-ingress/router-default-755474c78d-xctf5" Jul 19 16:42:24.813535 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.813399 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ea535ad5-b3c6-46f0-9444-424035178c81-registry-tls\") pod \"image-registry-5c456d485-qvzgh\" (UID: \"ea535ad5-b3c6-46f0-9444-424035178c81\") " pod="openshift-image-registry/image-registry-5c456d485-qvzgh" Jul 19 16:42:24.813535 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.813419 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwpmw\" (UniqueName: \"kubernetes.io/projected/ea535ad5-b3c6-46f0-9444-424035178c81-kube-api-access-cwpmw\") pod \"image-registry-5c456d485-qvzgh\" (UID: \"ea535ad5-b3c6-46f0-9444-424035178c81\") " pod="openshift-image-registry/image-registry-5c456d485-qvzgh" Jul 19 16:42:24.813535 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.813452 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7b5796d8-6465-4c8b-835e-7bf00c972c9e-config\") pod \"service-ca-operator-5bcbc94c78-d2qfz\" (UID: \"7b5796d8-6465-4c8b-835e-7bf00c972c9e\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-d2qfz" Jul 19 16:42:24.813535 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.813476 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29z25\" (UniqueName: \"kubernetes.io/projected/1f1806d2-ba68-40ef-b5e8-d1820c604275-kube-api-access-29z25\") pod \"network-check-source-7bc5b6bf85-9w62k\" (UID: \"1f1806d2-ba68-40ef-b5e8-d1820c604275\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-9w62k" Jul 19 16:42:24.813535 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.813492 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ea535ad5-b3c6-46f0-9444-424035178c81-trusted-ca\") pod \"image-registry-5c456d485-qvzgh\" (UID: \"ea535ad5-b3c6-46f0-9444-424035178c81\") " pod="openshift-image-registry/image-registry-5c456d485-qvzgh" Jul 19 16:42:24.813535 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.813524 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ea535ad5-b3c6-46f0-9444-424035178c81-registry-certificates\") pod \"image-registry-5c456d485-qvzgh\" (UID: \"ea535ad5-b3c6-46f0-9444-424035178c81\") " pod="openshift-image-registry/image-registry-5c456d485-qvzgh" Jul 19 16:42:24.813705 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.813546 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c580fe52-93d3-46c6-bf10-b90f9af30ee5-service-ca-bundle\") pod \"router-default-755474c78d-xctf5\" (UID: \"c580fe52-93d3-46c6-bf10-b90f9af30ee5\") " pod="openshift-ingress/router-default-755474c78d-xctf5" Jul 19 16:42:24.813705 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.813564 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c580fe52-93d3-46c6-bf10-b90f9af30ee5-metrics-certs\") pod \"router-default-755474c78d-xctf5\" (UID: \"c580fe52-93d3-46c6-bf10-b90f9af30ee5\") " pod="openshift-ingress/router-default-755474c78d-xctf5" Jul 19 16:42:24.813705 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.813616 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ea535ad5-b3c6-46f0-9444-424035178c81-installation-pull-secrets\") pod \"image-registry-5c456d485-qvzgh\" (UID: \"ea535ad5-b3c6-46f0-9444-424035178c81\") " pod="openshift-image-registry/image-registry-5c456d485-qvzgh" Jul 19 16:42:24.813705 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.813656 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4248ffdc-b491-4236-9d08-af8a40e27903-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-p9vzx\" (UID: \"4248ffdc-b491-4236-9d08-af8a40e27903\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-p9vzx" Jul 19 16:42:24.813864 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:24.813756 2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 19 16:42:24.813864 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:24.813811 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4248ffdc-b491-4236-9d08-af8a40e27903-samples-operator-tls podName:4248ffdc-b491-4236-9d08-af8a40e27903 nodeName:}" failed. No retries permitted until 2026-07-19 16:42:25.313792668 +0000 UTC m=+240.300368131 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/4248ffdc-b491-4236-9d08-af8a40e27903-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-p9vzx" (UID: "4248ffdc-b491-4236-9d08-af8a40e27903") : secret "samples-operator-tls" not found Jul 19 16:42:24.820361 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.820330 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqrhm\" (UniqueName: \"kubernetes.io/projected/4248ffdc-b491-4236-9d08-af8a40e27903-kube-api-access-wqrhm\") pod \"cluster-samples-operator-79b7c55f87-p9vzx\" (UID: \"4248ffdc-b491-4236-9d08-af8a40e27903\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-p9vzx" Jul 19 16:42:24.914604 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.914509 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c580fe52-93d3-46c6-bf10-b90f9af30ee5-service-ca-bundle\") pod \"router-default-755474c78d-xctf5\" (UID: \"c580fe52-93d3-46c6-bf10-b90f9af30ee5\") " pod="openshift-ingress/router-default-755474c78d-xctf5" Jul 19 16:42:24.914604 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.914553 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c580fe52-93d3-46c6-bf10-b90f9af30ee5-metrics-certs\") pod \"router-default-755474c78d-xctf5\" (UID: \"c580fe52-93d3-46c6-bf10-b90f9af30ee5\") " pod="openshift-ingress/router-default-755474c78d-xctf5" Jul 19 16:42:24.914604 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.914581 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ea535ad5-b3c6-46f0-9444-424035178c81-installation-pull-secrets\") pod \"image-registry-5c456d485-qvzgh\" (UID: \"ea535ad5-b3c6-46f0-9444-424035178c81\") " pod="openshift-image-registry/image-registry-5c456d485-qvzgh" Jul 19 16:42:24.914830 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:24.914671 2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 19 16:42:24.914830 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:24.914697 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c580fe52-93d3-46c6-bf10-b90f9af30ee5-service-ca-bundle podName:c580fe52-93d3-46c6-bf10-b90f9af30ee5 nodeName:}" failed. No retries permitted until 2026-07-19 16:42:25.414675491 +0000 UTC m=+240.401250956 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c580fe52-93d3-46c6-bf10-b90f9af30ee5-service-ca-bundle") pod "router-default-755474c78d-xctf5" (UID: "c580fe52-93d3-46c6-bf10-b90f9af30ee5") : configmap references non-existent config key: service-ca.crt Jul 19 16:42:24.914830 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.914740 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ea535ad5-b3c6-46f0-9444-424035178c81-ca-trust-extracted\") pod \"image-registry-5c456d485-qvzgh\" (UID: \"ea535ad5-b3c6-46f0-9444-424035178c81\") " pod="openshift-image-registry/image-registry-5c456d485-qvzgh" Jul 19 16:42:24.914830 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:24.914760 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c580fe52-93d3-46c6-bf10-b90f9af30ee5-metrics-certs podName:c580fe52-93d3-46c6-bf10-b90f9af30ee5 nodeName:}" failed. No retries permitted until 2026-07-19 16:42:25.414744709 +0000 UTC m=+240.401320170 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c580fe52-93d3-46c6-bf10-b90f9af30ee5-metrics-certs") pod "router-default-755474c78d-xctf5" (UID: "c580fe52-93d3-46c6-bf10-b90f9af30ee5") : secret "router-metrics-certs-default" not found Jul 19 16:42:24.914830 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.914807 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kl42n\" (UniqueName: \"kubernetes.io/projected/c580fe52-93d3-46c6-bf10-b90f9af30ee5-kube-api-access-kl42n\") pod \"router-default-755474c78d-xctf5\" (UID: \"c580fe52-93d3-46c6-bf10-b90f9af30ee5\") " pod="openshift-ingress/router-default-755474c78d-xctf5" Jul 19 16:42:24.915079 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.914849 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c580fe52-93d3-46c6-bf10-b90f9af30ee5-default-certificate\") pod \"router-default-755474c78d-xctf5\" (UID: \"c580fe52-93d3-46c6-bf10-b90f9af30ee5\") " pod="openshift-ingress/router-default-755474c78d-xctf5" Jul 19 16:42:24.915079 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.914865 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ea535ad5-b3c6-46f0-9444-424035178c81-bound-sa-token\") pod \"image-registry-5c456d485-qvzgh\" (UID: \"ea535ad5-b3c6-46f0-9444-424035178c81\") " pod="openshift-image-registry/image-registry-5c456d485-qvzgh" Jul 19 16:42:24.915079 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.914997 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ea535ad5-b3c6-46f0-9444-424035178c81-image-registry-private-configuration\") pod \"image-registry-5c456d485-qvzgh\" (UID: \"ea535ad5-b3c6-46f0-9444-424035178c81\") " pod="openshift-image-registry/image-registry-5c456d485-qvzgh" Jul 19 16:42:24.915079 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.915028 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glcpj\" (UniqueName: \"kubernetes.io/projected/7b5796d8-6465-4c8b-835e-7bf00c972c9e-kube-api-access-glcpj\") pod \"service-ca-operator-5bcbc94c78-d2qfz\" (UID: \"7b5796d8-6465-4c8b-835e-7bf00c972c9e\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-d2qfz" Jul 19 16:42:24.915079 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.915075 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7b5796d8-6465-4c8b-835e-7bf00c972c9e-serving-cert\") pod \"service-ca-operator-5bcbc94c78-d2qfz\" (UID: \"7b5796d8-6465-4c8b-835e-7bf00c972c9e\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-d2qfz" Jul 19 16:42:24.915339 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.915085 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ea535ad5-b3c6-46f0-9444-424035178c81-ca-trust-extracted\") pod \"image-registry-5c456d485-qvzgh\" (UID: \"ea535ad5-b3c6-46f0-9444-424035178c81\") " pod="openshift-image-registry/image-registry-5c456d485-qvzgh" Jul 19 16:42:24.915339 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.915109 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c580fe52-93d3-46c6-bf10-b90f9af30ee5-stats-auth\") pod \"router-default-755474c78d-xctf5\" (UID: \"c580fe52-93d3-46c6-bf10-b90f9af30ee5\") " pod="openshift-ingress/router-default-755474c78d-xctf5" Jul 19 16:42:24.915339 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.915137 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ea535ad5-b3c6-46f0-9444-424035178c81-registry-tls\") pod \"image-registry-5c456d485-qvzgh\" (UID: \"ea535ad5-b3c6-46f0-9444-424035178c81\") " pod="openshift-image-registry/image-registry-5c456d485-qvzgh" Jul 19 16:42:24.915339 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.915163 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwpmw\" (UniqueName: \"kubernetes.io/projected/ea535ad5-b3c6-46f0-9444-424035178c81-kube-api-access-cwpmw\") pod \"image-registry-5c456d485-qvzgh\" (UID: \"ea535ad5-b3c6-46f0-9444-424035178c81\") " pod="openshift-image-registry/image-registry-5c456d485-qvzgh" Jul 19 16:42:24.915339 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.915190 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7b5796d8-6465-4c8b-835e-7bf00c972c9e-config\") pod \"service-ca-operator-5bcbc94c78-d2qfz\" (UID: \"7b5796d8-6465-4c8b-835e-7bf00c972c9e\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-d2qfz" Jul 19 16:42:24.915339 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.915224 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29z25\" (UniqueName: \"kubernetes.io/projected/1f1806d2-ba68-40ef-b5e8-d1820c604275-kube-api-access-29z25\") pod \"network-check-source-7bc5b6bf85-9w62k\" (UID: \"1f1806d2-ba68-40ef-b5e8-d1820c604275\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-9w62k" Jul 19 16:42:24.915339 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.915249 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ea535ad5-b3c6-46f0-9444-424035178c81-trusted-ca\") pod \"image-registry-5c456d485-qvzgh\" (UID: \"ea535ad5-b3c6-46f0-9444-424035178c81\") " pod="openshift-image-registry/image-registry-5c456d485-qvzgh" Jul 19 16:42:24.915339 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.915286 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ea535ad5-b3c6-46f0-9444-424035178c81-registry-certificates\") pod \"image-registry-5c456d485-qvzgh\" (UID: \"ea535ad5-b3c6-46f0-9444-424035178c81\") " pod="openshift-image-registry/image-registry-5c456d485-qvzgh" Jul 19 16:42:24.915841 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:24.915818 2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 19 16:42:24.915951 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:24.915938 2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5c456d485-qvzgh: secret "image-registry-tls" not found Jul 19 16:42:24.916113 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:24.916099 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ea535ad5-b3c6-46f0-9444-424035178c81-registry-tls podName:ea535ad5-b3c6-46f0-9444-424035178c81 nodeName:}" failed. No retries permitted until 2026-07-19 16:42:25.416080748 +0000 UTC m=+240.402656227 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ea535ad5-b3c6-46f0-9444-424035178c81-registry-tls") pod "image-registry-5c456d485-qvzgh" (UID: "ea535ad5-b3c6-46f0-9444-424035178c81") : secret "image-registry-tls" not found Jul 19 16:42:24.916208 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.915857 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7b5796d8-6465-4c8b-835e-7bf00c972c9e-config\") pod \"service-ca-operator-5bcbc94c78-d2qfz\" (UID: \"7b5796d8-6465-4c8b-835e-7bf00c972c9e\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-d2qfz" Jul 19 16:42:24.916320 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.915870 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ea535ad5-b3c6-46f0-9444-424035178c81-registry-certificates\") pod \"image-registry-5c456d485-qvzgh\" (UID: \"ea535ad5-b3c6-46f0-9444-424035178c81\") " pod="openshift-image-registry/image-registry-5c456d485-qvzgh" Jul 19 16:42:24.916969 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.916945 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ea535ad5-b3c6-46f0-9444-424035178c81-trusted-ca\") pod \"image-registry-5c456d485-qvzgh\" (UID: \"ea535ad5-b3c6-46f0-9444-424035178c81\") " pod="openshift-image-registry/image-registry-5c456d485-qvzgh" Jul 19 16:42:24.917405 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.917383 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ea535ad5-b3c6-46f0-9444-424035178c81-installation-pull-secrets\") pod \"image-registry-5c456d485-qvzgh\" (UID: \"ea535ad5-b3c6-46f0-9444-424035178c81\") " pod="openshift-image-registry/image-registry-5c456d485-qvzgh" Jul 19 16:42:24.917613 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.917592 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7b5796d8-6465-4c8b-835e-7bf00c972c9e-serving-cert\") pod \"service-ca-operator-5bcbc94c78-d2qfz\" (UID: \"7b5796d8-6465-4c8b-835e-7bf00c972c9e\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-d2qfz" Jul 19 16:42:24.917652 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.917599 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c580fe52-93d3-46c6-bf10-b90f9af30ee5-default-certificate\") pod \"router-default-755474c78d-xctf5\" (UID: \"c580fe52-93d3-46c6-bf10-b90f9af30ee5\") " pod="openshift-ingress/router-default-755474c78d-xctf5" Jul 19 16:42:24.917723 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.917708 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ea535ad5-b3c6-46f0-9444-424035178c81-image-registry-private-configuration\") pod \"image-registry-5c456d485-qvzgh\" (UID: \"ea535ad5-b3c6-46f0-9444-424035178c81\") " pod="openshift-image-registry/image-registry-5c456d485-qvzgh" Jul 19 16:42:24.917984 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.917966 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c580fe52-93d3-46c6-bf10-b90f9af30ee5-stats-auth\") pod \"router-default-755474c78d-xctf5\" (UID: \"c580fe52-93d3-46c6-bf10-b90f9af30ee5\") " pod="openshift-ingress/router-default-755474c78d-xctf5" Jul 19 16:42:24.923692 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.923665 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-glcpj\" (UniqueName: \"kubernetes.io/projected/7b5796d8-6465-4c8b-835e-7bf00c972c9e-kube-api-access-glcpj\") pod \"service-ca-operator-5bcbc94c78-d2qfz\" (UID: \"7b5796d8-6465-4c8b-835e-7bf00c972c9e\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-d2qfz" Jul 19 16:42:24.923941 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.923924 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ea535ad5-b3c6-46f0-9444-424035178c81-bound-sa-token\") pod \"image-registry-5c456d485-qvzgh\" (UID: \"ea535ad5-b3c6-46f0-9444-424035178c81\") " pod="openshift-image-registry/image-registry-5c456d485-qvzgh" Jul 19 16:42:24.924175 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.924155 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwpmw\" (UniqueName: \"kubernetes.io/projected/ea535ad5-b3c6-46f0-9444-424035178c81-kube-api-access-cwpmw\") pod \"image-registry-5c456d485-qvzgh\" (UID: \"ea535ad5-b3c6-46f0-9444-424035178c81\") " pod="openshift-image-registry/image-registry-5c456d485-qvzgh" Jul 19 16:42:24.924748 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.924724 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-29z25\" (UniqueName: \"kubernetes.io/projected/1f1806d2-ba68-40ef-b5e8-d1820c604275-kube-api-access-29z25\") pod \"network-check-source-7bc5b6bf85-9w62k\" (UID: \"1f1806d2-ba68-40ef-b5e8-d1820c604275\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-9w62k" Jul 19 16:42:24.924826 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:24.924730 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kl42n\" (UniqueName: \"kubernetes.io/projected/c580fe52-93d3-46c6-bf10-b90f9af30ee5-kube-api-access-kl42n\") pod \"router-default-755474c78d-xctf5\" (UID: \"c580fe52-93d3-46c6-bf10-b90f9af30ee5\") " pod="openshift-ingress/router-default-755474c78d-xctf5" Jul 19 16:42:25.018205 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:25.018155 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-9w62k" Jul 19 16:42:25.026489 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:25.026466 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-d2qfz" Jul 19 16:42:25.168113 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:25.168025 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-9w62k"] Jul 19 16:42:25.170758 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:42:25.170724 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f1806d2_ba68_40ef_b5e8_d1820c604275.slice/crio-ce31fc38bc2534b364246c6e77a62f6a4cc42b59bb69b29ca64c4767001bb2db WatchSource:0}: Error finding container ce31fc38bc2534b364246c6e77a62f6a4cc42b59bb69b29ca64c4767001bb2db: Status 404 returned error can't find the container with id ce31fc38bc2534b364246c6e77a62f6a4cc42b59bb69b29ca64c4767001bb2db Jul 19 16:42:25.189485 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:25.189461 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-d2qfz"] Jul 19 16:42:25.199100 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:42:25.199033 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b5796d8_6465_4c8b_835e_7bf00c972c9e.slice/crio-a6c9d408a9dfbdd2e9c8341bea3a927bd3e3912004b6ff1d399d2ad9ed852be5 WatchSource:0}: Error finding container a6c9d408a9dfbdd2e9c8341bea3a927bd3e3912004b6ff1d399d2ad9ed852be5: Status 404 returned error can't find the container with id a6c9d408a9dfbdd2e9c8341bea3a927bd3e3912004b6ff1d399d2ad9ed852be5 Jul 19 16:42:25.318016 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:25.317977 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4248ffdc-b491-4236-9d08-af8a40e27903-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-p9vzx\" (UID: \"4248ffdc-b491-4236-9d08-af8a40e27903\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-p9vzx" Jul 19 16:42:25.318243 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:25.318148 2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 19 16:42:25.318243 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:25.318217 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4248ffdc-b491-4236-9d08-af8a40e27903-samples-operator-tls podName:4248ffdc-b491-4236-9d08-af8a40e27903 nodeName:}" failed. No retries permitted until 2026-07-19 16:42:26.318198739 +0000 UTC m=+241.304774202 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/4248ffdc-b491-4236-9d08-af8a40e27903-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-p9vzx" (UID: "4248ffdc-b491-4236-9d08-af8a40e27903") : secret "samples-operator-tls" not found Jul 19 16:42:25.358294 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:25.358250 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-d2qfz" event={"ID":"7b5796d8-6465-4c8b-835e-7bf00c972c9e","Type":"ContainerStarted","Data":"a6c9d408a9dfbdd2e9c8341bea3a927bd3e3912004b6ff1d399d2ad9ed852be5"} Jul 19 16:42:25.359559 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:25.359539 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-9w62k" event={"ID":"1f1806d2-ba68-40ef-b5e8-d1820c604275","Type":"ContainerStarted","Data":"5bca9d7e977f74875adc8e4b893a614d0aa8309ad720a7f5a710b6de4d888e89"} Jul 19 16:42:25.359644 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:25.359576 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-9w62k" event={"ID":"1f1806d2-ba68-40ef-b5e8-d1820c604275","Type":"ContainerStarted","Data":"ce31fc38bc2534b364246c6e77a62f6a4cc42b59bb69b29ca64c4767001bb2db"} Jul 19 16:42:25.375369 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:25.375323 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-9w62k" podStartSLOduration=1.375308507 podStartE2EDuration="1.375308507s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 16:42:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 16:42:25.374464622 +0000 UTC m=+240.361040107" watchObservedRunningTime="2026-07-19 16:42:25.375308507 +0000 UTC m=+240.361883991" Jul 19 16:42:25.419263 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:25.419189 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ea535ad5-b3c6-46f0-9444-424035178c81-registry-tls\") pod \"image-registry-5c456d485-qvzgh\" (UID: \"ea535ad5-b3c6-46f0-9444-424035178c81\") " pod="openshift-image-registry/image-registry-5c456d485-qvzgh" Jul 19 16:42:25.419377 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:25.419299 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c580fe52-93d3-46c6-bf10-b90f9af30ee5-service-ca-bundle\") pod \"router-default-755474c78d-xctf5\" (UID: \"c580fe52-93d3-46c6-bf10-b90f9af30ee5\") " pod="openshift-ingress/router-default-755474c78d-xctf5" Jul 19 16:42:25.419377 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:25.419353 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c580fe52-93d3-46c6-bf10-b90f9af30ee5-metrics-certs\") pod \"router-default-755474c78d-xctf5\" (UID: \"c580fe52-93d3-46c6-bf10-b90f9af30ee5\") " pod="openshift-ingress/router-default-755474c78d-xctf5" Jul 19 16:42:25.419491 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:25.419472 2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 19 16:42:25.419527 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:25.419496 2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5c456d485-qvzgh: secret "image-registry-tls" not found Jul 19 16:42:25.419527 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:25.419476 2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 19 16:42:25.419589 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:25.419526 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c580fe52-93d3-46c6-bf10-b90f9af30ee5-service-ca-bundle podName:c580fe52-93d3-46c6-bf10-b90f9af30ee5 nodeName:}" failed. No retries permitted until 2026-07-19 16:42:26.419504396 +0000 UTC m=+241.406079871 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c580fe52-93d3-46c6-bf10-b90f9af30ee5-service-ca-bundle") pod "router-default-755474c78d-xctf5" (UID: "c580fe52-93d3-46c6-bf10-b90f9af30ee5") : configmap references non-existent config key: service-ca.crt Jul 19 16:42:25.419589 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:25.419556 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c580fe52-93d3-46c6-bf10-b90f9af30ee5-metrics-certs podName:c580fe52-93d3-46c6-bf10-b90f9af30ee5 nodeName:}" failed. No retries permitted until 2026-07-19 16:42:26.419540751 +0000 UTC m=+241.406116217 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c580fe52-93d3-46c6-bf10-b90f9af30ee5-metrics-certs") pod "router-default-755474c78d-xctf5" (UID: "c580fe52-93d3-46c6-bf10-b90f9af30ee5") : secret "router-metrics-certs-default" not found Jul 19 16:42:25.419589 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:25.419569 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ea535ad5-b3c6-46f0-9444-424035178c81-registry-tls podName:ea535ad5-b3c6-46f0-9444-424035178c81 nodeName:}" failed. No retries permitted until 2026-07-19 16:42:26.419563793 +0000 UTC m=+241.406139255 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ea535ad5-b3c6-46f0-9444-424035178c81-registry-tls") pod "image-registry-5c456d485-qvzgh" (UID: "ea535ad5-b3c6-46f0-9444-424035178c81") : secret "image-registry-tls" not found Jul 19 16:42:26.326412 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:26.326373 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4248ffdc-b491-4236-9d08-af8a40e27903-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-p9vzx\" (UID: \"4248ffdc-b491-4236-9d08-af8a40e27903\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-p9vzx" Jul 19 16:42:26.326842 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:26.326513 2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 19 16:42:26.326842 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:26.326571 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4248ffdc-b491-4236-9d08-af8a40e27903-samples-operator-tls podName:4248ffdc-b491-4236-9d08-af8a40e27903 nodeName:}" failed. No retries permitted until 2026-07-19 16:42:28.326556771 +0000 UTC m=+243.313132233 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/4248ffdc-b491-4236-9d08-af8a40e27903-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-p9vzx" (UID: "4248ffdc-b491-4236-9d08-af8a40e27903") : secret "samples-operator-tls" not found Jul 19 16:42:26.426953 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:26.426917 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c580fe52-93d3-46c6-bf10-b90f9af30ee5-metrics-certs\") pod \"router-default-755474c78d-xctf5\" (UID: \"c580fe52-93d3-46c6-bf10-b90f9af30ee5\") " pod="openshift-ingress/router-default-755474c78d-xctf5" Jul 19 16:42:26.427142 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:26.427024 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ea535ad5-b3c6-46f0-9444-424035178c81-registry-tls\") pod \"image-registry-5c456d485-qvzgh\" (UID: \"ea535ad5-b3c6-46f0-9444-424035178c81\") " pod="openshift-image-registry/image-registry-5c456d485-qvzgh" Jul 19 16:42:26.427142 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:26.427094 2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 19 16:42:26.427142 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:26.427108 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c580fe52-93d3-46c6-bf10-b90f9af30ee5-service-ca-bundle\") pod \"router-default-755474c78d-xctf5\" (UID: \"c580fe52-93d3-46c6-bf10-b90f9af30ee5\") " pod="openshift-ingress/router-default-755474c78d-xctf5" Jul 19 16:42:26.427322 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:26.427159 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c580fe52-93d3-46c6-bf10-b90f9af30ee5-metrics-certs podName:c580fe52-93d3-46c6-bf10-b90f9af30ee5 nodeName:}" failed. No retries permitted until 2026-07-19 16:42:28.427141586 +0000 UTC m=+243.413717060 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c580fe52-93d3-46c6-bf10-b90f9af30ee5-metrics-certs") pod "router-default-755474c78d-xctf5" (UID: "c580fe52-93d3-46c6-bf10-b90f9af30ee5") : secret "router-metrics-certs-default" not found Jul 19 16:42:26.427322 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:26.427182 2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 19 16:42:26.427322 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:26.427206 2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5c456d485-qvzgh: secret "image-registry-tls" not found Jul 19 16:42:26.427322 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:26.427207 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c580fe52-93d3-46c6-bf10-b90f9af30ee5-service-ca-bundle podName:c580fe52-93d3-46c6-bf10-b90f9af30ee5 nodeName:}" failed. No retries permitted until 2026-07-19 16:42:28.427197071 +0000 UTC m=+243.413772533 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c580fe52-93d3-46c6-bf10-b90f9af30ee5-service-ca-bundle") pod "router-default-755474c78d-xctf5" (UID: "c580fe52-93d3-46c6-bf10-b90f9af30ee5") : configmap references non-existent config key: service-ca.crt Jul 19 16:42:26.427322 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:26.427273 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ea535ad5-b3c6-46f0-9444-424035178c81-registry-tls podName:ea535ad5-b3c6-46f0-9444-424035178c81 nodeName:}" failed. No retries permitted until 2026-07-19 16:42:28.427253941 +0000 UTC m=+243.413829409 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ea535ad5-b3c6-46f0-9444-424035178c81-registry-tls") pod "image-registry-5c456d485-qvzgh" (UID: "ea535ad5-b3c6-46f0-9444-424035178c81") : secret "image-registry-tls" not found Jul 19 16:42:27.367885 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:27.367847 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-d2qfz" event={"ID":"7b5796d8-6465-4c8b-835e-7bf00c972c9e","Type":"ContainerStarted","Data":"04389da1b0b89fc4279585614b5883c5ffb35e75d49fae213683c3896fe6392e"} Jul 19 16:42:27.383110 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:27.383034 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-d2qfz" podStartSLOduration=1.324721354 podStartE2EDuration="3.38301151s" totalImagesPullingTime="2.058290156s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 16:42:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 16:42:27.38169125 +0000 UTC m=+242.368266747" watchObservedRunningTime="2026-07-19 16:42:27.38301151 +0000 UTC m=+242.369586997" Jul 19 16:42:28.341009 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:28.340973 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4248ffdc-b491-4236-9d08-af8a40e27903-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-p9vzx\" (UID: \"4248ffdc-b491-4236-9d08-af8a40e27903\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-p9vzx" Jul 19 16:42:28.341247 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:28.341166 2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 19 16:42:28.341311 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:28.341245 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4248ffdc-b491-4236-9d08-af8a40e27903-samples-operator-tls podName:4248ffdc-b491-4236-9d08-af8a40e27903 nodeName:}" failed. No retries permitted until 2026-07-19 16:42:32.341226263 +0000 UTC m=+247.327801726 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/4248ffdc-b491-4236-9d08-af8a40e27903-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-p9vzx" (UID: "4248ffdc-b491-4236-9d08-af8a40e27903") : secret "samples-operator-tls" not found Jul 19 16:42:28.441537 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:28.441495 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c580fe52-93d3-46c6-bf10-b90f9af30ee5-service-ca-bundle\") pod \"router-default-755474c78d-xctf5\" (UID: \"c580fe52-93d3-46c6-bf10-b90f9af30ee5\") " pod="openshift-ingress/router-default-755474c78d-xctf5" Jul 19 16:42:28.441537 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:28.441542 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c580fe52-93d3-46c6-bf10-b90f9af30ee5-metrics-certs\") pod \"router-default-755474c78d-xctf5\" (UID: \"c580fe52-93d3-46c6-bf10-b90f9af30ee5\") " pod="openshift-ingress/router-default-755474c78d-xctf5" Jul 19 16:42:28.442010 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:28.441663 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c580fe52-93d3-46c6-bf10-b90f9af30ee5-service-ca-bundle podName:c580fe52-93d3-46c6-bf10-b90f9af30ee5 nodeName:}" failed. No retries permitted until 2026-07-19 16:42:32.441639452 +0000 UTC m=+247.428214915 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c580fe52-93d3-46c6-bf10-b90f9af30ee5-service-ca-bundle") pod "router-default-755474c78d-xctf5" (UID: "c580fe52-93d3-46c6-bf10-b90f9af30ee5") : configmap references non-existent config key: service-ca.crt Jul 19 16:42:28.442010 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:28.441702 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ea535ad5-b3c6-46f0-9444-424035178c81-registry-tls\") pod \"image-registry-5c456d485-qvzgh\" (UID: \"ea535ad5-b3c6-46f0-9444-424035178c81\") " pod="openshift-image-registry/image-registry-5c456d485-qvzgh" Jul 19 16:42:28.442010 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:28.441723 2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 19 16:42:28.442010 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:28.441803 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c580fe52-93d3-46c6-bf10-b90f9af30ee5-metrics-certs podName:c580fe52-93d3-46c6-bf10-b90f9af30ee5 nodeName:}" failed. No retries permitted until 2026-07-19 16:42:32.441786794 +0000 UTC m=+247.428362262 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c580fe52-93d3-46c6-bf10-b90f9af30ee5-metrics-certs") pod "router-default-755474c78d-xctf5" (UID: "c580fe52-93d3-46c6-bf10-b90f9af30ee5") : secret "router-metrics-certs-default" not found Jul 19 16:42:28.442010 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:28.441805 2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 19 16:42:28.442010 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:28.441819 2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5c456d485-qvzgh: secret "image-registry-tls" not found Jul 19 16:42:28.442010 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:28.441856 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ea535ad5-b3c6-46f0-9444-424035178c81-registry-tls podName:ea535ad5-b3c6-46f0-9444-424035178c81 nodeName:}" failed. No retries permitted until 2026-07-19 16:42:32.441841969 +0000 UTC m=+247.428417431 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ea535ad5-b3c6-46f0-9444-424035178c81-registry-tls") pod "image-registry-5c456d485-qvzgh" (UID: "ea535ad5-b3c6-46f0-9444-424035178c81") : secret "image-registry-tls" not found Jul 19 16:42:29.601204 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:29.601170 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-4cjfp"] Jul 19 16:42:29.604306 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:29.604289 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4cjfp" Jul 19 16:42:29.606679 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:29.606655 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 19 16:42:29.606834 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:29.606811 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 19 16:42:29.607085 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:29.607069 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 19 16:42:29.607604 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:29.607585 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-z6jkb\"" Jul 19 16:42:29.607661 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:29.607615 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 19 16:42:29.618747 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:29.618724 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4cjfp"] Jul 19 16:42:29.752557 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:29.752512 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7387825e-ee08-4bcd-a6e3-7dcc34ed8036-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4cjfp\" (UID: \"7387825e-ee08-4bcd-a6e3-7dcc34ed8036\") " pod="openshift-insights/insights-runtime-extractor-4cjfp" Jul 19 16:42:29.752557 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:29.752559 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7387825e-ee08-4bcd-a6e3-7dcc34ed8036-data-volume\") pod \"insights-runtime-extractor-4cjfp\" (UID: \"7387825e-ee08-4bcd-a6e3-7dcc34ed8036\") " pod="openshift-insights/insights-runtime-extractor-4cjfp" Jul 19 16:42:29.752770 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:29.752592 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7387825e-ee08-4bcd-a6e3-7dcc34ed8036-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4cjfp\" (UID: \"7387825e-ee08-4bcd-a6e3-7dcc34ed8036\") " pod="openshift-insights/insights-runtime-extractor-4cjfp" Jul 19 16:42:29.752770 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:29.752650 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xz5vr\" (UniqueName: \"kubernetes.io/projected/7387825e-ee08-4bcd-a6e3-7dcc34ed8036-kube-api-access-xz5vr\") pod \"insights-runtime-extractor-4cjfp\" (UID: \"7387825e-ee08-4bcd-a6e3-7dcc34ed8036\") " pod="openshift-insights/insights-runtime-extractor-4cjfp" Jul 19 16:42:29.752770 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:29.752692 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7387825e-ee08-4bcd-a6e3-7dcc34ed8036-crio-socket\") pod \"insights-runtime-extractor-4cjfp\" (UID: \"7387825e-ee08-4bcd-a6e3-7dcc34ed8036\") " pod="openshift-insights/insights-runtime-extractor-4cjfp" Jul 19 16:42:29.853911 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:29.853836 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xz5vr\" (UniqueName: \"kubernetes.io/projected/7387825e-ee08-4bcd-a6e3-7dcc34ed8036-kube-api-access-xz5vr\") pod \"insights-runtime-extractor-4cjfp\" (UID: \"7387825e-ee08-4bcd-a6e3-7dcc34ed8036\") " pod="openshift-insights/insights-runtime-extractor-4cjfp" Jul 19 16:42:29.853911 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:29.853884 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7387825e-ee08-4bcd-a6e3-7dcc34ed8036-crio-socket\") pod \"insights-runtime-extractor-4cjfp\" (UID: \"7387825e-ee08-4bcd-a6e3-7dcc34ed8036\") " pod="openshift-insights/insights-runtime-extractor-4cjfp" Jul 19 16:42:29.854100 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:29.853978 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7387825e-ee08-4bcd-a6e3-7dcc34ed8036-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4cjfp\" (UID: \"7387825e-ee08-4bcd-a6e3-7dcc34ed8036\") " pod="openshift-insights/insights-runtime-extractor-4cjfp" Jul 19 16:42:29.854100 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:29.854005 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7387825e-ee08-4bcd-a6e3-7dcc34ed8036-data-volume\") pod \"insights-runtime-extractor-4cjfp\" (UID: \"7387825e-ee08-4bcd-a6e3-7dcc34ed8036\") " pod="openshift-insights/insights-runtime-extractor-4cjfp" Jul 19 16:42:29.854100 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:29.854030 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7387825e-ee08-4bcd-a6e3-7dcc34ed8036-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4cjfp\" (UID: \"7387825e-ee08-4bcd-a6e3-7dcc34ed8036\") " pod="openshift-insights/insights-runtime-extractor-4cjfp" Jul 19 16:42:29.854100 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:29.854049 2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 19 16:42:29.854253 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:29.854051 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7387825e-ee08-4bcd-a6e3-7dcc34ed8036-crio-socket\") pod \"insights-runtime-extractor-4cjfp\" (UID: \"7387825e-ee08-4bcd-a6e3-7dcc34ed8036\") " pod="openshift-insights/insights-runtime-extractor-4cjfp" Jul 19 16:42:29.854253 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:29.854142 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7387825e-ee08-4bcd-a6e3-7dcc34ed8036-kube-rbac-proxy-cm podName:7387825e-ee08-4bcd-a6e3-7dcc34ed8036 nodeName:}" failed. No retries permitted until 2026-07-19 16:42:30.354120667 +0000 UTC m=+245.340696137 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7387825e-ee08-4bcd-a6e3-7dcc34ed8036-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4cjfp" (UID: "7387825e-ee08-4bcd-a6e3-7dcc34ed8036") : configmap "kube-rbac-proxy" not found Jul 19 16:42:29.854253 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:29.854191 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 19 16:42:29.854253 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:29.854231 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7387825e-ee08-4bcd-a6e3-7dcc34ed8036-insights-runtime-extractor-tls podName:7387825e-ee08-4bcd-a6e3-7dcc34ed8036 nodeName:}" failed. No retries permitted until 2026-07-19 16:42:30.354221391 +0000 UTC m=+245.340796853 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7387825e-ee08-4bcd-a6e3-7dcc34ed8036-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4cjfp" (UID: "7387825e-ee08-4bcd-a6e3-7dcc34ed8036") : secret "insights-runtime-extractor-tls" not found Jul 19 16:42:29.854405 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:29.854329 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7387825e-ee08-4bcd-a6e3-7dcc34ed8036-data-volume\") pod \"insights-runtime-extractor-4cjfp\" (UID: \"7387825e-ee08-4bcd-a6e3-7dcc34ed8036\") " pod="openshift-insights/insights-runtime-extractor-4cjfp" Jul 19 16:42:29.864548 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:29.864511 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xz5vr\" (UniqueName: \"kubernetes.io/projected/7387825e-ee08-4bcd-a6e3-7dcc34ed8036-kube-api-access-xz5vr\") pod \"insights-runtime-extractor-4cjfp\" (UID: \"7387825e-ee08-4bcd-a6e3-7dcc34ed8036\") " pod="openshift-insights/insights-runtime-extractor-4cjfp" Jul 19 16:42:30.358770 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:30.358738 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7387825e-ee08-4bcd-a6e3-7dcc34ed8036-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4cjfp\" (UID: \"7387825e-ee08-4bcd-a6e3-7dcc34ed8036\") " pod="openshift-insights/insights-runtime-extractor-4cjfp" Jul 19 16:42:30.359030 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:30.358777 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7387825e-ee08-4bcd-a6e3-7dcc34ed8036-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4cjfp\" (UID: \"7387825e-ee08-4bcd-a6e3-7dcc34ed8036\") " pod="openshift-insights/insights-runtime-extractor-4cjfp" Jul 19 16:42:30.359030 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:30.358822 2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 19 16:42:30.359030 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:30.358893 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7387825e-ee08-4bcd-a6e3-7dcc34ed8036-kube-rbac-proxy-cm podName:7387825e-ee08-4bcd-a6e3-7dcc34ed8036 nodeName:}" failed. No retries permitted until 2026-07-19 16:42:31.358872449 +0000 UTC m=+246.345447917 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7387825e-ee08-4bcd-a6e3-7dcc34ed8036-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4cjfp" (UID: "7387825e-ee08-4bcd-a6e3-7dcc34ed8036") : configmap "kube-rbac-proxy" not found Jul 19 16:42:30.359030 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:30.358916 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 19 16:42:30.359030 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:30.358964 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7387825e-ee08-4bcd-a6e3-7dcc34ed8036-insights-runtime-extractor-tls podName:7387825e-ee08-4bcd-a6e3-7dcc34ed8036 nodeName:}" failed. No retries permitted until 2026-07-19 16:42:31.358952219 +0000 UTC m=+246.345527686 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7387825e-ee08-4bcd-a6e3-7dcc34ed8036-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4cjfp" (UID: "7387825e-ee08-4bcd-a6e3-7dcc34ed8036") : secret "insights-runtime-extractor-tls" not found Jul 19 16:42:31.367052 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:31.367006 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7387825e-ee08-4bcd-a6e3-7dcc34ed8036-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4cjfp\" (UID: \"7387825e-ee08-4bcd-a6e3-7dcc34ed8036\") " pod="openshift-insights/insights-runtime-extractor-4cjfp" Jul 19 16:42:31.367510 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:31.367081 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7387825e-ee08-4bcd-a6e3-7dcc34ed8036-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4cjfp\" (UID: \"7387825e-ee08-4bcd-a6e3-7dcc34ed8036\") " pod="openshift-insights/insights-runtime-extractor-4cjfp" Jul 19 16:42:31.367510 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:31.367134 2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 19 16:42:31.367510 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:31.367205 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7387825e-ee08-4bcd-a6e3-7dcc34ed8036-kube-rbac-proxy-cm podName:7387825e-ee08-4bcd-a6e3-7dcc34ed8036 nodeName:}" failed. No retries permitted until 2026-07-19 16:42:33.367189684 +0000 UTC m=+248.353765147 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7387825e-ee08-4bcd-a6e3-7dcc34ed8036-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4cjfp" (UID: "7387825e-ee08-4bcd-a6e3-7dcc34ed8036") : configmap "kube-rbac-proxy" not found Jul 19 16:42:31.367510 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:31.367220 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 19 16:42:31.367510 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:31.367277 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7387825e-ee08-4bcd-a6e3-7dcc34ed8036-insights-runtime-extractor-tls podName:7387825e-ee08-4bcd-a6e3-7dcc34ed8036 nodeName:}" failed. No retries permitted until 2026-07-19 16:42:33.36726438 +0000 UTC m=+248.353839847 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7387825e-ee08-4bcd-a6e3-7dcc34ed8036-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4cjfp" (UID: "7387825e-ee08-4bcd-a6e3-7dcc34ed8036") : secret "insights-runtime-extractor-tls" not found Jul 19 16:42:32.375427 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:32.375385 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4248ffdc-b491-4236-9d08-af8a40e27903-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-p9vzx\" (UID: \"4248ffdc-b491-4236-9d08-af8a40e27903\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-p9vzx" Jul 19 16:42:32.375815 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:32.375538 2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 19 16:42:32.375815 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:32.375605 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4248ffdc-b491-4236-9d08-af8a40e27903-samples-operator-tls podName:4248ffdc-b491-4236-9d08-af8a40e27903 nodeName:}" failed. No retries permitted until 2026-07-19 16:42:40.37558846 +0000 UTC m=+255.362163922 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/4248ffdc-b491-4236-9d08-af8a40e27903-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-p9vzx" (UID: "4248ffdc-b491-4236-9d08-af8a40e27903") : secret "samples-operator-tls" not found Jul 19 16:42:32.403591 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:32.403560 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-cv2rk_c1a2fb73-9db3-436a-8b9b-f4e5e7355889/dns-node-resolver/0.log" Jul 19 16:42:32.476291 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:32.476255 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c580fe52-93d3-46c6-bf10-b90f9af30ee5-service-ca-bundle\") pod \"router-default-755474c78d-xctf5\" (UID: \"c580fe52-93d3-46c6-bf10-b90f9af30ee5\") " pod="openshift-ingress/router-default-755474c78d-xctf5" Jul 19 16:42:32.476469 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:32.476298 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c580fe52-93d3-46c6-bf10-b90f9af30ee5-metrics-certs\") pod \"router-default-755474c78d-xctf5\" (UID: \"c580fe52-93d3-46c6-bf10-b90f9af30ee5\") " pod="openshift-ingress/router-default-755474c78d-xctf5" Jul 19 16:42:32.476469 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:32.476408 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ea535ad5-b3c6-46f0-9444-424035178c81-registry-tls\") pod \"image-registry-5c456d485-qvzgh\" (UID: \"ea535ad5-b3c6-46f0-9444-424035178c81\") " pod="openshift-image-registry/image-registry-5c456d485-qvzgh" Jul 19 16:42:32.476469 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:32.476423 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c580fe52-93d3-46c6-bf10-b90f9af30ee5-service-ca-bundle podName:c580fe52-93d3-46c6-bf10-b90f9af30ee5 nodeName:}" failed. No retries permitted until 2026-07-19 16:42:40.476405674 +0000 UTC m=+255.462981140 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c580fe52-93d3-46c6-bf10-b90f9af30ee5-service-ca-bundle") pod "router-default-755474c78d-xctf5" (UID: "c580fe52-93d3-46c6-bf10-b90f9af30ee5") : configmap references non-existent config key: service-ca.crt Jul 19 16:42:32.476618 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:32.476472 2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 19 16:42:32.476618 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:32.476484 2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 19 16:42:32.476618 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:32.476498 2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5c456d485-qvzgh: secret "image-registry-tls" not found Jul 19 16:42:32.476618 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:32.476519 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c580fe52-93d3-46c6-bf10-b90f9af30ee5-metrics-certs podName:c580fe52-93d3-46c6-bf10-b90f9af30ee5 nodeName:}" failed. No retries permitted until 2026-07-19 16:42:40.476506265 +0000 UTC m=+255.463081727 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c580fe52-93d3-46c6-bf10-b90f9af30ee5-metrics-certs") pod "router-default-755474c78d-xctf5" (UID: "c580fe52-93d3-46c6-bf10-b90f9af30ee5") : secret "router-metrics-certs-default" not found Jul 19 16:42:32.476618 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:32.476549 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ea535ad5-b3c6-46f0-9444-424035178c81-registry-tls podName:ea535ad5-b3c6-46f0-9444-424035178c81 nodeName:}" failed. No retries permitted until 2026-07-19 16:42:40.47653521 +0000 UTC m=+255.463110703 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ea535ad5-b3c6-46f0-9444-424035178c81-registry-tls") pod "image-registry-5c456d485-qvzgh" (UID: "ea535ad5-b3c6-46f0-9444-424035178c81") : secret "image-registry-tls" not found Jul 19 16:42:33.204098 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:33.204048 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-j6dwh_07d22c2c-bf9e-47af-bdf6-8e535bad46b2/node-ca/0.log" Jul 19 16:42:33.383484 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:33.383450 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7387825e-ee08-4bcd-a6e3-7dcc34ed8036-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4cjfp\" (UID: \"7387825e-ee08-4bcd-a6e3-7dcc34ed8036\") " pod="openshift-insights/insights-runtime-extractor-4cjfp" Jul 19 16:42:33.383484 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:33.383490 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7387825e-ee08-4bcd-a6e3-7dcc34ed8036-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4cjfp\" (UID: \"7387825e-ee08-4bcd-a6e3-7dcc34ed8036\") " pod="openshift-insights/insights-runtime-extractor-4cjfp" Jul 19 16:42:33.383948 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:33.383534 2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 19 16:42:33.383948 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:33.383590 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7387825e-ee08-4bcd-a6e3-7dcc34ed8036-kube-rbac-proxy-cm podName:7387825e-ee08-4bcd-a6e3-7dcc34ed8036 nodeName:}" failed. No retries permitted until 2026-07-19 16:42:37.383572614 +0000 UTC m=+252.370148076 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7387825e-ee08-4bcd-a6e3-7dcc34ed8036-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4cjfp" (UID: "7387825e-ee08-4bcd-a6e3-7dcc34ed8036") : configmap "kube-rbac-proxy" not found Jul 19 16:42:33.383948 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:33.383600 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 19 16:42:33.383948 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:33.383637 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7387825e-ee08-4bcd-a6e3-7dcc34ed8036-insights-runtime-extractor-tls podName:7387825e-ee08-4bcd-a6e3-7dcc34ed8036 nodeName:}" failed. No retries permitted until 2026-07-19 16:42:37.383626004 +0000 UTC m=+252.370201465 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7387825e-ee08-4bcd-a6e3-7dcc34ed8036-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4cjfp" (UID: "7387825e-ee08-4bcd-a6e3-7dcc34ed8036") : secret "insights-runtime-extractor-tls" not found Jul 19 16:42:34.414780 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:34.414748 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-txwnf_f133de90-2db7-4454-9d67-26e266dec955/serve-healthcheck-canary/0.log" Jul 19 16:42:37.318579 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:37.318535 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9c487ff0-684c-41bd-aca1-74636fae5571-metrics-certs\") pod \"network-metrics-daemon-thqxw\" (UID: \"9c487ff0-684c-41bd-aca1-74636fae5571\") " pod="openshift-multus/network-metrics-daemon-thqxw" Jul 19 16:42:37.318962 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:37.318662 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 19 16:42:37.318962 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:37.318716 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c487ff0-684c-41bd-aca1-74636fae5571-metrics-certs podName:9c487ff0-684c-41bd-aca1-74636fae5571 nodeName:}" failed. No retries permitted until 2026-07-19 16:44:39.318701862 +0000 UTC m=+374.305277324 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9c487ff0-684c-41bd-aca1-74636fae5571-metrics-certs") pod "network-metrics-daemon-thqxw" (UID: "9c487ff0-684c-41bd-aca1-74636fae5571") : secret "metrics-daemon-secret" not found Jul 19 16:42:37.419832 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:37.419784 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7387825e-ee08-4bcd-a6e3-7dcc34ed8036-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4cjfp\" (UID: \"7387825e-ee08-4bcd-a6e3-7dcc34ed8036\") " pod="openshift-insights/insights-runtime-extractor-4cjfp" Jul 19 16:42:37.419832 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:37.419838 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7387825e-ee08-4bcd-a6e3-7dcc34ed8036-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4cjfp\" (UID: \"7387825e-ee08-4bcd-a6e3-7dcc34ed8036\") " pod="openshift-insights/insights-runtime-extractor-4cjfp" Jul 19 16:42:37.420103 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:37.419904 2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 19 16:42:37.420103 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:37.419925 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 19 16:42:37.420103 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:37.419965 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7387825e-ee08-4bcd-a6e3-7dcc34ed8036-kube-rbac-proxy-cm podName:7387825e-ee08-4bcd-a6e3-7dcc34ed8036 nodeName:}" failed. No retries permitted until 2026-07-19 16:42:45.419950602 +0000 UTC m=+260.406526063 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7387825e-ee08-4bcd-a6e3-7dcc34ed8036-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4cjfp" (UID: "7387825e-ee08-4bcd-a6e3-7dcc34ed8036") : configmap "kube-rbac-proxy" not found Jul 19 16:42:37.420103 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:42:37.420041 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7387825e-ee08-4bcd-a6e3-7dcc34ed8036-insights-runtime-extractor-tls podName:7387825e-ee08-4bcd-a6e3-7dcc34ed8036 nodeName:}" failed. No retries permitted until 2026-07-19 16:42:45.420024791 +0000 UTC m=+260.406600253 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7387825e-ee08-4bcd-a6e3-7dcc34ed8036-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4cjfp" (UID: "7387825e-ee08-4bcd-a6e3-7dcc34ed8036") : secret "insights-runtime-extractor-tls" not found Jul 19 16:42:40.443560 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:40.443522 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4248ffdc-b491-4236-9d08-af8a40e27903-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-p9vzx\" (UID: \"4248ffdc-b491-4236-9d08-af8a40e27903\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-p9vzx" Jul 19 16:42:40.445925 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:40.445904 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4248ffdc-b491-4236-9d08-af8a40e27903-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-p9vzx\" (UID: \"4248ffdc-b491-4236-9d08-af8a40e27903\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-p9vzx" Jul 19 16:42:40.536378 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:40.536347 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-qs9kw\"" Jul 19 16:42:40.544227 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:40.544201 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ea535ad5-b3c6-46f0-9444-424035178c81-registry-tls\") pod \"image-registry-5c456d485-qvzgh\" (UID: \"ea535ad5-b3c6-46f0-9444-424035178c81\") " pod="openshift-image-registry/image-registry-5c456d485-qvzgh" Jul 19 16:42:40.544321 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:40.544250 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c580fe52-93d3-46c6-bf10-b90f9af30ee5-service-ca-bundle\") pod \"router-default-755474c78d-xctf5\" (UID: \"c580fe52-93d3-46c6-bf10-b90f9af30ee5\") " pod="openshift-ingress/router-default-755474c78d-xctf5" Jul 19 16:42:40.544321 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:40.544270 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c580fe52-93d3-46c6-bf10-b90f9af30ee5-metrics-certs\") pod \"router-default-755474c78d-xctf5\" (UID: \"c580fe52-93d3-46c6-bf10-b90f9af30ee5\") " pod="openshift-ingress/router-default-755474c78d-xctf5" Jul 19 16:42:40.544438 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:40.544416 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-p9vzx" Jul 19 16:42:40.544862 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:40.544841 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c580fe52-93d3-46c6-bf10-b90f9af30ee5-service-ca-bundle\") pod \"router-default-755474c78d-xctf5\" (UID: \"c580fe52-93d3-46c6-bf10-b90f9af30ee5\") " pod="openshift-ingress/router-default-755474c78d-xctf5" Jul 19 16:42:40.546657 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:40.546631 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ea535ad5-b3c6-46f0-9444-424035178c81-registry-tls\") pod \"image-registry-5c456d485-qvzgh\" (UID: \"ea535ad5-b3c6-46f0-9444-424035178c81\") " pod="openshift-image-registry/image-registry-5c456d485-qvzgh" Jul 19 16:42:40.546762 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:40.546707 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c580fe52-93d3-46c6-bf10-b90f9af30ee5-metrics-certs\") pod \"router-default-755474c78d-xctf5\" (UID: \"c580fe52-93d3-46c6-bf10-b90f9af30ee5\") " pod="openshift-ingress/router-default-755474c78d-xctf5" Jul 19 16:42:40.636770 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:40.636739 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-bpkmv\"" Jul 19 16:42:40.642879 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:40.642850 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-qblxf\"" Jul 19 16:42:40.644548 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:40.644506 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-755474c78d-xctf5" Jul 19 16:42:40.650939 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:40.650910 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5c456d485-qvzgh" Jul 19 16:42:40.679205 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:40.679177 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-p9vzx"] Jul 19 16:42:40.788347 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:40.788317 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-755474c78d-xctf5"] Jul 19 16:42:40.791218 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:42:40.791190 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc580fe52_93d3_46c6_bf10_b90f9af30ee5.slice/crio-bdc5c97fd9fd5b0b625c1a5d992f41cccf605f523e2d1c62da8dd6e9da606411 WatchSource:0}: Error finding container bdc5c97fd9fd5b0b625c1a5d992f41cccf605f523e2d1c62da8dd6e9da606411: Status 404 returned error can't find the container with id bdc5c97fd9fd5b0b625c1a5d992f41cccf605f523e2d1c62da8dd6e9da606411 Jul 19 16:42:40.806154 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:40.806114 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5c456d485-qvzgh"] Jul 19 16:42:40.809605 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:42:40.809582 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea535ad5_b3c6_46f0_9444_424035178c81.slice/crio-ba620364612cbcb78f6df4204ea86a988723e1191ab83667087ae615bb9bad25 WatchSource:0}: Error finding container ba620364612cbcb78f6df4204ea86a988723e1191ab83667087ae615bb9bad25: Status 404 returned error can't find the container with id ba620364612cbcb78f6df4204ea86a988723e1191ab83667087ae615bb9bad25 Jul 19 16:42:41.412474 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:41.412435 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-755474c78d-xctf5" event={"ID":"c580fe52-93d3-46c6-bf10-b90f9af30ee5","Type":"ContainerStarted","Data":"68b724f24eeb07b0ebcb53a6cd4c927f1ad96f1b26b527aaee013f9cb43b355e"} Jul 19 16:42:41.412663 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:41.412496 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-755474c78d-xctf5" event={"ID":"c580fe52-93d3-46c6-bf10-b90f9af30ee5","Type":"ContainerStarted","Data":"bdc5c97fd9fd5b0b625c1a5d992f41cccf605f523e2d1c62da8dd6e9da606411"} Jul 19 16:42:41.414083 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:41.414039 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5c456d485-qvzgh" event={"ID":"ea535ad5-b3c6-46f0-9444-424035178c81","Type":"ContainerStarted","Data":"5afaa8655db886b2a37ab6c5fc2af3a0e607b1d193ab71ae78a557d9e3bef513"} Jul 19 16:42:41.414210 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:41.414104 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5c456d485-qvzgh" Jul 19 16:42:41.414210 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:41.414124 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5c456d485-qvzgh" event={"ID":"ea535ad5-b3c6-46f0-9444-424035178c81","Type":"ContainerStarted","Data":"ba620364612cbcb78f6df4204ea86a988723e1191ab83667087ae615bb9bad25"} Jul 19 16:42:41.415226 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:41.415204 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-p9vzx" event={"ID":"4248ffdc-b491-4236-9d08-af8a40e27903","Type":"ContainerStarted","Data":"967264e8e8ac81065a16fcffc9fef0a0fabf0de5a1726263dbe1545ba039201a"} Jul 19 16:42:41.433960 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:41.433914 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-755474c78d-xctf5" podStartSLOduration=17.43390146 podStartE2EDuration="17.43390146s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 16:42:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 16:42:41.432973612 +0000 UTC m=+256.419549114" watchObservedRunningTime="2026-07-19 16:42:41.43390146 +0000 UTC m=+256.420476944" Jul 19 16:42:41.452945 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:41.452901 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5c456d485-qvzgh" podStartSLOduration=17.452886861 podStartE2EDuration="17.452886861s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 16:42:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 16:42:41.45193654 +0000 UTC m=+256.438512018" watchObservedRunningTime="2026-07-19 16:42:41.452886861 +0000 UTC m=+256.439462345" Jul 19 16:42:41.644958 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:41.644922 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-755474c78d-xctf5" Jul 19 16:42:41.647419 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:41.647394 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-755474c78d-xctf5" Jul 19 16:42:42.418916 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:42.418885 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-755474c78d-xctf5" Jul 19 16:42:42.420308 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:42.420282 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-755474c78d-xctf5" Jul 19 16:42:43.422888 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:43.422851 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-p9vzx" event={"ID":"4248ffdc-b491-4236-9d08-af8a40e27903","Type":"ContainerStarted","Data":"29843de52d2fce934aa65d5f369531536032628e1df16cbaebf0344c762bfcc5"} Jul 19 16:42:43.422888 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:43.422907 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-p9vzx" event={"ID":"4248ffdc-b491-4236-9d08-af8a40e27903","Type":"ContainerStarted","Data":"4648592575ed0b9da7385207e92be609f4af3c9e6f208da91f76a3d1734db285"} Jul 19 16:42:43.437762 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:43.437714 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-p9vzx" podStartSLOduration=17.416255895 podStartE2EDuration="19.437698782s" totalImagesPullingTime="2.021442887s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 16:42:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 16:42:43.436889758 +0000 UTC m=+258.423465244" watchObservedRunningTime="2026-07-19 16:42:43.437698782 +0000 UTC m=+258.424274291" Jul 19 16:42:45.484310 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:45.484277 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7387825e-ee08-4bcd-a6e3-7dcc34ed8036-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4cjfp\" (UID: \"7387825e-ee08-4bcd-a6e3-7dcc34ed8036\") " pod="openshift-insights/insights-runtime-extractor-4cjfp" Jul 19 16:42:45.484716 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:45.484374 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7387825e-ee08-4bcd-a6e3-7dcc34ed8036-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4cjfp\" (UID: \"7387825e-ee08-4bcd-a6e3-7dcc34ed8036\") " pod="openshift-insights/insights-runtime-extractor-4cjfp" Jul 19 16:42:45.484850 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:45.484832 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7387825e-ee08-4bcd-a6e3-7dcc34ed8036-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4cjfp\" (UID: \"7387825e-ee08-4bcd-a6e3-7dcc34ed8036\") " pod="openshift-insights/insights-runtime-extractor-4cjfp" Jul 19 16:42:45.486552 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:45.486532 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7387825e-ee08-4bcd-a6e3-7dcc34ed8036-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4cjfp\" (UID: \"7387825e-ee08-4bcd-a6e3-7dcc34ed8036\") " pod="openshift-insights/insights-runtime-extractor-4cjfp" Jul 19 16:42:45.513721 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:45.513690 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4cjfp" Jul 19 16:42:45.645093 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:45.645040 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4cjfp"] Jul 19 16:42:45.648028 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:42:45.647990 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7387825e_ee08_4bcd_a6e3_7dcc34ed8036.slice/crio-c148277d0c9fdc015ac2cafa30bd49bdd91a09106c82f5e75ae224af9d4b44ab WatchSource:0}: Error finding container c148277d0c9fdc015ac2cafa30bd49bdd91a09106c82f5e75ae224af9d4b44ab: Status 404 returned error can't find the container with id c148277d0c9fdc015ac2cafa30bd49bdd91a09106c82f5e75ae224af9d4b44ab Jul 19 16:42:46.439805 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:46.439764 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4cjfp" event={"ID":"7387825e-ee08-4bcd-a6e3-7dcc34ed8036","Type":"ContainerStarted","Data":"c148277d0c9fdc015ac2cafa30bd49bdd91a09106c82f5e75ae224af9d4b44ab"} Jul 19 16:42:50.427744 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:50.427668 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5c456d485-qvzgh"] Jul 19 16:42:51.461401 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:51.461356 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4cjfp" event={"ID":"7387825e-ee08-4bcd-a6e3-7dcc34ed8036","Type":"ContainerStarted","Data":"5384a84e4b6a7e2052362212e956a7be17e66a7ec3237843a2e36fe3672443fc"} Jul 19 16:42:54.474758 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:54.474711 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4cjfp" event={"ID":"7387825e-ee08-4bcd-a6e3-7dcc34ed8036","Type":"ContainerStarted","Data":"4595e1c42ddf4e6c11c705f1e542423f484dc379757074fc9cc181efdc9a3baf"} Jul 19 16:42:58.489523 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:58.489483 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4cjfp" event={"ID":"7387825e-ee08-4bcd-a6e3-7dcc34ed8036","Type":"ContainerStarted","Data":"739d12008ab17d5d3796e5644c5f06d5294eab13c513e4f0ecfa7b798db39a47"} Jul 19 16:42:58.506704 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:42:58.506651 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-4cjfp" podStartSLOduration=17.691784385 podStartE2EDuration="29.506634221s" totalImagesPullingTime="11.814849836s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 16:42:29 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 16:42:58.505264763 +0000 UTC m=+273.491840248" watchObservedRunningTime="2026-07-19 16:42:58.506634221 +0000 UTC m=+273.493209705" Jul 19 16:43:00.434004 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:00.433968 2583 patch_prober.go:28] interesting pod/image-registry-5c456d485-qvzgh container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 19 16:43:00.434395 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:00.434020 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-5c456d485-qvzgh" podUID="ea535ad5-b3c6-46f0-9444-424035178c81" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 19 16:43:07.164622 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:43:07.164568 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-znbhx" podUID="9e1a462c-d8cc-4635-9eac-2271aed8e352" Jul 19 16:43:07.518955 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:07.518880 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-znbhx" Jul 19 16:43:09.794302 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:09.794271 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-69c5f65c6-5jtsm"] Jul 19 16:43:09.797278 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:09.797255 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69c5f65c6-5jtsm" Jul 19 16:43:09.799378 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:09.799357 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 19 16:43:09.800306 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:09.800286 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 19 16:43:09.800306 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:09.800295 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 19 16:43:09.800482 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:09.800344 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 19 16:43:09.800482 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:09.800346 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 19 16:43:09.800482 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:09.800422 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 19 16:43:09.800640 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:09.800626 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-l4jtz\"" Jul 19 16:43:09.800690 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:09.800671 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 19 16:43:09.804328 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:09.804308 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 19 16:43:09.807357 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:09.807337 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-69c5f65c6-5jtsm"] Jul 19 16:43:09.868963 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:09.868932 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f37a3556-1276-4995-9739-32f6fece238d-service-ca\") pod \"console-69c5f65c6-5jtsm\" (UID: \"f37a3556-1276-4995-9739-32f6fece238d\") " pod="openshift-console/console-69c5f65c6-5jtsm" Jul 19 16:43:09.869165 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:09.868973 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f37a3556-1276-4995-9739-32f6fece238d-oauth-serving-cert\") pod \"console-69c5f65c6-5jtsm\" (UID: \"f37a3556-1276-4995-9739-32f6fece238d\") " pod="openshift-console/console-69c5f65c6-5jtsm" Jul 19 16:43:09.869165 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:09.868992 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f37a3556-1276-4995-9739-32f6fece238d-trusted-ca-bundle\") pod \"console-69c5f65c6-5jtsm\" (UID: \"f37a3556-1276-4995-9739-32f6fece238d\") " pod="openshift-console/console-69c5f65c6-5jtsm" Jul 19 16:43:09.869165 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:09.869143 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f37a3556-1276-4995-9739-32f6fece238d-console-oauth-config\") pod \"console-69c5f65c6-5jtsm\" (UID: \"f37a3556-1276-4995-9739-32f6fece238d\") " pod="openshift-console/console-69c5f65c6-5jtsm" Jul 19 16:43:09.869332 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:09.869181 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9h2xg\" (UniqueName: \"kubernetes.io/projected/f37a3556-1276-4995-9739-32f6fece238d-kube-api-access-9h2xg\") pod \"console-69c5f65c6-5jtsm\" (UID: \"f37a3556-1276-4995-9739-32f6fece238d\") " pod="openshift-console/console-69c5f65c6-5jtsm" Jul 19 16:43:09.869332 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:09.869215 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f37a3556-1276-4995-9739-32f6fece238d-console-serving-cert\") pod \"console-69c5f65c6-5jtsm\" (UID: \"f37a3556-1276-4995-9739-32f6fece238d\") " pod="openshift-console/console-69c5f65c6-5jtsm" Jul 19 16:43:09.869332 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:09.869243 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f37a3556-1276-4995-9739-32f6fece238d-console-config\") pod \"console-69c5f65c6-5jtsm\" (UID: \"f37a3556-1276-4995-9739-32f6fece238d\") " pod="openshift-console/console-69c5f65c6-5jtsm" Jul 19 16:43:09.970566 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:09.970512 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f37a3556-1276-4995-9739-32f6fece238d-console-oauth-config\") pod \"console-69c5f65c6-5jtsm\" (UID: \"f37a3556-1276-4995-9739-32f6fece238d\") " pod="openshift-console/console-69c5f65c6-5jtsm" Jul 19 16:43:09.970566 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:09.970570 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9h2xg\" (UniqueName: \"kubernetes.io/projected/f37a3556-1276-4995-9739-32f6fece238d-kube-api-access-9h2xg\") pod \"console-69c5f65c6-5jtsm\" (UID: \"f37a3556-1276-4995-9739-32f6fece238d\") " pod="openshift-console/console-69c5f65c6-5jtsm" Jul 19 16:43:09.970822 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:09.970604 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f37a3556-1276-4995-9739-32f6fece238d-console-serving-cert\") pod \"console-69c5f65c6-5jtsm\" (UID: \"f37a3556-1276-4995-9739-32f6fece238d\") " pod="openshift-console/console-69c5f65c6-5jtsm" Jul 19 16:43:09.970822 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:09.970628 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f37a3556-1276-4995-9739-32f6fece238d-console-config\") pod \"console-69c5f65c6-5jtsm\" (UID: \"f37a3556-1276-4995-9739-32f6fece238d\") " pod="openshift-console/console-69c5f65c6-5jtsm" Jul 19 16:43:09.970822 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:09.970686 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f37a3556-1276-4995-9739-32f6fece238d-service-ca\") pod \"console-69c5f65c6-5jtsm\" (UID: \"f37a3556-1276-4995-9739-32f6fece238d\") " pod="openshift-console/console-69c5f65c6-5jtsm" Jul 19 16:43:09.970822 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:09.970720 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f37a3556-1276-4995-9739-32f6fece238d-oauth-serving-cert\") pod \"console-69c5f65c6-5jtsm\" (UID: \"f37a3556-1276-4995-9739-32f6fece238d\") " pod="openshift-console/console-69c5f65c6-5jtsm" Jul 19 16:43:09.970822 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:09.970744 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f37a3556-1276-4995-9739-32f6fece238d-trusted-ca-bundle\") pod \"console-69c5f65c6-5jtsm\" (UID: \"f37a3556-1276-4995-9739-32f6fece238d\") " pod="openshift-console/console-69c5f65c6-5jtsm" Jul 19 16:43:09.971572 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:09.971544 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f37a3556-1276-4995-9739-32f6fece238d-oauth-serving-cert\") pod \"console-69c5f65c6-5jtsm\" (UID: \"f37a3556-1276-4995-9739-32f6fece238d\") " pod="openshift-console/console-69c5f65c6-5jtsm" Jul 19 16:43:09.971684 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:09.971588 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f37a3556-1276-4995-9739-32f6fece238d-console-config\") pod \"console-69c5f65c6-5jtsm\" (UID: \"f37a3556-1276-4995-9739-32f6fece238d\") " pod="openshift-console/console-69c5f65c6-5jtsm" Jul 19 16:43:09.971684 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:09.971588 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f37a3556-1276-4995-9739-32f6fece238d-service-ca\") pod \"console-69c5f65c6-5jtsm\" (UID: \"f37a3556-1276-4995-9739-32f6fece238d\") " pod="openshift-console/console-69c5f65c6-5jtsm" Jul 19 16:43:09.971920 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:09.971901 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f37a3556-1276-4995-9739-32f6fece238d-trusted-ca-bundle\") pod \"console-69c5f65c6-5jtsm\" (UID: \"f37a3556-1276-4995-9739-32f6fece238d\") " pod="openshift-console/console-69c5f65c6-5jtsm" Jul 19 16:43:09.973032 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:09.973005 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f37a3556-1276-4995-9739-32f6fece238d-console-oauth-config\") pod \"console-69c5f65c6-5jtsm\" (UID: \"f37a3556-1276-4995-9739-32f6fece238d\") " pod="openshift-console/console-69c5f65c6-5jtsm" Jul 19 16:43:09.973149 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:09.973133 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f37a3556-1276-4995-9739-32f6fece238d-console-serving-cert\") pod \"console-69c5f65c6-5jtsm\" (UID: \"f37a3556-1276-4995-9739-32f6fece238d\") " pod="openshift-console/console-69c5f65c6-5jtsm" Jul 19 16:43:09.977888 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:09.977867 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9h2xg\" (UniqueName: \"kubernetes.io/projected/f37a3556-1276-4995-9739-32f6fece238d-kube-api-access-9h2xg\") pod \"console-69c5f65c6-5jtsm\" (UID: \"f37a3556-1276-4995-9739-32f6fece238d\") " pod="openshift-console/console-69c5f65c6-5jtsm" Jul 19 16:43:10.107052 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:10.107016 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69c5f65c6-5jtsm" Jul 19 16:43:10.229988 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:10.229960 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-69c5f65c6-5jtsm"] Jul 19 16:43:10.233873 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:43:10.233848 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf37a3556_1276_4995_9739_32f6fece238d.slice/crio-78aafccce12616b4a32777214b958556075c236d9a845f60b107daeb3baa6191 WatchSource:0}: Error finding container 78aafccce12616b4a32777214b958556075c236d9a845f60b107daeb3baa6191: Status 404 returned error can't find the container with id 78aafccce12616b4a32777214b958556075c236d9a845f60b107daeb3baa6191 Jul 19 16:43:10.273917 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:10.273890 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e1a462c-d8cc-4635-9eac-2271aed8e352-metrics-tls\") pod \"dns-default-znbhx\" (UID: \"9e1a462c-d8cc-4635-9eac-2271aed8e352\") " pod="openshift-dns/dns-default-znbhx" Jul 19 16:43:10.276110 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:10.276092 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e1a462c-d8cc-4635-9eac-2271aed8e352-metrics-tls\") pod \"dns-default-znbhx\" (UID: \"9e1a462c-d8cc-4635-9eac-2271aed8e352\") " pod="openshift-dns/dns-default-znbhx" Jul 19 16:43:10.432780 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:10.432699 2583 patch_prober.go:28] interesting pod/image-registry-5c456d485-qvzgh container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 19 16:43:10.432780 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:10.432755 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-5c456d485-qvzgh" podUID="ea535ad5-b3c6-46f0-9444-424035178c81" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 19 16:43:10.521708 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:10.521678 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-pk4kh\"" Jul 19 16:43:10.529797 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:10.529768 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-znbhx" Jul 19 16:43:10.530087 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:10.530045 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69c5f65c6-5jtsm" event={"ID":"f37a3556-1276-4995-9739-32f6fece238d","Type":"ContainerStarted","Data":"78aafccce12616b4a32777214b958556075c236d9a845f60b107daeb3baa6191"} Jul 19 16:43:10.652131 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:10.652097 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-znbhx"] Jul 19 16:43:10.655842 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:43:10.655817 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e1a462c_d8cc_4635_9eac_2271aed8e352.slice/crio-bff2de2a9e4974286239a0c93752372737e0aacbf8cdfb3a2d025e478131ab65 WatchSource:0}: Error finding container bff2de2a9e4974286239a0c93752372737e0aacbf8cdfb3a2d025e478131ab65: Status 404 returned error can't find the container with id bff2de2a9e4974286239a0c93752372737e0aacbf8cdfb3a2d025e478131ab65 Jul 19 16:43:11.535161 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:11.535124 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-znbhx" event={"ID":"9e1a462c-d8cc-4635-9eac-2271aed8e352","Type":"ContainerStarted","Data":"bff2de2a9e4974286239a0c93752372737e0aacbf8cdfb3a2d025e478131ab65"} Jul 19 16:43:12.906600 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:12.906567 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-p2ghv"] Jul 19 16:43:12.911512 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:12.911484 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-p2ghv" Jul 19 16:43:12.913997 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:12.913970 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 19 16:43:12.913997 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:12.913991 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 19 16:43:12.914220 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:12.914035 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 19 16:43:12.914220 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:12.914046 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 19 16:43:12.915214 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:12.915195 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 19 16:43:12.915214 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:12.915208 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-m9n6g\"" Jul 19 16:43:12.919819 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:12.919774 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-p2ghv"] Jul 19 16:43:13.000041 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:12.999998 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c9c72ebb-d55e-4020-b6e0-e9a384bae506-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-p2ghv\" (UID: \"c9c72ebb-d55e-4020-b6e0-e9a384bae506\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-p2ghv" Jul 19 16:43:13.000219 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:13.000159 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnrjg\" (UniqueName: \"kubernetes.io/projected/c9c72ebb-d55e-4020-b6e0-e9a384bae506-kube-api-access-nnrjg\") pod \"prometheus-operator-5945b5cfd-p2ghv\" (UID: \"c9c72ebb-d55e-4020-b6e0-e9a384bae506\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-p2ghv" Jul 19 16:43:13.000283 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:13.000215 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c9c72ebb-d55e-4020-b6e0-e9a384bae506-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-p2ghv\" (UID: \"c9c72ebb-d55e-4020-b6e0-e9a384bae506\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-p2ghv" Jul 19 16:43:13.000338 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:13.000302 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c9c72ebb-d55e-4020-b6e0-e9a384bae506-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-p2ghv\" (UID: \"c9c72ebb-d55e-4020-b6e0-e9a384bae506\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-p2ghv" Jul 19 16:43:13.101554 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:13.101515 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c9c72ebb-d55e-4020-b6e0-e9a384bae506-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-p2ghv\" (UID: \"c9c72ebb-d55e-4020-b6e0-e9a384bae506\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-p2ghv" Jul 19 16:43:13.101554 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:13.101561 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c9c72ebb-d55e-4020-b6e0-e9a384bae506-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-p2ghv\" (UID: \"c9c72ebb-d55e-4020-b6e0-e9a384bae506\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-p2ghv" Jul 19 16:43:13.101802 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:13.101585 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c9c72ebb-d55e-4020-b6e0-e9a384bae506-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-p2ghv\" (UID: \"c9c72ebb-d55e-4020-b6e0-e9a384bae506\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-p2ghv" Jul 19 16:43:13.101802 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:13.101649 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnrjg\" (UniqueName: \"kubernetes.io/projected/c9c72ebb-d55e-4020-b6e0-e9a384bae506-kube-api-access-nnrjg\") pod \"prometheus-operator-5945b5cfd-p2ghv\" (UID: \"c9c72ebb-d55e-4020-b6e0-e9a384bae506\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-p2ghv" Jul 19 16:43:13.102487 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:13.102459 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c9c72ebb-d55e-4020-b6e0-e9a384bae506-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-p2ghv\" (UID: \"c9c72ebb-d55e-4020-b6e0-e9a384bae506\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-p2ghv" Jul 19 16:43:13.104251 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:13.104229 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c9c72ebb-d55e-4020-b6e0-e9a384bae506-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-p2ghv\" (UID: \"c9c72ebb-d55e-4020-b6e0-e9a384bae506\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-p2ghv" Jul 19 16:43:13.104374 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:13.104253 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c9c72ebb-d55e-4020-b6e0-e9a384bae506-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-p2ghv\" (UID: \"c9c72ebb-d55e-4020-b6e0-e9a384bae506\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-p2ghv" Jul 19 16:43:13.109516 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:13.109495 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnrjg\" (UniqueName: \"kubernetes.io/projected/c9c72ebb-d55e-4020-b6e0-e9a384bae506-kube-api-access-nnrjg\") pod \"prometheus-operator-5945b5cfd-p2ghv\" (UID: \"c9c72ebb-d55e-4020-b6e0-e9a384bae506\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-p2ghv" Jul 19 16:43:13.222884 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:13.222791 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-p2ghv" Jul 19 16:43:13.513927 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:13.513845 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-p2ghv"] Jul 19 16:43:13.543823 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:13.543768 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69c5f65c6-5jtsm" event={"ID":"f37a3556-1276-4995-9739-32f6fece238d","Type":"ContainerStarted","Data":"8df436a190ec81d626af1fb2b76429878efbe21d368caadb54eb3468ae687d97"} Jul 19 16:43:13.545597 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:13.545567 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-znbhx" event={"ID":"9e1a462c-d8cc-4635-9eac-2271aed8e352","Type":"ContainerStarted","Data":"ccb3e88fd183d72c76382f1f6d9476507d1175aa23d92de6d18c2fdb52de10c1"} Jul 19 16:43:13.546798 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:13.546770 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-p2ghv" event={"ID":"c9c72ebb-d55e-4020-b6e0-e9a384bae506","Type":"ContainerStarted","Data":"76ac0a8b9a0ced3f45a56432a7e9db910eebc15f2adaa557dbbca202aef69df4"} Jul 19 16:43:13.562286 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:13.562244 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-69c5f65c6-5jtsm" podStartSLOduration=1.414437232 podStartE2EDuration="4.562229193s" totalImagesPullingTime="3.147791961s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 16:43:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 16:43:13.561136655 +0000 UTC m=+288.547712139" watchObservedRunningTime="2026-07-19 16:43:13.562229193 +0000 UTC m=+288.548804678" Jul 19 16:43:14.552630 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:14.552587 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-znbhx" event={"ID":"9e1a462c-d8cc-4635-9eac-2271aed8e352","Type":"ContainerStarted","Data":"ffcdd3676bd9aee40c0716ee982b01d81d57860dedb704e091e67c18a4b38350"} Jul 19 16:43:14.553114 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:14.552804 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-znbhx" Jul 19 16:43:14.569971 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:14.569917 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-znbhx" podStartSLOduration=251.850310237 podStartE2EDuration="4m14.569901601s" totalImagesPullingTime="2.719591364s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 16:39:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 16:43:14.567986841 +0000 UTC m=+289.554562341" watchObservedRunningTime="2026-07-19 16:43:14.569901601 +0000 UTC m=+289.556477084" Jul 19 16:43:15.456178 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:15.456134 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5c456d485-qvzgh" podUID="ea535ad5-b3c6-46f0-9444-424035178c81" containerName="registry" containerID="cri-o://5afaa8655db886b2a37ab6c5fc2af3a0e607b1d193ab71ae78a557d9e3bef513" gracePeriod=30 Jul 19 16:43:15.561206 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:15.561153 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-p2ghv" event={"ID":"c9c72ebb-d55e-4020-b6e0-e9a384bae506","Type":"ContainerStarted","Data":"2ee91c00e5560c65082d093c411dc713c44a433cb20c6a5819c2e403e4ba1630"} Jul 19 16:43:15.561566 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:15.561261 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-p2ghv" event={"ID":"c9c72ebb-d55e-4020-b6e0-e9a384bae506","Type":"ContainerStarted","Data":"c4b6079094fb366f4b67d3b1dfa67fc2e0ea2a5cd7c461b0242bbcb0e5badda6"} Jul 19 16:43:15.578169 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:15.578114 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-5945b5cfd-p2ghv" podStartSLOduration=2.25378226 podStartE2EDuration="3.578099256s" totalImagesPullingTime="1.324316996s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 16:43:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 16:43:15.576598416 +0000 UTC m=+290.563173899" watchObservedRunningTime="2026-07-19 16:43:15.578099256 +0000 UTC m=+290.564674739" Jul 19 16:43:15.694633 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:15.694609 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5c456d485-qvzgh" Jul 19 16:43:15.823376 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:15.823290 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ea535ad5-b3c6-46f0-9444-424035178c81-ca-trust-extracted\") pod \"ea535ad5-b3c6-46f0-9444-424035178c81\" (UID: \"ea535ad5-b3c6-46f0-9444-424035178c81\") " Jul 19 16:43:15.823376 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:15.823351 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cwpmw\" (UniqueName: \"kubernetes.io/projected/ea535ad5-b3c6-46f0-9444-424035178c81-kube-api-access-cwpmw\") pod \"ea535ad5-b3c6-46f0-9444-424035178c81\" (UID: \"ea535ad5-b3c6-46f0-9444-424035178c81\") " Jul 19 16:43:15.823376 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:15.823375 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ea535ad5-b3c6-46f0-9444-424035178c81-installation-pull-secrets\") pod \"ea535ad5-b3c6-46f0-9444-424035178c81\" (UID: \"ea535ad5-b3c6-46f0-9444-424035178c81\") " Jul 19 16:43:15.823643 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:15.823397 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ea535ad5-b3c6-46f0-9444-424035178c81-image-registry-private-configuration\") pod \"ea535ad5-b3c6-46f0-9444-424035178c81\" (UID: \"ea535ad5-b3c6-46f0-9444-424035178c81\") " Jul 19 16:43:15.823643 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:15.823430 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ea535ad5-b3c6-46f0-9444-424035178c81-registry-certificates\") pod \"ea535ad5-b3c6-46f0-9444-424035178c81\" (UID: \"ea535ad5-b3c6-46f0-9444-424035178c81\") " Jul 19 16:43:15.823643 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:15.823453 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ea535ad5-b3c6-46f0-9444-424035178c81-trusted-ca\") pod \"ea535ad5-b3c6-46f0-9444-424035178c81\" (UID: \"ea535ad5-b3c6-46f0-9444-424035178c81\") " Jul 19 16:43:15.823643 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:15.823476 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ea535ad5-b3c6-46f0-9444-424035178c81-registry-tls\") pod \"ea535ad5-b3c6-46f0-9444-424035178c81\" (UID: \"ea535ad5-b3c6-46f0-9444-424035178c81\") " Jul 19 16:43:15.823643 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:15.823502 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ea535ad5-b3c6-46f0-9444-424035178c81-bound-sa-token\") pod \"ea535ad5-b3c6-46f0-9444-424035178c81\" (UID: \"ea535ad5-b3c6-46f0-9444-424035178c81\") " Jul 19 16:43:15.824520 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:15.824436 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea535ad5-b3c6-46f0-9444-424035178c81-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "ea535ad5-b3c6-46f0-9444-424035178c81" (UID: "ea535ad5-b3c6-46f0-9444-424035178c81"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 19 16:43:15.825277 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:15.825233 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea535ad5-b3c6-46f0-9444-424035178c81-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "ea535ad5-b3c6-46f0-9444-424035178c81" (UID: "ea535ad5-b3c6-46f0-9444-424035178c81"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 19 16:43:15.826144 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:15.826107 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea535ad5-b3c6-46f0-9444-424035178c81-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "ea535ad5-b3c6-46f0-9444-424035178c81" (UID: "ea535ad5-b3c6-46f0-9444-424035178c81"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 19 16:43:15.826225 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:15.826140 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea535ad5-b3c6-46f0-9444-424035178c81-kube-api-access-cwpmw" (OuterVolumeSpecName: "kube-api-access-cwpmw") pod "ea535ad5-b3c6-46f0-9444-424035178c81" (UID: "ea535ad5-b3c6-46f0-9444-424035178c81"). InnerVolumeSpecName "kube-api-access-cwpmw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 19 16:43:15.826285 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:15.826261 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea535ad5-b3c6-46f0-9444-424035178c81-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "ea535ad5-b3c6-46f0-9444-424035178c81" (UID: "ea535ad5-b3c6-46f0-9444-424035178c81"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 19 16:43:15.826285 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:15.826269 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea535ad5-b3c6-46f0-9444-424035178c81-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "ea535ad5-b3c6-46f0-9444-424035178c81" (UID: "ea535ad5-b3c6-46f0-9444-424035178c81"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 19 16:43:15.826639 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:15.826608 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea535ad5-b3c6-46f0-9444-424035178c81-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "ea535ad5-b3c6-46f0-9444-424035178c81" (UID: "ea535ad5-b3c6-46f0-9444-424035178c81"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 19 16:43:15.832331 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:15.832307 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea535ad5-b3c6-46f0-9444-424035178c81-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "ea535ad5-b3c6-46f0-9444-424035178c81" (UID: "ea535ad5-b3c6-46f0-9444-424035178c81"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 16:43:15.924432 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:15.924394 2583 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ea535ad5-b3c6-46f0-9444-424035178c81-image-registry-private-configuration\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 16:43:15.924432 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:15.924427 2583 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ea535ad5-b3c6-46f0-9444-424035178c81-registry-certificates\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 16:43:15.924619 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:15.924442 2583 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ea535ad5-b3c6-46f0-9444-424035178c81-trusted-ca\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 16:43:15.924619 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:15.924455 2583 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ea535ad5-b3c6-46f0-9444-424035178c81-registry-tls\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 16:43:15.924619 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:15.924467 2583 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ea535ad5-b3c6-46f0-9444-424035178c81-bound-sa-token\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 16:43:15.924619 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:15.924478 2583 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ea535ad5-b3c6-46f0-9444-424035178c81-ca-trust-extracted\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 16:43:15.924619 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:15.924488 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cwpmw\" (UniqueName: \"kubernetes.io/projected/ea535ad5-b3c6-46f0-9444-424035178c81-kube-api-access-cwpmw\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 16:43:15.924619 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:15.924499 2583 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ea535ad5-b3c6-46f0-9444-424035178c81-installation-pull-secrets\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 16:43:16.565606 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:16.565571 2583 generic.go:356] "Generic (PLEG): container finished" podID="ea535ad5-b3c6-46f0-9444-424035178c81" containerID="5afaa8655db886b2a37ab6c5fc2af3a0e607b1d193ab71ae78a557d9e3bef513" exitCode=0 Jul 19 16:43:16.565606 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:16.565610 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5c456d485-qvzgh" Jul 19 16:43:16.566160 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:16.565657 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5c456d485-qvzgh" event={"ID":"ea535ad5-b3c6-46f0-9444-424035178c81","Type":"ContainerDied","Data":"5afaa8655db886b2a37ab6c5fc2af3a0e607b1d193ab71ae78a557d9e3bef513"} Jul 19 16:43:16.566160 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:16.565714 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5c456d485-qvzgh" event={"ID":"ea535ad5-b3c6-46f0-9444-424035178c81","Type":"ContainerDied","Data":"ba620364612cbcb78f6df4204ea86a988723e1191ab83667087ae615bb9bad25"} Jul 19 16:43:16.566160 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:16.565743 2583 scope.go:117] "RemoveContainer" containerID="5afaa8655db886b2a37ab6c5fc2af3a0e607b1d193ab71ae78a557d9e3bef513" Jul 19 16:43:16.574468 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:16.574446 2583 scope.go:117] "RemoveContainer" containerID="5afaa8655db886b2a37ab6c5fc2af3a0e607b1d193ab71ae78a557d9e3bef513" Jul 19 16:43:16.574748 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:43:16.574726 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5afaa8655db886b2a37ab6c5fc2af3a0e607b1d193ab71ae78a557d9e3bef513\": container with ID starting with 5afaa8655db886b2a37ab6c5fc2af3a0e607b1d193ab71ae78a557d9e3bef513 not found: ID does not exist" containerID="5afaa8655db886b2a37ab6c5fc2af3a0e607b1d193ab71ae78a557d9e3bef513" Jul 19 16:43:16.574823 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:16.574760 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5afaa8655db886b2a37ab6c5fc2af3a0e607b1d193ab71ae78a557d9e3bef513"} err="failed to get container status \"5afaa8655db886b2a37ab6c5fc2af3a0e607b1d193ab71ae78a557d9e3bef513\": rpc error: code = NotFound desc = could not find container \"5afaa8655db886b2a37ab6c5fc2af3a0e607b1d193ab71ae78a557d9e3bef513\": container with ID starting with 5afaa8655db886b2a37ab6c5fc2af3a0e607b1d193ab71ae78a557d9e3bef513 not found: ID does not exist" Jul 19 16:43:16.584596 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:16.584570 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5c456d485-qvzgh"] Jul 19 16:43:16.590882 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:16.590861 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5c456d485-qvzgh"] Jul 19 16:43:17.292041 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.292007 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-bnvf8"] Jul 19 16:43:17.292287 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.292269 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea535ad5-b3c6-46f0-9444-424035178c81" containerName="registry" Jul 19 16:43:17.292287 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.292283 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea535ad5-b3c6-46f0-9444-424035178c81" containerName="registry" Jul 19 16:43:17.292357 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.292333 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea535ad5-b3c6-46f0-9444-424035178c81" containerName="registry" Jul 19 16:43:17.295493 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.295471 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-bnvf8" Jul 19 16:43:17.296557 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.296533 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-4x8vz"] Jul 19 16:43:17.298315 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.298296 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-2zx5t\"" Jul 19 16:43:17.298426 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.298312 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 19 16:43:17.298426 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.298365 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 19 16:43:17.298426 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.298403 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 19 16:43:17.299781 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.299765 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-4x8vz" Jul 19 16:43:17.301852 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.301835 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 19 16:43:17.302321 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.302091 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 19 16:43:17.302321 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.302172 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-5kc4x\"" Jul 19 16:43:17.302443 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.302349 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 19 16:43:17.305958 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.305924 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-bnvf8"] Jul 19 16:43:17.436881 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.436841 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dfeed6fd-e812-449b-ae0e-9746e3953fa3-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-bnvf8\" (UID: \"dfeed6fd-e812-449b-ae0e-9746e3953fa3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-bnvf8" Jul 19 16:43:17.436881 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.436883 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8-root\") pod \"node-exporter-4x8vz\" (UID: \"b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8\") " pod="openshift-monitoring/node-exporter-4x8vz" Jul 19 16:43:17.437106 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.436914 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/dfeed6fd-e812-449b-ae0e-9746e3953fa3-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-bnvf8\" (UID: \"dfeed6fd-e812-449b-ae0e-9746e3953fa3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-bnvf8" Jul 19 16:43:17.437106 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.436952 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8-node-exporter-textfile\") pod \"node-exporter-4x8vz\" (UID: \"b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8\") " pod="openshift-monitoring/node-exporter-4x8vz" Jul 19 16:43:17.437106 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.436973 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8-node-exporter-accelerators-collector-config\") pod \"node-exporter-4x8vz\" (UID: \"b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8\") " pod="openshift-monitoring/node-exporter-4x8vz" Jul 19 16:43:17.437106 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.437002 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8-node-exporter-tls\") pod \"node-exporter-4x8vz\" (UID: \"b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8\") " pod="openshift-monitoring/node-exporter-4x8vz" Jul 19 16:43:17.437106 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.437041 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bg2sv\" (UniqueName: \"kubernetes.io/projected/dfeed6fd-e812-449b-ae0e-9746e3953fa3-kube-api-access-bg2sv\") pod \"kube-state-metrics-56c5594674-bnvf8\" (UID: \"dfeed6fd-e812-449b-ae0e-9746e3953fa3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-bnvf8" Jul 19 16:43:17.437106 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.437083 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9x8f\" (UniqueName: \"kubernetes.io/projected/b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8-kube-api-access-m9x8f\") pod \"node-exporter-4x8vz\" (UID: \"b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8\") " pod="openshift-monitoring/node-exporter-4x8vz" Jul 19 16:43:17.437287 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.437109 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8-node-exporter-wtmp\") pod \"node-exporter-4x8vz\" (UID: \"b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8\") " pod="openshift-monitoring/node-exporter-4x8vz" Jul 19 16:43:17.437287 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.437130 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8-metrics-client-ca\") pod \"node-exporter-4x8vz\" (UID: \"b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8\") " pod="openshift-monitoring/node-exporter-4x8vz" Jul 19 16:43:17.437287 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.437147 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4x8vz\" (UID: \"b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8\") " pod="openshift-monitoring/node-exporter-4x8vz" Jul 19 16:43:17.437287 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.437188 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8-sys\") pod \"node-exporter-4x8vz\" (UID: \"b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8\") " pod="openshift-monitoring/node-exporter-4x8vz" Jul 19 16:43:17.437287 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.437228 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/dfeed6fd-e812-449b-ae0e-9746e3953fa3-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-bnvf8\" (UID: \"dfeed6fd-e812-449b-ae0e-9746e3953fa3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-bnvf8" Jul 19 16:43:17.437287 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.437265 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/dfeed6fd-e812-449b-ae0e-9746e3953fa3-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-bnvf8\" (UID: \"dfeed6fd-e812-449b-ae0e-9746e3953fa3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-bnvf8" Jul 19 16:43:17.437287 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.437284 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dfeed6fd-e812-449b-ae0e-9746e3953fa3-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-bnvf8\" (UID: \"dfeed6fd-e812-449b-ae0e-9746e3953fa3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-bnvf8" Jul 19 16:43:17.538220 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.538186 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8-root\") pod \"node-exporter-4x8vz\" (UID: \"b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8\") " pod="openshift-monitoring/node-exporter-4x8vz" Jul 19 16:43:17.538220 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.538223 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/dfeed6fd-e812-449b-ae0e-9746e3953fa3-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-bnvf8\" (UID: \"dfeed6fd-e812-449b-ae0e-9746e3953fa3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-bnvf8" Jul 19 16:43:17.538431 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.538247 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8-node-exporter-textfile\") pod \"node-exporter-4x8vz\" (UID: \"b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8\") " pod="openshift-monitoring/node-exporter-4x8vz" Jul 19 16:43:17.538431 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.538263 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8-node-exporter-accelerators-collector-config\") pod \"node-exporter-4x8vz\" (UID: \"b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8\") " pod="openshift-monitoring/node-exporter-4x8vz" Jul 19 16:43:17.538431 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.538301 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8-node-exporter-tls\") pod \"node-exporter-4x8vz\" (UID: \"b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8\") " pod="openshift-monitoring/node-exporter-4x8vz" Jul 19 16:43:17.538431 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.538337 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bg2sv\" (UniqueName: \"kubernetes.io/projected/dfeed6fd-e812-449b-ae0e-9746e3953fa3-kube-api-access-bg2sv\") pod \"kube-state-metrics-56c5594674-bnvf8\" (UID: \"dfeed6fd-e812-449b-ae0e-9746e3953fa3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-bnvf8" Jul 19 16:43:17.538431 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:43:17.538348 2583 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 19 16:43:17.538431 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.538365 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9x8f\" (UniqueName: \"kubernetes.io/projected/b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8-kube-api-access-m9x8f\") pod \"node-exporter-4x8vz\" (UID: \"b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8\") " pod="openshift-monitoring/node-exporter-4x8vz" Jul 19 16:43:17.538431 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.538298 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8-root\") pod \"node-exporter-4x8vz\" (UID: \"b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8\") " pod="openshift-monitoring/node-exporter-4x8vz" Jul 19 16:43:17.538431 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.538393 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8-node-exporter-wtmp\") pod \"node-exporter-4x8vz\" (UID: \"b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8\") " pod="openshift-monitoring/node-exporter-4x8vz" Jul 19 16:43:17.538431 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:43:17.538423 2583 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 19 16:43:17.538431 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:43:17.538432 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfeed6fd-e812-449b-ae0e-9746e3953fa3-kube-state-metrics-tls podName:dfeed6fd-e812-449b-ae0e-9746e3953fa3 nodeName:}" failed. No retries permitted until 2026-07-19 16:43:18.038410384 +0000 UTC m=+293.024985854 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/dfeed6fd-e812-449b-ae0e-9746e3953fa3-kube-state-metrics-tls") pod "kube-state-metrics-56c5594674-bnvf8" (UID: "dfeed6fd-e812-449b-ae0e-9746e3953fa3") : secret "kube-state-metrics-tls" not found Jul 19 16:43:17.538907 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.538462 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8-metrics-client-ca\") pod \"node-exporter-4x8vz\" (UID: \"b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8\") " pod="openshift-monitoring/node-exporter-4x8vz" Jul 19 16:43:17.538907 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:43:17.538472 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8-node-exporter-tls podName:b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8 nodeName:}" failed. No retries permitted until 2026-07-19 16:43:18.038456189 +0000 UTC m=+293.025031653 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8-node-exporter-tls") pod "node-exporter-4x8vz" (UID: "b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8") : secret "node-exporter-tls" not found Jul 19 16:43:17.538907 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.538503 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4x8vz\" (UID: \"b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8\") " pod="openshift-monitoring/node-exporter-4x8vz" Jul 19 16:43:17.538907 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.538535 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8-sys\") pod \"node-exporter-4x8vz\" (UID: \"b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8\") " pod="openshift-monitoring/node-exporter-4x8vz" Jul 19 16:43:17.538907 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.538574 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/dfeed6fd-e812-449b-ae0e-9746e3953fa3-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-bnvf8\" (UID: \"dfeed6fd-e812-449b-ae0e-9746e3953fa3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-bnvf8" Jul 19 16:43:17.538907 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.538600 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/dfeed6fd-e812-449b-ae0e-9746e3953fa3-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-bnvf8\" (UID: \"dfeed6fd-e812-449b-ae0e-9746e3953fa3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-bnvf8" Jul 19 16:43:17.538907 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.538615 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8-node-exporter-textfile\") pod \"node-exporter-4x8vz\" (UID: \"b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8\") " pod="openshift-monitoring/node-exporter-4x8vz" Jul 19 16:43:17.538907 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.538625 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dfeed6fd-e812-449b-ae0e-9746e3953fa3-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-bnvf8\" (UID: \"dfeed6fd-e812-449b-ae0e-9746e3953fa3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-bnvf8" Jul 19 16:43:17.538907 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.538678 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dfeed6fd-e812-449b-ae0e-9746e3953fa3-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-bnvf8\" (UID: \"dfeed6fd-e812-449b-ae0e-9746e3953fa3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-bnvf8" Jul 19 16:43:17.538907 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.538683 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8-node-exporter-wtmp\") pod \"node-exporter-4x8vz\" (UID: \"b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8\") " pod="openshift-monitoring/node-exporter-4x8vz" Jul 19 16:43:17.538907 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.538786 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8-sys\") pod \"node-exporter-4x8vz\" (UID: \"b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8\") " pod="openshift-monitoring/node-exporter-4x8vz" Jul 19 16:43:17.539408 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.538948 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8-metrics-client-ca\") pod \"node-exporter-4x8vz\" (UID: \"b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8\") " pod="openshift-monitoring/node-exporter-4x8vz" Jul 19 16:43:17.539408 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.539012 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8-node-exporter-accelerators-collector-config\") pod \"node-exporter-4x8vz\" (UID: \"b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8\") " pod="openshift-monitoring/node-exporter-4x8vz" Jul 19 16:43:17.539408 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.539166 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/dfeed6fd-e812-449b-ae0e-9746e3953fa3-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-bnvf8\" (UID: \"dfeed6fd-e812-449b-ae0e-9746e3953fa3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-bnvf8" Jul 19 16:43:17.539509 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.539423 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dfeed6fd-e812-449b-ae0e-9746e3953fa3-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-bnvf8\" (UID: \"dfeed6fd-e812-449b-ae0e-9746e3953fa3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-bnvf8" Jul 19 16:43:17.539640 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.539615 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/dfeed6fd-e812-449b-ae0e-9746e3953fa3-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-bnvf8\" (UID: \"dfeed6fd-e812-449b-ae0e-9746e3953fa3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-bnvf8" Jul 19 16:43:17.540719 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.540698 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4x8vz\" (UID: \"b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8\") " pod="openshift-monitoring/node-exporter-4x8vz" Jul 19 16:43:17.541070 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.541042 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dfeed6fd-e812-449b-ae0e-9746e3953fa3-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-bnvf8\" (UID: \"dfeed6fd-e812-449b-ae0e-9746e3953fa3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-bnvf8" Jul 19 16:43:17.549358 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.549314 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9x8f\" (UniqueName: \"kubernetes.io/projected/b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8-kube-api-access-m9x8f\") pod \"node-exporter-4x8vz\" (UID: \"b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8\") " pod="openshift-monitoring/node-exporter-4x8vz" Jul 19 16:43:17.549961 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.549938 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bg2sv\" (UniqueName: \"kubernetes.io/projected/dfeed6fd-e812-449b-ae0e-9746e3953fa3-kube-api-access-bg2sv\") pod \"kube-state-metrics-56c5594674-bnvf8\" (UID: \"dfeed6fd-e812-449b-ae0e-9746e3953fa3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-bnvf8" Jul 19 16:43:17.600803 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:17.600770 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea535ad5-b3c6-46f0-9444-424035178c81" path="/var/lib/kubelet/pods/ea535ad5-b3c6-46f0-9444-424035178c81/volumes" Jul 19 16:43:18.044206 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:18.044165 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/dfeed6fd-e812-449b-ae0e-9746e3953fa3-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-bnvf8\" (UID: \"dfeed6fd-e812-449b-ae0e-9746e3953fa3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-bnvf8" Jul 19 16:43:18.044393 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:18.044236 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8-node-exporter-tls\") pod \"node-exporter-4x8vz\" (UID: \"b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8\") " pod="openshift-monitoring/node-exporter-4x8vz" Jul 19 16:43:18.046474 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:18.046444 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8-node-exporter-tls\") pod \"node-exporter-4x8vz\" (UID: \"b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8\") " pod="openshift-monitoring/node-exporter-4x8vz" Jul 19 16:43:18.046665 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:18.046641 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/dfeed6fd-e812-449b-ae0e-9746e3953fa3-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-bnvf8\" (UID: \"dfeed6fd-e812-449b-ae0e-9746e3953fa3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-bnvf8" Jul 19 16:43:18.206239 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:18.206201 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-bnvf8" Jul 19 16:43:18.211487 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:18.211466 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-4x8vz" Jul 19 16:43:18.223599 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:43:18.223571 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb569186e_76bf_4bc9_bd0f_b9aa4eddb2a8.slice/crio-bc37fb21ec121a38772a281260f8531b301bce98bd2333f9dacd7efc35c27a22 WatchSource:0}: Error finding container bc37fb21ec121a38772a281260f8531b301bce98bd2333f9dacd7efc35c27a22: Status 404 returned error can't find the container with id bc37fb21ec121a38772a281260f8531b301bce98bd2333f9dacd7efc35c27a22 Jul 19 16:43:18.341961 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:18.341922 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-bnvf8"] Jul 19 16:43:18.344644 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:43:18.344613 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfeed6fd_e812_449b_ae0e_9746e3953fa3.slice/crio-95aa915b4151be0cda81e9e464245b631d049954b023209bdb973e28387c7285 WatchSource:0}: Error finding container 95aa915b4151be0cda81e9e464245b631d049954b023209bdb973e28387c7285: Status 404 returned error can't find the container with id 95aa915b4151be0cda81e9e464245b631d049954b023209bdb973e28387c7285 Jul 19 16:43:18.573943 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:18.573897 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4x8vz" event={"ID":"b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8","Type":"ContainerStarted","Data":"bc37fb21ec121a38772a281260f8531b301bce98bd2333f9dacd7efc35c27a22"} Jul 19 16:43:18.574940 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:18.574912 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-bnvf8" event={"ID":"dfeed6fd-e812-449b-ae0e-9746e3953fa3","Type":"ContainerStarted","Data":"95aa915b4151be0cda81e9e464245b631d049954b023209bdb973e28387c7285"} Jul 19 16:43:19.232554 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:19.232516 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-59bc996f9b-9jtxb"] Jul 19 16:43:19.237719 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:19.237697 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-59bc996f9b-9jtxb" Jul 19 16:43:19.239992 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:19.239968 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 19 16:43:19.240134 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:19.239965 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 19 16:43:19.240134 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:19.240031 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-kswhh\"" Jul 19 16:43:19.240134 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:19.239972 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 19 16:43:19.240408 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:19.240389 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-1q64fp2a68fbb\"" Jul 19 16:43:19.240609 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:19.240594 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 19 16:43:19.240609 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:19.240606 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 19 16:43:19.248426 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:19.248401 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-59bc996f9b-9jtxb"] Jul 19 16:43:19.355101 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:19.355070 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/6cb1f54c-e94c-4d10-be59-e5fb67c1539b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-59bc996f9b-9jtxb\" (UID: \"6cb1f54c-e94c-4d10-be59-e5fb67c1539b\") " pod="openshift-monitoring/thanos-querier-59bc996f9b-9jtxb" Jul 19 16:43:19.355216 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:19.355118 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6cb1f54c-e94c-4d10-be59-e5fb67c1539b-metrics-client-ca\") pod \"thanos-querier-59bc996f9b-9jtxb\" (UID: \"6cb1f54c-e94c-4d10-be59-e5fb67c1539b\") " pod="openshift-monitoring/thanos-querier-59bc996f9b-9jtxb" Jul 19 16:43:19.355216 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:19.355144 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6cb1f54c-e94c-4d10-be59-e5fb67c1539b-secret-grpc-tls\") pod \"thanos-querier-59bc996f9b-9jtxb\" (UID: \"6cb1f54c-e94c-4d10-be59-e5fb67c1539b\") " pod="openshift-monitoring/thanos-querier-59bc996f9b-9jtxb" Jul 19 16:43:19.355216 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:19.355175 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/6cb1f54c-e94c-4d10-be59-e5fb67c1539b-secret-thanos-querier-tls\") pod \"thanos-querier-59bc996f9b-9jtxb\" (UID: \"6cb1f54c-e94c-4d10-be59-e5fb67c1539b\") " pod="openshift-monitoring/thanos-querier-59bc996f9b-9jtxb" Jul 19 16:43:19.355216 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:19.355204 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6cb1f54c-e94c-4d10-be59-e5fb67c1539b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-59bc996f9b-9jtxb\" (UID: \"6cb1f54c-e94c-4d10-be59-e5fb67c1539b\") " pod="openshift-monitoring/thanos-querier-59bc996f9b-9jtxb" Jul 19 16:43:19.355441 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:19.355305 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/6cb1f54c-e94c-4d10-be59-e5fb67c1539b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-59bc996f9b-9jtxb\" (UID: \"6cb1f54c-e94c-4d10-be59-e5fb67c1539b\") " pod="openshift-monitoring/thanos-querier-59bc996f9b-9jtxb" Jul 19 16:43:19.355441 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:19.355402 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-749tr\" (UniqueName: \"kubernetes.io/projected/6cb1f54c-e94c-4d10-be59-e5fb67c1539b-kube-api-access-749tr\") pod \"thanos-querier-59bc996f9b-9jtxb\" (UID: \"6cb1f54c-e94c-4d10-be59-e5fb67c1539b\") " pod="openshift-monitoring/thanos-querier-59bc996f9b-9jtxb" Jul 19 16:43:19.355502 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:19.355445 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6cb1f54c-e94c-4d10-be59-e5fb67c1539b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-59bc996f9b-9jtxb\" (UID: \"6cb1f54c-e94c-4d10-be59-e5fb67c1539b\") " pod="openshift-monitoring/thanos-querier-59bc996f9b-9jtxb" Jul 19 16:43:19.455848 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:19.455806 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-749tr\" (UniqueName: \"kubernetes.io/projected/6cb1f54c-e94c-4d10-be59-e5fb67c1539b-kube-api-access-749tr\") pod \"thanos-querier-59bc996f9b-9jtxb\" (UID: \"6cb1f54c-e94c-4d10-be59-e5fb67c1539b\") " pod="openshift-monitoring/thanos-querier-59bc996f9b-9jtxb" Jul 19 16:43:19.455848 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:19.455856 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6cb1f54c-e94c-4d10-be59-e5fb67c1539b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-59bc996f9b-9jtxb\" (UID: \"6cb1f54c-e94c-4d10-be59-e5fb67c1539b\") " pod="openshift-monitoring/thanos-querier-59bc996f9b-9jtxb" Jul 19 16:43:19.456107 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:19.455967 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/6cb1f54c-e94c-4d10-be59-e5fb67c1539b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-59bc996f9b-9jtxb\" (UID: \"6cb1f54c-e94c-4d10-be59-e5fb67c1539b\") " pod="openshift-monitoring/thanos-querier-59bc996f9b-9jtxb" Jul 19 16:43:19.456107 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:19.455999 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6cb1f54c-e94c-4d10-be59-e5fb67c1539b-metrics-client-ca\") pod \"thanos-querier-59bc996f9b-9jtxb\" (UID: \"6cb1f54c-e94c-4d10-be59-e5fb67c1539b\") " pod="openshift-monitoring/thanos-querier-59bc996f9b-9jtxb" Jul 19 16:43:19.456107 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:19.456015 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6cb1f54c-e94c-4d10-be59-e5fb67c1539b-secret-grpc-tls\") pod \"thanos-querier-59bc996f9b-9jtxb\" (UID: \"6cb1f54c-e94c-4d10-be59-e5fb67c1539b\") " pod="openshift-monitoring/thanos-querier-59bc996f9b-9jtxb" Jul 19 16:43:19.456107 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:19.456033 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/6cb1f54c-e94c-4d10-be59-e5fb67c1539b-secret-thanos-querier-tls\") pod \"thanos-querier-59bc996f9b-9jtxb\" (UID: \"6cb1f54c-e94c-4d10-be59-e5fb67c1539b\") " pod="openshift-monitoring/thanos-querier-59bc996f9b-9jtxb" Jul 19 16:43:19.456107 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:19.456065 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6cb1f54c-e94c-4d10-be59-e5fb67c1539b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-59bc996f9b-9jtxb\" (UID: \"6cb1f54c-e94c-4d10-be59-e5fb67c1539b\") " pod="openshift-monitoring/thanos-querier-59bc996f9b-9jtxb" Jul 19 16:43:19.456385 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:19.456114 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/6cb1f54c-e94c-4d10-be59-e5fb67c1539b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-59bc996f9b-9jtxb\" (UID: \"6cb1f54c-e94c-4d10-be59-e5fb67c1539b\") " pod="openshift-monitoring/thanos-querier-59bc996f9b-9jtxb" Jul 19 16:43:19.456782 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:19.456753 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6cb1f54c-e94c-4d10-be59-e5fb67c1539b-metrics-client-ca\") pod \"thanos-querier-59bc996f9b-9jtxb\" (UID: \"6cb1f54c-e94c-4d10-be59-e5fb67c1539b\") " pod="openshift-monitoring/thanos-querier-59bc996f9b-9jtxb" Jul 19 16:43:19.458649 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:19.458613 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6cb1f54c-e94c-4d10-be59-e5fb67c1539b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-59bc996f9b-9jtxb\" (UID: \"6cb1f54c-e94c-4d10-be59-e5fb67c1539b\") " pod="openshift-monitoring/thanos-querier-59bc996f9b-9jtxb" Jul 19 16:43:19.458865 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:19.458841 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/6cb1f54c-e94c-4d10-be59-e5fb67c1539b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-59bc996f9b-9jtxb\" (UID: \"6cb1f54c-e94c-4d10-be59-e5fb67c1539b\") " pod="openshift-monitoring/thanos-querier-59bc996f9b-9jtxb" Jul 19 16:43:19.458953 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:19.458932 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/6cb1f54c-e94c-4d10-be59-e5fb67c1539b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-59bc996f9b-9jtxb\" (UID: \"6cb1f54c-e94c-4d10-be59-e5fb67c1539b\") " pod="openshift-monitoring/thanos-querier-59bc996f9b-9jtxb" Jul 19 16:43:19.458990 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:19.458932 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6cb1f54c-e94c-4d10-be59-e5fb67c1539b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-59bc996f9b-9jtxb\" (UID: \"6cb1f54c-e94c-4d10-be59-e5fb67c1539b\") " pod="openshift-monitoring/thanos-querier-59bc996f9b-9jtxb" Jul 19 16:43:19.459043 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:19.459025 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6cb1f54c-e94c-4d10-be59-e5fb67c1539b-secret-grpc-tls\") pod \"thanos-querier-59bc996f9b-9jtxb\" (UID: \"6cb1f54c-e94c-4d10-be59-e5fb67c1539b\") " pod="openshift-monitoring/thanos-querier-59bc996f9b-9jtxb" Jul 19 16:43:19.459292 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:19.459273 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/6cb1f54c-e94c-4d10-be59-e5fb67c1539b-secret-thanos-querier-tls\") pod \"thanos-querier-59bc996f9b-9jtxb\" (UID: \"6cb1f54c-e94c-4d10-be59-e5fb67c1539b\") " pod="openshift-monitoring/thanos-querier-59bc996f9b-9jtxb" Jul 19 16:43:19.463146 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:19.463125 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-749tr\" (UniqueName: \"kubernetes.io/projected/6cb1f54c-e94c-4d10-be59-e5fb67c1539b-kube-api-access-749tr\") pod \"thanos-querier-59bc996f9b-9jtxb\" (UID: \"6cb1f54c-e94c-4d10-be59-e5fb67c1539b\") " pod="openshift-monitoring/thanos-querier-59bc996f9b-9jtxb" Jul 19 16:43:19.549411 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:19.549323 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-59bc996f9b-9jtxb" Jul 19 16:43:19.581031 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:19.580990 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4x8vz" event={"ID":"b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8","Type":"ContainerStarted","Data":"2f3f2c01064d34b67313ed212664605432e97a06b6fb50487f0c6acbc97c50b5"} Jul 19 16:43:19.596731 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:43:19.596451 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-thqxw" podUID="9c487ff0-684c-41bd-aca1-74636fae5571" Jul 19 16:43:19.916807 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:19.916767 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-59bc996f9b-9jtxb"] Jul 19 16:43:19.930300 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:43:19.930266 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cb1f54c_e94c_4d10_be59_e5fb67c1539b.slice/crio-5162c258579aafbd327bc7a18f9792d9249c852fde94d07110f0d64b23aa129e WatchSource:0}: Error finding container 5162c258579aafbd327bc7a18f9792d9249c852fde94d07110f0d64b23aa129e: Status 404 returned error can't find the container with id 5162c258579aafbd327bc7a18f9792d9249c852fde94d07110f0d64b23aa129e Jul 19 16:43:20.107950 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:20.107912 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-69c5f65c6-5jtsm" Jul 19 16:43:20.108156 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:20.108000 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-69c5f65c6-5jtsm" Jul 19 16:43:20.112908 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:20.112882 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-69c5f65c6-5jtsm" Jul 19 16:43:20.586118 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:20.586083 2583 generic.go:356] "Generic (PLEG): container finished" podID="b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8" containerID="2f3f2c01064d34b67313ed212664605432e97a06b6fb50487f0c6acbc97c50b5" exitCode=0 Jul 19 16:43:20.586570 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:20.586180 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4x8vz" event={"ID":"b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8","Type":"ContainerDied","Data":"2f3f2c01064d34b67313ed212664605432e97a06b6fb50487f0c6acbc97c50b5"} Jul 19 16:43:20.588408 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:20.588377 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-bnvf8" event={"ID":"dfeed6fd-e812-449b-ae0e-9746e3953fa3","Type":"ContainerStarted","Data":"b41ff2d979130c94d32a98fcac397f34e9a01a05d66183bb07d7bcdb9841021e"} Jul 19 16:43:20.588523 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:20.588433 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-bnvf8" event={"ID":"dfeed6fd-e812-449b-ae0e-9746e3953fa3","Type":"ContainerStarted","Data":"54232d8942e2920d6db54f0562e1672acc38554292211f8e0a675baa7e276411"} Jul 19 16:43:20.588523 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:20.588455 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-bnvf8" event={"ID":"dfeed6fd-e812-449b-ae0e-9746e3953fa3","Type":"ContainerStarted","Data":"e43d1638be98f243b26c3cf64a62afac4ef5aaca856913fce0a280c630eb0ef0"} Jul 19 16:43:20.589767 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:20.589740 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-59bc996f9b-9jtxb" event={"ID":"6cb1f54c-e94c-4d10-be59-e5fb67c1539b","Type":"ContainerStarted","Data":"5162c258579aafbd327bc7a18f9792d9249c852fde94d07110f0d64b23aa129e"} Jul 19 16:43:20.593743 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:20.593719 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-69c5f65c6-5jtsm" Jul 19 16:43:20.622762 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:20.622704 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56c5594674-bnvf8" podStartSLOduration=2.1410513780000002 podStartE2EDuration="3.622686018s" totalImagesPullingTime="1.48163464s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 16:43:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 16:43:20.621320017 +0000 UTC m=+295.607895493" watchObservedRunningTime="2026-07-19 16:43:20.622686018 +0000 UTC m=+295.609261503" Jul 19 16:43:21.600555 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:21.600519 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4x8vz" event={"ID":"b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8","Type":"ContainerStarted","Data":"3ee73fcebb64249078e6dff1b7e460ea06ff1dfcb12c952ce0fc85afda395d80"} Jul 19 16:43:21.600979 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:21.600580 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4x8vz" event={"ID":"b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8","Type":"ContainerStarted","Data":"2d344ff437655281a5d8991619aade633ed7d3d592ecc273d9d1ad9d6ecbefa1"} Jul 19 16:43:21.613824 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:21.613764 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-4x8vz" podStartSLOduration=3.472068028 podStartE2EDuration="4.613744122s" totalImagesPullingTime="1.074127107s" totalInitContainerRuntime="67.548987ms" isStatefulPod=true podCreationTimestamp="2026-07-19 16:43:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 16:43:21.611872613 +0000 UTC m=+296.598448098" watchObservedRunningTime="2026-07-19 16:43:21.613744122 +0000 UTC m=+296.600319652" Jul 19 16:43:22.601124 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:22.601088 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-59bc996f9b-9jtxb" event={"ID":"6cb1f54c-e94c-4d10-be59-e5fb67c1539b","Type":"ContainerStarted","Data":"1800adc9325664a97f72c0c65a04a16627c02975693f14ef67a345db39f3af78"} Jul 19 16:43:22.601476 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:22.601153 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-59bc996f9b-9jtxb" event={"ID":"6cb1f54c-e94c-4d10-be59-e5fb67c1539b","Type":"ContainerStarted","Data":"3cc200dc41ebc297659cd470e97cd9ffbcbdb3960d73440f5aea60935622c1f2"} Jul 19 16:43:22.601476 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:22.601181 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-59bc996f9b-9jtxb" event={"ID":"6cb1f54c-e94c-4d10-be59-e5fb67c1539b","Type":"ContainerStarted","Data":"3ed4c639e0e89ff366c5d8b84f2099977fac900e6e9ac2fafa222dec8ceb422c"} Jul 19 16:43:23.442787 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.442638 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 19 16:43:23.460586 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.460555 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.462361 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.462331 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 19 16:43:23.463375 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.463348 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 19 16:43:23.463375 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.463368 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 19 16:43:23.463584 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.463398 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 19 16:43:23.463584 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.463352 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 19 16:43:23.463584 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.463513 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 19 16:43:23.463584 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.463542 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 19 16:43:23.463584 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.463352 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7ek68k7g6neh9\"" Jul 19 16:43:23.463835 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.463749 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 19 16:43:23.463835 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.463769 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 19 16:43:23.463835 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.463827 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 19 16:43:23.464156 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.464093 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 19 16:43:23.464156 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.464052 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 19 16:43:23.464671 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.464477 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-b5l9d\"" Jul 19 16:43:23.464671 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.464478 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 19 16:43:23.464671 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.464531 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 19 16:43:23.464856 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.464719 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 19 16:43:23.466292 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.465950 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 19 16:43:23.597268 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.597229 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-config\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.597442 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.597275 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.597442 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.597366 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.597442 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.597406 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.597442 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.597439 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2r7z\" (UniqueName: \"kubernetes.io/projected/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-kube-api-access-b2r7z\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.597652 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.597516 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.597652 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.597555 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.597652 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.597581 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.597652 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.597605 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.597652 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.597631 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.597863 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.597664 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.597863 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.597729 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.597863 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.597752 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.597863 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.597780 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.597863 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.597837 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.598009 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.597872 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.598009 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.597896 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-config-out\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.598009 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.597936 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-web-config\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.598009 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.597952 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.598009 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.597976 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.698749 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.698722 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.699155 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.698754 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.699155 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.698770 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.699155 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.698787 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.699155 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.698812 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.699155 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.698838 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.699155 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.698862 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.699155 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.699027 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.699155 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.699089 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.699155 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.699126 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.699155 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.699151 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.699654 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.699184 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-config-out\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.699654 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.699242 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-web-config\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.699654 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.699265 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.699654 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.699291 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.699654 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.699333 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-config\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.699654 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.699358 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.699654 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.699399 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.699654 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.699428 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.699654 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.699454 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2r7z\" (UniqueName: \"kubernetes.io/projected/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-kube-api-access-b2r7z\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.700102 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.699790 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.700102 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.699838 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.700611 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.700581 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.702031 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.701143 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.702031 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.701702 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.702680 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.702654 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.702680 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.702667 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.703126 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.703052 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.703234 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.703108 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.703446 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.703423 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.703603 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.703579 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.703718 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.703693 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.704613 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.704589 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.704804 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.704779 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.705110 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.705086 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-config-out\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.705241 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.705096 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-config\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.708228 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.706288 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.708228 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.706644 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-web-config\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.708228 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.707587 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2r7z\" (UniqueName: \"kubernetes.io/projected/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-kube-api-access-b2r7z\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.708918 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.708902 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.780526 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.780495 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:23.971079 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:23.971033 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 19 16:43:23.973998 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:43:23.973974 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfd2cbe6_9b0f_4751_acfc_71dbba9ca989.slice/crio-aa1d9b9eeb2888da26f8919124193079cd0473f07928c283d160dc4bc4eb47d0 WatchSource:0}: Error finding container aa1d9b9eeb2888da26f8919124193079cd0473f07928c283d160dc4bc4eb47d0: Status 404 returned error can't find the container with id aa1d9b9eeb2888da26f8919124193079cd0473f07928c283d160dc4bc4eb47d0 Jul 19 16:43:24.564482 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:24.564451 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-znbhx" Jul 19 16:43:24.610834 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:24.610750 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989","Type":"ContainerStarted","Data":"aa1d9b9eeb2888da26f8919124193079cd0473f07928c283d160dc4bc4eb47d0"} Jul 19 16:43:24.614712 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:24.614673 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-59bc996f9b-9jtxb" event={"ID":"6cb1f54c-e94c-4d10-be59-e5fb67c1539b","Type":"ContainerStarted","Data":"7d4da6be827045e915383b8ecd8c3f90d362deee2670d75a31648d436c77aead"} Jul 19 16:43:24.614844 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:24.614760 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-59bc996f9b-9jtxb" event={"ID":"6cb1f54c-e94c-4d10-be59-e5fb67c1539b","Type":"ContainerStarted","Data":"4f95477a0b65b4c8b591dfa3c6c1364149163ce7b7f87b81257902c41c81151e"} Jul 19 16:43:24.614844 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:24.614800 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-59bc996f9b-9jtxb" event={"ID":"6cb1f54c-e94c-4d10-be59-e5fb67c1539b","Type":"ContainerStarted","Data":"340bff848676c81e61f806c5f1bc4f8c3f137013d7cc1a8dcb90c6a7b9aec513"} Jul 19 16:43:24.614958 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:24.614941 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-59bc996f9b-9jtxb" Jul 19 16:43:24.636258 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:24.636203 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-59bc996f9b-9jtxb" podStartSLOduration=2.107869659 podStartE2EDuration="5.63618027s" totalImagesPullingTime="3.528310611s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 16:43:19 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 16:43:24.635073075 +0000 UTC m=+299.621648553" watchObservedRunningTime="2026-07-19 16:43:24.63618027 +0000 UTC m=+299.622755755" Jul 19 16:43:25.467010 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:25.466986 2583 kubelet.go:1648] "Image garbage collection succeeded" Jul 19 16:43:25.619953 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:25.619808 2583 generic.go:356] "Generic (PLEG): container finished" podID="dfd2cbe6-9b0f-4751-acfc-71dbba9ca989" containerID="6c7c4400137af6e29fc20ba2fb5391c3980a8d132729168366dd2f3f41859a8a" exitCode=0 Jul 19 16:43:25.619953 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:25.619902 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989","Type":"ContainerDied","Data":"6c7c4400137af6e29fc20ba2fb5391c3980a8d132729168366dd2f3f41859a8a"} Jul 19 16:43:30.626359 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:30.626320 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-59bc996f9b-9jtxb" Jul 19 16:43:30.649195 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:30.649153 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989","Type":"ContainerStarted","Data":"2c14a34aa62649d407b0b5e8bfa6e7db398bcf4924e27ab63614c9caf462e2de"} Jul 19 16:43:30.649376 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:30.649254 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989","Type":"ContainerStarted","Data":"eee18131bf1ac347964fca93207f238f9cbcb350e36f5949b8150c83f8fc502a"} Jul 19 16:43:30.649376 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:30.649308 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989","Type":"ContainerStarted","Data":"de33cb394ac35ddae3b49af5adbbc54042b36cea95e90f80384593f78760c0cd"} Jul 19 16:43:30.649376 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:30.649346 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989","Type":"ContainerStarted","Data":"9085817976b3674ad0d8a53c31787102a9bc306864ce29bcd2b5c53ca9c1abee"} Jul 19 16:43:30.649529 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:30.649396 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989","Type":"ContainerStarted","Data":"40fe6c40615e212265c60ba92bb2c1b4be67450e23241ab1188c09d426246789"} Jul 19 16:43:30.649529 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:30.649444 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989","Type":"ContainerStarted","Data":"6e5c17437f4c971a597369d9d0ada4949a9910e2a2925bce3614d3a3375bd246"} Jul 19 16:43:30.678590 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:30.678534 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.407104605 podStartE2EDuration="7.678516438s" totalImagesPullingTime="5.252280019s" totalInitContainerRuntime="19.131814ms" isStatefulPod=false podCreationTimestamp="2026-07-19 16:43:23 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 16:43:30.677239573 +0000 UTC m=+305.663815083" watchObservedRunningTime="2026-07-19 16:43:30.678516438 +0000 UTC m=+305.665091979" Jul 19 16:43:33.595614 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:33.595578 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-thqxw" Jul 19 16:43:33.781191 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:33.781155 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:43:33.954430 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:33.954342 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-69c5f65c6-5jtsm"] Jul 19 16:43:42.693787 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:42.693755 2583 generic.go:356] "Generic (PLEG): container finished" podID="7b5796d8-6465-4c8b-835e-7bf00c972c9e" containerID="04389da1b0b89fc4279585614b5883c5ffb35e75d49fae213683c3896fe6392e" exitCode=0 Jul 19 16:43:42.694201 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:42.693826 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-d2qfz" event={"ID":"7b5796d8-6465-4c8b-835e-7bf00c972c9e","Type":"ContainerDied","Data":"04389da1b0b89fc4279585614b5883c5ffb35e75d49fae213683c3896fe6392e"} Jul 19 16:43:42.694265 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:42.694239 2583 scope.go:117] "RemoveContainer" containerID="04389da1b0b89fc4279585614b5883c5ffb35e75d49fae213683c3896fe6392e" Jul 19 16:43:43.699557 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:43.699520 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-d2qfz" event={"ID":"7b5796d8-6465-4c8b-835e-7bf00c972c9e","Type":"ContainerStarted","Data":"a94cb4f04f090478a1137fd332e22d4535ba1dc5d18e593358c216edc9295c9b"} Jul 19 16:43:58.974340 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:58.974280 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-69c5f65c6-5jtsm" podUID="f37a3556-1276-4995-9739-32f6fece238d" containerName="console" containerID="cri-o://8df436a190ec81d626af1fb2b76429878efbe21d368caadb54eb3468ae687d97" gracePeriod=15 Jul 19 16:43:59.220187 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:59.220163 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69c5f65c6-5jtsm" Jul 19 16:43:59.293026 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:59.292944 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9h2xg\" (UniqueName: \"kubernetes.io/projected/f37a3556-1276-4995-9739-32f6fece238d-kube-api-access-9h2xg\") pod \"f37a3556-1276-4995-9739-32f6fece238d\" (UID: \"f37a3556-1276-4995-9739-32f6fece238d\") " Jul 19 16:43:59.293026 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:59.292984 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f37a3556-1276-4995-9739-32f6fece238d-trusted-ca-bundle\") pod \"f37a3556-1276-4995-9739-32f6fece238d\" (UID: \"f37a3556-1276-4995-9739-32f6fece238d\") " Jul 19 16:43:59.293026 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:59.293003 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f37a3556-1276-4995-9739-32f6fece238d-console-oauth-config\") pod \"f37a3556-1276-4995-9739-32f6fece238d\" (UID: \"f37a3556-1276-4995-9739-32f6fece238d\") " Jul 19 16:43:59.293026 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:59.293028 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f37a3556-1276-4995-9739-32f6fece238d-console-serving-cert\") pod \"f37a3556-1276-4995-9739-32f6fece238d\" (UID: \"f37a3556-1276-4995-9739-32f6fece238d\") " Jul 19 16:43:59.293353 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:59.293113 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f37a3556-1276-4995-9739-32f6fece238d-oauth-serving-cert\") pod \"f37a3556-1276-4995-9739-32f6fece238d\" (UID: \"f37a3556-1276-4995-9739-32f6fece238d\") " Jul 19 16:43:59.293353 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:59.293189 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f37a3556-1276-4995-9739-32f6fece238d-service-ca\") pod \"f37a3556-1276-4995-9739-32f6fece238d\" (UID: \"f37a3556-1276-4995-9739-32f6fece238d\") " Jul 19 16:43:59.293353 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:59.293215 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f37a3556-1276-4995-9739-32f6fece238d-console-config\") pod \"f37a3556-1276-4995-9739-32f6fece238d\" (UID: \"f37a3556-1276-4995-9739-32f6fece238d\") " Jul 19 16:43:59.293677 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:59.293566 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f37a3556-1276-4995-9739-32f6fece238d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f37a3556-1276-4995-9739-32f6fece238d" (UID: "f37a3556-1276-4995-9739-32f6fece238d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 19 16:43:59.293781 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:59.293573 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f37a3556-1276-4995-9739-32f6fece238d-service-ca" (OuterVolumeSpecName: "service-ca") pod "f37a3556-1276-4995-9739-32f6fece238d" (UID: "f37a3556-1276-4995-9739-32f6fece238d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 19 16:43:59.293839 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:59.293795 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f37a3556-1276-4995-9739-32f6fece238d-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "f37a3556-1276-4995-9739-32f6fece238d" (UID: "f37a3556-1276-4995-9739-32f6fece238d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 19 16:43:59.293982 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:59.293958 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f37a3556-1276-4995-9739-32f6fece238d-console-config" (OuterVolumeSpecName: "console-config") pod "f37a3556-1276-4995-9739-32f6fece238d" (UID: "f37a3556-1276-4995-9739-32f6fece238d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 19 16:43:59.295312 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:59.295287 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f37a3556-1276-4995-9739-32f6fece238d-kube-api-access-9h2xg" (OuterVolumeSpecName: "kube-api-access-9h2xg") pod "f37a3556-1276-4995-9739-32f6fece238d" (UID: "f37a3556-1276-4995-9739-32f6fece238d"). InnerVolumeSpecName "kube-api-access-9h2xg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 19 16:43:59.295415 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:59.295336 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f37a3556-1276-4995-9739-32f6fece238d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f37a3556-1276-4995-9739-32f6fece238d" (UID: "f37a3556-1276-4995-9739-32f6fece238d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 19 16:43:59.295415 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:59.295361 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f37a3556-1276-4995-9739-32f6fece238d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f37a3556-1276-4995-9739-32f6fece238d" (UID: "f37a3556-1276-4995-9739-32f6fece238d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 19 16:43:59.394115 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:59.394044 2583 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f37a3556-1276-4995-9739-32f6fece238d-trusted-ca-bundle\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 16:43:59.394115 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:59.394106 2583 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f37a3556-1276-4995-9739-32f6fece238d-console-oauth-config\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 16:43:59.394115 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:59.394116 2583 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f37a3556-1276-4995-9739-32f6fece238d-console-serving-cert\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 16:43:59.394115 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:59.394125 2583 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f37a3556-1276-4995-9739-32f6fece238d-oauth-serving-cert\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 16:43:59.394115 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:59.394134 2583 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f37a3556-1276-4995-9739-32f6fece238d-service-ca\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 16:43:59.394393 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:59.394142 2583 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f37a3556-1276-4995-9739-32f6fece238d-console-config\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 16:43:59.394393 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:59.394150 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9h2xg\" (UniqueName: \"kubernetes.io/projected/f37a3556-1276-4995-9739-32f6fece238d-kube-api-access-9h2xg\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 16:43:59.758530 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:59.758498 2583 generic.go:356] "Generic (PLEG): container finished" podID="f37a3556-1276-4995-9739-32f6fece238d" containerID="8df436a190ec81d626af1fb2b76429878efbe21d368caadb54eb3468ae687d97" exitCode=0 Jul 19 16:43:59.758692 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:59.758551 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69c5f65c6-5jtsm" event={"ID":"f37a3556-1276-4995-9739-32f6fece238d","Type":"ContainerDied","Data":"8df436a190ec81d626af1fb2b76429878efbe21d368caadb54eb3468ae687d97"} Jul 19 16:43:59.758692 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:59.758555 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69c5f65c6-5jtsm" Jul 19 16:43:59.758692 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:59.758588 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69c5f65c6-5jtsm" event={"ID":"f37a3556-1276-4995-9739-32f6fece238d","Type":"ContainerDied","Data":"78aafccce12616b4a32777214b958556075c236d9a845f60b107daeb3baa6191"} Jul 19 16:43:59.758692 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:59.758607 2583 scope.go:117] "RemoveContainer" containerID="8df436a190ec81d626af1fb2b76429878efbe21d368caadb54eb3468ae687d97" Jul 19 16:43:59.768670 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:59.768643 2583 scope.go:117] "RemoveContainer" containerID="8df436a190ec81d626af1fb2b76429878efbe21d368caadb54eb3468ae687d97" Jul 19 16:43:59.769002 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:43:59.768981 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8df436a190ec81d626af1fb2b76429878efbe21d368caadb54eb3468ae687d97\": container with ID starting with 8df436a190ec81d626af1fb2b76429878efbe21d368caadb54eb3468ae687d97 not found: ID does not exist" containerID="8df436a190ec81d626af1fb2b76429878efbe21d368caadb54eb3468ae687d97" Jul 19 16:43:59.769107 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:59.769010 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8df436a190ec81d626af1fb2b76429878efbe21d368caadb54eb3468ae687d97"} err="failed to get container status \"8df436a190ec81d626af1fb2b76429878efbe21d368caadb54eb3468ae687d97\": rpc error: code = NotFound desc = could not find container \"8df436a190ec81d626af1fb2b76429878efbe21d368caadb54eb3468ae687d97\": container with ID starting with 8df436a190ec81d626af1fb2b76429878efbe21d368caadb54eb3468ae687d97 not found: ID does not exist" Jul 19 16:43:59.773994 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:59.773972 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-69c5f65c6-5jtsm"] Jul 19 16:43:59.777726 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:43:59.777706 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-69c5f65c6-5jtsm"] Jul 19 16:44:01.601581 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:01.601551 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f37a3556-1276-4995-9739-32f6fece238d" path="/var/lib/kubelet/pods/f37a3556-1276-4995-9739-32f6fece238d/volumes" Jul 19 16:44:23.780773 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:23.780682 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:23.796284 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:23.796254 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:23.864918 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:23.864893 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:39.337155 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:39.337112 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9c487ff0-684c-41bd-aca1-74636fae5571-metrics-certs\") pod \"network-metrics-daemon-thqxw\" (UID: \"9c487ff0-684c-41bd-aca1-74636fae5571\") " pod="openshift-multus/network-metrics-daemon-thqxw" Jul 19 16:44:39.339511 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:39.339487 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9c487ff0-684c-41bd-aca1-74636fae5571-metrics-certs\") pod \"network-metrics-daemon-thqxw\" (UID: \"9c487ff0-684c-41bd-aca1-74636fae5571\") " pod="openshift-multus/network-metrics-daemon-thqxw" Jul 19 16:44:39.599182 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:39.599098 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pkhnw\"" Jul 19 16:44:39.606767 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:39.606741 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-thqxw" Jul 19 16:44:39.735996 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:39.735954 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-thqxw"] Jul 19 16:44:39.738969 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:44:39.738936 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c487ff0_684c_41bd_aca1_74636fae5571.slice/crio-b28f682f748f2c0b2588e5ba6871f745238db9cb57eb676d290cfaa39d9f329e WatchSource:0}: Error finding container b28f682f748f2c0b2588e5ba6871f745238db9cb57eb676d290cfaa39d9f329e: Status 404 returned error can't find the container with id b28f682f748f2c0b2588e5ba6871f745238db9cb57eb676d290cfaa39d9f329e Jul 19 16:44:39.740826 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:39.740811 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 19 16:44:39.907444 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:39.907362 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-thqxw" event={"ID":"9c487ff0-684c-41bd-aca1-74636fae5571","Type":"ContainerStarted","Data":"b28f682f748f2c0b2588e5ba6871f745238db9cb57eb676d290cfaa39d9f329e"} Jul 19 16:44:40.913187 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:40.913143 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-thqxw" event={"ID":"9c487ff0-684c-41bd-aca1-74636fae5571","Type":"ContainerStarted","Data":"619ddd251ea43f74a81a1da5effd8c8070e755506c7cd047dcab64f57d11aed8"} Jul 19 16:44:40.913587 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:40.913201 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-thqxw" event={"ID":"9c487ff0-684c-41bd-aca1-74636fae5571","Type":"ContainerStarted","Data":"9aee46e2921b0eeae48c070642c2a4b58f9f3aa244e873f08afbd6b38b3f3c65"} Jul 19 16:44:40.929860 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:40.929802 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-thqxw" podStartSLOduration=374.097307941 podStartE2EDuration="6m14.929782764s" totalImagesPullingTime="832.474823ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 16:38:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 16:44:40.927596971 +0000 UTC m=+375.914172493" watchObservedRunningTime="2026-07-19 16:44:40.929782764 +0000 UTC m=+375.916358251" Jul 19 16:44:41.810402 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:41.810179 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 19 16:44:41.810602 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:41.810550 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="dfd2cbe6-9b0f-4751-acfc-71dbba9ca989" containerName="prometheus" containerID="cri-o://6e5c17437f4c971a597369d9d0ada4949a9910e2a2925bce3614d3a3375bd246" gracePeriod=600 Jul 19 16:44:41.810673 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:41.810593 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="dfd2cbe6-9b0f-4751-acfc-71dbba9ca989" containerName="kube-rbac-proxy-web" containerID="cri-o://de33cb394ac35ddae3b49af5adbbc54042b36cea95e90f80384593f78760c0cd" gracePeriod=600 Jul 19 16:44:41.810673 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:41.810604 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="dfd2cbe6-9b0f-4751-acfc-71dbba9ca989" containerName="thanos-sidecar" containerID="cri-o://9085817976b3674ad0d8a53c31787102a9bc306864ce29bcd2b5c53ca9c1abee" gracePeriod=600 Jul 19 16:44:41.810673 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:41.810635 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="dfd2cbe6-9b0f-4751-acfc-71dbba9ca989" containerName="kube-rbac-proxy" containerID="cri-o://eee18131bf1ac347964fca93207f238f9cbcb350e36f5949b8150c83f8fc502a" gracePeriod=600 Jul 19 16:44:41.810819 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:41.810554 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="dfd2cbe6-9b0f-4751-acfc-71dbba9ca989" containerName="kube-rbac-proxy-thanos" containerID="cri-o://2c14a34aa62649d407b0b5e8bfa6e7db398bcf4924e27ab63614c9caf462e2de" gracePeriod=600 Jul 19 16:44:41.810819 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:41.810712 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="dfd2cbe6-9b0f-4751-acfc-71dbba9ca989" containerName="config-reloader" containerID="cri-o://40fe6c40615e212265c60ba92bb2c1b4be67450e23241ab1188c09d426246789" gracePeriod=600 Jul 19 16:44:42.072234 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.072202 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:42.159515 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.159485 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-config\") pod \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " Jul 19 16:44:42.159515 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.159522 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-configmap-metrics-client-ca\") pod \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " Jul 19 16:44:42.159767 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.159540 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-secret-prometheus-k8s-tls\") pod \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " Jul 19 16:44:42.159767 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.159585 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-secret-kube-rbac-proxy\") pod \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " Jul 19 16:44:42.159767 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.159640 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-configmap-kubelet-serving-ca-bundle\") pod \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " Jul 19 16:44:42.159767 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.159673 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-prometheus-k8s-rulefiles-2\") pod \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " Jul 19 16:44:42.159767 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.159697 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-configmap-serving-certs-ca-bundle\") pod \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " Jul 19 16:44:42.159767 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.159739 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-prometheus-k8s-db\") pod \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " Jul 19 16:44:42.159767 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.159769 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-prometheus-trusted-ca-bundle\") pod \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " Jul 19 16:44:42.160131 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.159797 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-config-out\") pod \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " Jul 19 16:44:42.160131 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.159821 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-web-config\") pod \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " Jul 19 16:44:42.160131 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.159850 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "dfd2cbe6-9b0f-4751-acfc-71dbba9ca989" (UID: "dfd2cbe6-9b0f-4751-acfc-71dbba9ca989"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 19 16:44:42.160131 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.159860 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-prometheus-k8s-rulefiles-1\") pod \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " Jul 19 16:44:42.160131 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.159950 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b2r7z\" (UniqueName: \"kubernetes.io/projected/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-kube-api-access-b2r7z\") pod \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " Jul 19 16:44:42.160131 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.159988 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-prometheus-k8s-rulefiles-0\") pod \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " Jul 19 16:44:42.160131 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.160019 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-secret-grpc-tls\") pod \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " Jul 19 16:44:42.160131 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.160045 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-thanos-prometheus-http-client-file\") pod \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " Jul 19 16:44:42.160131 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.160087 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "dfd2cbe6-9b0f-4751-acfc-71dbba9ca989" (UID: "dfd2cbe6-9b0f-4751-acfc-71dbba9ca989"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 19 16:44:42.160131 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.160103 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " Jul 19 16:44:42.160624 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.160158 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " Jul 19 16:44:42.160624 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.160184 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-tls-assets\") pod \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " Jul 19 16:44:42.160624 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.160195 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "dfd2cbe6-9b0f-4751-acfc-71dbba9ca989" (UID: "dfd2cbe6-9b0f-4751-acfc-71dbba9ca989"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 19 16:44:42.160624 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.160201 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-secret-metrics-client-certs\") pod \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\" (UID: \"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989\") " Jul 19 16:44:42.160624 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.160485 2583 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 16:44:42.160624 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.160504 2583 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-configmap-metrics-client-ca\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 16:44:42.160624 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.160588 2583 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 16:44:42.160985 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.160884 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "dfd2cbe6-9b0f-4751-acfc-71dbba9ca989" (UID: "dfd2cbe6-9b0f-4751-acfc-71dbba9ca989"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 19 16:44:42.161636 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.161284 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "dfd2cbe6-9b0f-4751-acfc-71dbba9ca989" (UID: "dfd2cbe6-9b0f-4751-acfc-71dbba9ca989"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 19 16:44:42.161636 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.161444 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "dfd2cbe6-9b0f-4751-acfc-71dbba9ca989" (UID: "dfd2cbe6-9b0f-4751-acfc-71dbba9ca989"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 19 16:44:42.162777 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.162747 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "dfd2cbe6-9b0f-4751-acfc-71dbba9ca989" (UID: "dfd2cbe6-9b0f-4751-acfc-71dbba9ca989"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 16:44:42.162892 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.162850 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "dfd2cbe6-9b0f-4751-acfc-71dbba9ca989" (UID: "dfd2cbe6-9b0f-4751-acfc-71dbba9ca989"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 19 16:44:42.162952 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.162912 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-config-out" (OuterVolumeSpecName: "config-out") pod "dfd2cbe6-9b0f-4751-acfc-71dbba9ca989" (UID: "dfd2cbe6-9b0f-4751-acfc-71dbba9ca989"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 16:44:42.163095 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.163074 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-config" (OuterVolumeSpecName: "config") pod "dfd2cbe6-9b0f-4751-acfc-71dbba9ca989" (UID: "dfd2cbe6-9b0f-4751-acfc-71dbba9ca989"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 19 16:44:42.163166 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.163153 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "dfd2cbe6-9b0f-4751-acfc-71dbba9ca989" (UID: "dfd2cbe6-9b0f-4751-acfc-71dbba9ca989"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 19 16:44:42.163478 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.163458 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "dfd2cbe6-9b0f-4751-acfc-71dbba9ca989" (UID: "dfd2cbe6-9b0f-4751-acfc-71dbba9ca989"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 19 16:44:42.164596 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.164545 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-kube-api-access-b2r7z" (OuterVolumeSpecName: "kube-api-access-b2r7z") pod "dfd2cbe6-9b0f-4751-acfc-71dbba9ca989" (UID: "dfd2cbe6-9b0f-4751-acfc-71dbba9ca989"). InnerVolumeSpecName "kube-api-access-b2r7z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 19 16:44:42.164596 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.164568 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "dfd2cbe6-9b0f-4751-acfc-71dbba9ca989" (UID: "dfd2cbe6-9b0f-4751-acfc-71dbba9ca989"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 19 16:44:42.165001 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.164974 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "dfd2cbe6-9b0f-4751-acfc-71dbba9ca989" (UID: "dfd2cbe6-9b0f-4751-acfc-71dbba9ca989"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 19 16:44:42.165184 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.165156 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "dfd2cbe6-9b0f-4751-acfc-71dbba9ca989" (UID: "dfd2cbe6-9b0f-4751-acfc-71dbba9ca989"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 19 16:44:42.165309 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.165272 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "dfd2cbe6-9b0f-4751-acfc-71dbba9ca989" (UID: "dfd2cbe6-9b0f-4751-acfc-71dbba9ca989"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 19 16:44:42.165484 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.165457 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "dfd2cbe6-9b0f-4751-acfc-71dbba9ca989" (UID: "dfd2cbe6-9b0f-4751-acfc-71dbba9ca989"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 19 16:44:42.166373 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.166353 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "dfd2cbe6-9b0f-4751-acfc-71dbba9ca989" (UID: "dfd2cbe6-9b0f-4751-acfc-71dbba9ca989"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 19 16:44:42.176369 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.176345 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-web-config" (OuterVolumeSpecName: "web-config") pod "dfd2cbe6-9b0f-4751-acfc-71dbba9ca989" (UID: "dfd2cbe6-9b0f-4751-acfc-71dbba9ca989"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 19 16:44:42.261465 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.261431 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b2r7z\" (UniqueName: \"kubernetes.io/projected/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-kube-api-access-b2r7z\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 16:44:42.261465 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.261461 2583 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 16:44:42.261465 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.261471 2583 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-secret-grpc-tls\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 16:44:42.261672 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.261482 2583 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-thanos-prometheus-http-client-file\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 16:44:42.261672 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.261492 2583 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 16:44:42.261672 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.261502 2583 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 16:44:42.261672 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.261510 2583 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-tls-assets\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 16:44:42.261672 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.261518 2583 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-secret-metrics-client-certs\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 16:44:42.261672 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.261528 2583 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-config\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 16:44:42.261672 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.261536 2583 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-secret-prometheus-k8s-tls\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 16:44:42.261672 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.261544 2583 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-secret-kube-rbac-proxy\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 16:44:42.261672 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.261553 2583 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 16:44:42.261672 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.261561 2583 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 16:44:42.261672 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.261570 2583 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-prometheus-k8s-db\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 16:44:42.261672 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.261579 2583 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-prometheus-trusted-ca-bundle\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 16:44:42.261672 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.261587 2583 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-config-out\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 16:44:42.261672 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.261595 2583 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989-web-config\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 16:44:42.925297 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.925262 2583 generic.go:356] "Generic (PLEG): container finished" podID="dfd2cbe6-9b0f-4751-acfc-71dbba9ca989" containerID="2c14a34aa62649d407b0b5e8bfa6e7db398bcf4924e27ab63614c9caf462e2de" exitCode=0 Jul 19 16:44:42.925297 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.925290 2583 generic.go:356] "Generic (PLEG): container finished" podID="dfd2cbe6-9b0f-4751-acfc-71dbba9ca989" containerID="eee18131bf1ac347964fca93207f238f9cbcb350e36f5949b8150c83f8fc502a" exitCode=0 Jul 19 16:44:42.925297 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.925299 2583 generic.go:356] "Generic (PLEG): container finished" podID="dfd2cbe6-9b0f-4751-acfc-71dbba9ca989" containerID="de33cb394ac35ddae3b49af5adbbc54042b36cea95e90f80384593f78760c0cd" exitCode=0 Jul 19 16:44:42.925297 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.925308 2583 generic.go:356] "Generic (PLEG): container finished" podID="dfd2cbe6-9b0f-4751-acfc-71dbba9ca989" containerID="9085817976b3674ad0d8a53c31787102a9bc306864ce29bcd2b5c53ca9c1abee" exitCode=0 Jul 19 16:44:42.925618 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.925316 2583 generic.go:356] "Generic (PLEG): container finished" podID="dfd2cbe6-9b0f-4751-acfc-71dbba9ca989" containerID="40fe6c40615e212265c60ba92bb2c1b4be67450e23241ab1188c09d426246789" exitCode=0 Jul 19 16:44:42.925618 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.925324 2583 generic.go:356] "Generic (PLEG): container finished" podID="dfd2cbe6-9b0f-4751-acfc-71dbba9ca989" containerID="6e5c17437f4c971a597369d9d0ada4949a9910e2a2925bce3614d3a3375bd246" exitCode=0 Jul 19 16:44:42.925618 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.925337 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:42.925618 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.925350 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989","Type":"ContainerDied","Data":"2c14a34aa62649d407b0b5e8bfa6e7db398bcf4924e27ab63614c9caf462e2de"} Jul 19 16:44:42.925618 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.925423 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989","Type":"ContainerDied","Data":"eee18131bf1ac347964fca93207f238f9cbcb350e36f5949b8150c83f8fc502a"} Jul 19 16:44:42.925618 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.925452 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989","Type":"ContainerDied","Data":"de33cb394ac35ddae3b49af5adbbc54042b36cea95e90f80384593f78760c0cd"} Jul 19 16:44:42.925618 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.925477 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989","Type":"ContainerDied","Data":"9085817976b3674ad0d8a53c31787102a9bc306864ce29bcd2b5c53ca9c1abee"} Jul 19 16:44:42.925618 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.925494 2583 scope.go:117] "RemoveContainer" containerID="2c14a34aa62649d407b0b5e8bfa6e7db398bcf4924e27ab63614c9caf462e2de" Jul 19 16:44:42.925618 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.925501 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989","Type":"ContainerDied","Data":"40fe6c40615e212265c60ba92bb2c1b4be67450e23241ab1188c09d426246789"} Jul 19 16:44:42.925618 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.925528 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989","Type":"ContainerDied","Data":"6e5c17437f4c971a597369d9d0ada4949a9910e2a2925bce3614d3a3375bd246"} Jul 19 16:44:42.925618 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.925554 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dfd2cbe6-9b0f-4751-acfc-71dbba9ca989","Type":"ContainerDied","Data":"aa1d9b9eeb2888da26f8919124193079cd0473f07928c283d160dc4bc4eb47d0"} Jul 19 16:44:42.933632 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.933611 2583 scope.go:117] "RemoveContainer" containerID="eee18131bf1ac347964fca93207f238f9cbcb350e36f5949b8150c83f8fc502a" Jul 19 16:44:42.941009 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.940986 2583 scope.go:117] "RemoveContainer" containerID="de33cb394ac35ddae3b49af5adbbc54042b36cea95e90f80384593f78760c0cd" Jul 19 16:44:42.948747 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.948303 2583 scope.go:117] "RemoveContainer" containerID="9085817976b3674ad0d8a53c31787102a9bc306864ce29bcd2b5c53ca9c1abee" Jul 19 16:44:42.949160 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.949133 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 19 16:44:42.955736 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.955703 2583 scope.go:117] "RemoveContainer" containerID="40fe6c40615e212265c60ba92bb2c1b4be67450e23241ab1188c09d426246789" Jul 19 16:44:42.957326 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.957301 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 19 16:44:42.962554 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.962537 2583 scope.go:117] "RemoveContainer" containerID="6e5c17437f4c971a597369d9d0ada4949a9910e2a2925bce3614d3a3375bd246" Jul 19 16:44:42.969693 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.969674 2583 scope.go:117] "RemoveContainer" containerID="6c7c4400137af6e29fc20ba2fb5391c3980a8d132729168366dd2f3f41859a8a" Jul 19 16:44:42.976553 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.976536 2583 scope.go:117] "RemoveContainer" containerID="2c14a34aa62649d407b0b5e8bfa6e7db398bcf4924e27ab63614c9caf462e2de" Jul 19 16:44:42.976819 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:44:42.976802 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c14a34aa62649d407b0b5e8bfa6e7db398bcf4924e27ab63614c9caf462e2de\": container with ID starting with 2c14a34aa62649d407b0b5e8bfa6e7db398bcf4924e27ab63614c9caf462e2de not found: ID does not exist" containerID="2c14a34aa62649d407b0b5e8bfa6e7db398bcf4924e27ab63614c9caf462e2de" Jul 19 16:44:42.976869 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.976827 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c14a34aa62649d407b0b5e8bfa6e7db398bcf4924e27ab63614c9caf462e2de"} err="failed to get container status \"2c14a34aa62649d407b0b5e8bfa6e7db398bcf4924e27ab63614c9caf462e2de\": rpc error: code = NotFound desc = could not find container \"2c14a34aa62649d407b0b5e8bfa6e7db398bcf4924e27ab63614c9caf462e2de\": container with ID starting with 2c14a34aa62649d407b0b5e8bfa6e7db398bcf4924e27ab63614c9caf462e2de not found: ID does not exist" Jul 19 16:44:42.976869 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.976845 2583 scope.go:117] "RemoveContainer" containerID="eee18131bf1ac347964fca93207f238f9cbcb350e36f5949b8150c83f8fc502a" Jul 19 16:44:42.977110 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:44:42.977091 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eee18131bf1ac347964fca93207f238f9cbcb350e36f5949b8150c83f8fc502a\": container with ID starting with eee18131bf1ac347964fca93207f238f9cbcb350e36f5949b8150c83f8fc502a not found: ID does not exist" containerID="eee18131bf1ac347964fca93207f238f9cbcb350e36f5949b8150c83f8fc502a" Jul 19 16:44:42.977179 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.977122 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eee18131bf1ac347964fca93207f238f9cbcb350e36f5949b8150c83f8fc502a"} err="failed to get container status \"eee18131bf1ac347964fca93207f238f9cbcb350e36f5949b8150c83f8fc502a\": rpc error: code = NotFound desc = could not find container \"eee18131bf1ac347964fca93207f238f9cbcb350e36f5949b8150c83f8fc502a\": container with ID starting with eee18131bf1ac347964fca93207f238f9cbcb350e36f5949b8150c83f8fc502a not found: ID does not exist" Jul 19 16:44:42.977179 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.977144 2583 scope.go:117] "RemoveContainer" containerID="de33cb394ac35ddae3b49af5adbbc54042b36cea95e90f80384593f78760c0cd" Jul 19 16:44:42.977398 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:44:42.977378 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de33cb394ac35ddae3b49af5adbbc54042b36cea95e90f80384593f78760c0cd\": container with ID starting with de33cb394ac35ddae3b49af5adbbc54042b36cea95e90f80384593f78760c0cd not found: ID does not exist" containerID="de33cb394ac35ddae3b49af5adbbc54042b36cea95e90f80384593f78760c0cd" Jul 19 16:44:42.977436 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.977402 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de33cb394ac35ddae3b49af5adbbc54042b36cea95e90f80384593f78760c0cd"} err="failed to get container status \"de33cb394ac35ddae3b49af5adbbc54042b36cea95e90f80384593f78760c0cd\": rpc error: code = NotFound desc = could not find container \"de33cb394ac35ddae3b49af5adbbc54042b36cea95e90f80384593f78760c0cd\": container with ID starting with de33cb394ac35ddae3b49af5adbbc54042b36cea95e90f80384593f78760c0cd not found: ID does not exist" Jul 19 16:44:42.977436 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.977416 2583 scope.go:117] "RemoveContainer" containerID="9085817976b3674ad0d8a53c31787102a9bc306864ce29bcd2b5c53ca9c1abee" Jul 19 16:44:42.977647 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:44:42.977608 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9085817976b3674ad0d8a53c31787102a9bc306864ce29bcd2b5c53ca9c1abee\": container with ID starting with 9085817976b3674ad0d8a53c31787102a9bc306864ce29bcd2b5c53ca9c1abee not found: ID does not exist" containerID="9085817976b3674ad0d8a53c31787102a9bc306864ce29bcd2b5c53ca9c1abee" Jul 19 16:44:42.977685 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.977651 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9085817976b3674ad0d8a53c31787102a9bc306864ce29bcd2b5c53ca9c1abee"} err="failed to get container status \"9085817976b3674ad0d8a53c31787102a9bc306864ce29bcd2b5c53ca9c1abee\": rpc error: code = NotFound desc = could not find container \"9085817976b3674ad0d8a53c31787102a9bc306864ce29bcd2b5c53ca9c1abee\": container with ID starting with 9085817976b3674ad0d8a53c31787102a9bc306864ce29bcd2b5c53ca9c1abee not found: ID does not exist" Jul 19 16:44:42.977685 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.977665 2583 scope.go:117] "RemoveContainer" containerID="40fe6c40615e212265c60ba92bb2c1b4be67450e23241ab1188c09d426246789" Jul 19 16:44:42.977853 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:44:42.977839 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40fe6c40615e212265c60ba92bb2c1b4be67450e23241ab1188c09d426246789\": container with ID starting with 40fe6c40615e212265c60ba92bb2c1b4be67450e23241ab1188c09d426246789 not found: ID does not exist" containerID="40fe6c40615e212265c60ba92bb2c1b4be67450e23241ab1188c09d426246789" Jul 19 16:44:42.977891 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.977855 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40fe6c40615e212265c60ba92bb2c1b4be67450e23241ab1188c09d426246789"} err="failed to get container status \"40fe6c40615e212265c60ba92bb2c1b4be67450e23241ab1188c09d426246789\": rpc error: code = NotFound desc = could not find container \"40fe6c40615e212265c60ba92bb2c1b4be67450e23241ab1188c09d426246789\": container with ID starting with 40fe6c40615e212265c60ba92bb2c1b4be67450e23241ab1188c09d426246789 not found: ID does not exist" Jul 19 16:44:42.977891 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.977870 2583 scope.go:117] "RemoveContainer" containerID="6e5c17437f4c971a597369d9d0ada4949a9910e2a2925bce3614d3a3375bd246" Jul 19 16:44:42.978037 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:44:42.978023 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e5c17437f4c971a597369d9d0ada4949a9910e2a2925bce3614d3a3375bd246\": container with ID starting with 6e5c17437f4c971a597369d9d0ada4949a9910e2a2925bce3614d3a3375bd246 not found: ID does not exist" containerID="6e5c17437f4c971a597369d9d0ada4949a9910e2a2925bce3614d3a3375bd246" Jul 19 16:44:42.978105 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.978042 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e5c17437f4c971a597369d9d0ada4949a9910e2a2925bce3614d3a3375bd246"} err="failed to get container status \"6e5c17437f4c971a597369d9d0ada4949a9910e2a2925bce3614d3a3375bd246\": rpc error: code = NotFound desc = could not find container \"6e5c17437f4c971a597369d9d0ada4949a9910e2a2925bce3614d3a3375bd246\": container with ID starting with 6e5c17437f4c971a597369d9d0ada4949a9910e2a2925bce3614d3a3375bd246 not found: ID does not exist" Jul 19 16:44:42.978105 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.978083 2583 scope.go:117] "RemoveContainer" containerID="6c7c4400137af6e29fc20ba2fb5391c3980a8d132729168366dd2f3f41859a8a" Jul 19 16:44:42.978291 ip-10-0-134-203 kubenswrapper[2583]: E0719 16:44:42.978272 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c7c4400137af6e29fc20ba2fb5391c3980a8d132729168366dd2f3f41859a8a\": container with ID starting with 6c7c4400137af6e29fc20ba2fb5391c3980a8d132729168366dd2f3f41859a8a not found: ID does not exist" containerID="6c7c4400137af6e29fc20ba2fb5391c3980a8d132729168366dd2f3f41859a8a" Jul 19 16:44:42.978335 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.978298 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c7c4400137af6e29fc20ba2fb5391c3980a8d132729168366dd2f3f41859a8a"} err="failed to get container status \"6c7c4400137af6e29fc20ba2fb5391c3980a8d132729168366dd2f3f41859a8a\": rpc error: code = NotFound desc = could not find container \"6c7c4400137af6e29fc20ba2fb5391c3980a8d132729168366dd2f3f41859a8a\": container with ID starting with 6c7c4400137af6e29fc20ba2fb5391c3980a8d132729168366dd2f3f41859a8a not found: ID does not exist" Jul 19 16:44:42.978335 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.978315 2583 scope.go:117] "RemoveContainer" containerID="2c14a34aa62649d407b0b5e8bfa6e7db398bcf4924e27ab63614c9caf462e2de" Jul 19 16:44:42.978538 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.978518 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c14a34aa62649d407b0b5e8bfa6e7db398bcf4924e27ab63614c9caf462e2de"} err="failed to get container status \"2c14a34aa62649d407b0b5e8bfa6e7db398bcf4924e27ab63614c9caf462e2de\": rpc error: code = NotFound desc = could not find container \"2c14a34aa62649d407b0b5e8bfa6e7db398bcf4924e27ab63614c9caf462e2de\": container with ID starting with 2c14a34aa62649d407b0b5e8bfa6e7db398bcf4924e27ab63614c9caf462e2de not found: ID does not exist" Jul 19 16:44:42.978583 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.978539 2583 scope.go:117] "RemoveContainer" containerID="eee18131bf1ac347964fca93207f238f9cbcb350e36f5949b8150c83f8fc502a" Jul 19 16:44:42.978742 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.978726 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eee18131bf1ac347964fca93207f238f9cbcb350e36f5949b8150c83f8fc502a"} err="failed to get container status \"eee18131bf1ac347964fca93207f238f9cbcb350e36f5949b8150c83f8fc502a\": rpc error: code = NotFound desc = could not find container \"eee18131bf1ac347964fca93207f238f9cbcb350e36f5949b8150c83f8fc502a\": container with ID starting with eee18131bf1ac347964fca93207f238f9cbcb350e36f5949b8150c83f8fc502a not found: ID does not exist" Jul 19 16:44:42.978784 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.978742 2583 scope.go:117] "RemoveContainer" containerID="de33cb394ac35ddae3b49af5adbbc54042b36cea95e90f80384593f78760c0cd" Jul 19 16:44:42.978914 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.978900 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de33cb394ac35ddae3b49af5adbbc54042b36cea95e90f80384593f78760c0cd"} err="failed to get container status \"de33cb394ac35ddae3b49af5adbbc54042b36cea95e90f80384593f78760c0cd\": rpc error: code = NotFound desc = could not find container \"de33cb394ac35ddae3b49af5adbbc54042b36cea95e90f80384593f78760c0cd\": container with ID starting with de33cb394ac35ddae3b49af5adbbc54042b36cea95e90f80384593f78760c0cd not found: ID does not exist" Jul 19 16:44:42.978914 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.978913 2583 scope.go:117] "RemoveContainer" containerID="9085817976b3674ad0d8a53c31787102a9bc306864ce29bcd2b5c53ca9c1abee" Jul 19 16:44:42.979087 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.979069 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9085817976b3674ad0d8a53c31787102a9bc306864ce29bcd2b5c53ca9c1abee"} err="failed to get container status \"9085817976b3674ad0d8a53c31787102a9bc306864ce29bcd2b5c53ca9c1abee\": rpc error: code = NotFound desc = could not find container \"9085817976b3674ad0d8a53c31787102a9bc306864ce29bcd2b5c53ca9c1abee\": container with ID starting with 9085817976b3674ad0d8a53c31787102a9bc306864ce29bcd2b5c53ca9c1abee not found: ID does not exist" Jul 19 16:44:42.979135 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.979088 2583 scope.go:117] "RemoveContainer" containerID="40fe6c40615e212265c60ba92bb2c1b4be67450e23241ab1188c09d426246789" Jul 19 16:44:42.979251 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.979231 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40fe6c40615e212265c60ba92bb2c1b4be67450e23241ab1188c09d426246789"} err="failed to get container status \"40fe6c40615e212265c60ba92bb2c1b4be67450e23241ab1188c09d426246789\": rpc error: code = NotFound desc = could not find container \"40fe6c40615e212265c60ba92bb2c1b4be67450e23241ab1188c09d426246789\": container with ID starting with 40fe6c40615e212265c60ba92bb2c1b4be67450e23241ab1188c09d426246789 not found: ID does not exist" Jul 19 16:44:42.979306 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.979252 2583 scope.go:117] "RemoveContainer" containerID="6e5c17437f4c971a597369d9d0ada4949a9910e2a2925bce3614d3a3375bd246" Jul 19 16:44:42.979494 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.979474 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e5c17437f4c971a597369d9d0ada4949a9910e2a2925bce3614d3a3375bd246"} err="failed to get container status \"6e5c17437f4c971a597369d9d0ada4949a9910e2a2925bce3614d3a3375bd246\": rpc error: code = NotFound desc = could not find container \"6e5c17437f4c971a597369d9d0ada4949a9910e2a2925bce3614d3a3375bd246\": container with ID starting with 6e5c17437f4c971a597369d9d0ada4949a9910e2a2925bce3614d3a3375bd246 not found: ID does not exist" Jul 19 16:44:42.979563 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.979496 2583 scope.go:117] "RemoveContainer" containerID="6c7c4400137af6e29fc20ba2fb5391c3980a8d132729168366dd2f3f41859a8a" Jul 19 16:44:42.979711 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.979692 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c7c4400137af6e29fc20ba2fb5391c3980a8d132729168366dd2f3f41859a8a"} err="failed to get container status \"6c7c4400137af6e29fc20ba2fb5391c3980a8d132729168366dd2f3f41859a8a\": rpc error: code = NotFound desc = could not find container \"6c7c4400137af6e29fc20ba2fb5391c3980a8d132729168366dd2f3f41859a8a\": container with ID starting with 6c7c4400137af6e29fc20ba2fb5391c3980a8d132729168366dd2f3f41859a8a not found: ID does not exist" Jul 19 16:44:42.979753 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.979713 2583 scope.go:117] "RemoveContainer" containerID="2c14a34aa62649d407b0b5e8bfa6e7db398bcf4924e27ab63614c9caf462e2de" Jul 19 16:44:42.979962 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.979935 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c14a34aa62649d407b0b5e8bfa6e7db398bcf4924e27ab63614c9caf462e2de"} err="failed to get container status \"2c14a34aa62649d407b0b5e8bfa6e7db398bcf4924e27ab63614c9caf462e2de\": rpc error: code = NotFound desc = could not find container \"2c14a34aa62649d407b0b5e8bfa6e7db398bcf4924e27ab63614c9caf462e2de\": container with ID starting with 2c14a34aa62649d407b0b5e8bfa6e7db398bcf4924e27ab63614c9caf462e2de not found: ID does not exist" Jul 19 16:44:42.979962 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.979961 2583 scope.go:117] "RemoveContainer" containerID="eee18131bf1ac347964fca93207f238f9cbcb350e36f5949b8150c83f8fc502a" Jul 19 16:44:42.980292 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.980265 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eee18131bf1ac347964fca93207f238f9cbcb350e36f5949b8150c83f8fc502a"} err="failed to get container status \"eee18131bf1ac347964fca93207f238f9cbcb350e36f5949b8150c83f8fc502a\": rpc error: code = NotFound desc = could not find container \"eee18131bf1ac347964fca93207f238f9cbcb350e36f5949b8150c83f8fc502a\": container with ID starting with eee18131bf1ac347964fca93207f238f9cbcb350e36f5949b8150c83f8fc502a not found: ID does not exist" Jul 19 16:44:42.980292 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.980292 2583 scope.go:117] "RemoveContainer" containerID="de33cb394ac35ddae3b49af5adbbc54042b36cea95e90f80384593f78760c0cd" Jul 19 16:44:42.980845 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.980816 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de33cb394ac35ddae3b49af5adbbc54042b36cea95e90f80384593f78760c0cd"} err="failed to get container status \"de33cb394ac35ddae3b49af5adbbc54042b36cea95e90f80384593f78760c0cd\": rpc error: code = NotFound desc = could not find container \"de33cb394ac35ddae3b49af5adbbc54042b36cea95e90f80384593f78760c0cd\": container with ID starting with de33cb394ac35ddae3b49af5adbbc54042b36cea95e90f80384593f78760c0cd not found: ID does not exist" Jul 19 16:44:42.980951 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.980846 2583 scope.go:117] "RemoveContainer" containerID="9085817976b3674ad0d8a53c31787102a9bc306864ce29bcd2b5c53ca9c1abee" Jul 19 16:44:42.981172 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.981148 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9085817976b3674ad0d8a53c31787102a9bc306864ce29bcd2b5c53ca9c1abee"} err="failed to get container status \"9085817976b3674ad0d8a53c31787102a9bc306864ce29bcd2b5c53ca9c1abee\": rpc error: code = NotFound desc = could not find container \"9085817976b3674ad0d8a53c31787102a9bc306864ce29bcd2b5c53ca9c1abee\": container with ID starting with 9085817976b3674ad0d8a53c31787102a9bc306864ce29bcd2b5c53ca9c1abee not found: ID does not exist" Jul 19 16:44:42.981257 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.981174 2583 scope.go:117] "RemoveContainer" containerID="40fe6c40615e212265c60ba92bb2c1b4be67450e23241ab1188c09d426246789" Jul 19 16:44:42.981440 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.981418 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40fe6c40615e212265c60ba92bb2c1b4be67450e23241ab1188c09d426246789"} err="failed to get container status \"40fe6c40615e212265c60ba92bb2c1b4be67450e23241ab1188c09d426246789\": rpc error: code = NotFound desc = could not find container \"40fe6c40615e212265c60ba92bb2c1b4be67450e23241ab1188c09d426246789\": container with ID starting with 40fe6c40615e212265c60ba92bb2c1b4be67450e23241ab1188c09d426246789 not found: ID does not exist" Jul 19 16:44:42.981495 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.981443 2583 scope.go:117] "RemoveContainer" containerID="6e5c17437f4c971a597369d9d0ada4949a9910e2a2925bce3614d3a3375bd246" Jul 19 16:44:42.981670 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.981651 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e5c17437f4c971a597369d9d0ada4949a9910e2a2925bce3614d3a3375bd246"} err="failed to get container status \"6e5c17437f4c971a597369d9d0ada4949a9910e2a2925bce3614d3a3375bd246\": rpc error: code = NotFound desc = could not find container \"6e5c17437f4c971a597369d9d0ada4949a9910e2a2925bce3614d3a3375bd246\": container with ID starting with 6e5c17437f4c971a597369d9d0ada4949a9910e2a2925bce3614d3a3375bd246 not found: ID does not exist" Jul 19 16:44:42.981709 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.981671 2583 scope.go:117] "RemoveContainer" containerID="6c7c4400137af6e29fc20ba2fb5391c3980a8d132729168366dd2f3f41859a8a" Jul 19 16:44:42.981846 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.981829 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c7c4400137af6e29fc20ba2fb5391c3980a8d132729168366dd2f3f41859a8a"} err="failed to get container status \"6c7c4400137af6e29fc20ba2fb5391c3980a8d132729168366dd2f3f41859a8a\": rpc error: code = NotFound desc = could not find container \"6c7c4400137af6e29fc20ba2fb5391c3980a8d132729168366dd2f3f41859a8a\": container with ID starting with 6c7c4400137af6e29fc20ba2fb5391c3980a8d132729168366dd2f3f41859a8a not found: ID does not exist" Jul 19 16:44:42.981914 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.981846 2583 scope.go:117] "RemoveContainer" containerID="2c14a34aa62649d407b0b5e8bfa6e7db398bcf4924e27ab63614c9caf462e2de" Jul 19 16:44:42.981972 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.981954 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 19 16:44:42.982107 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.982086 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c14a34aa62649d407b0b5e8bfa6e7db398bcf4924e27ab63614c9caf462e2de"} err="failed to get container status \"2c14a34aa62649d407b0b5e8bfa6e7db398bcf4924e27ab63614c9caf462e2de\": rpc error: code = NotFound desc = could not find container \"2c14a34aa62649d407b0b5e8bfa6e7db398bcf4924e27ab63614c9caf462e2de\": container with ID starting with 2c14a34aa62649d407b0b5e8bfa6e7db398bcf4924e27ab63614c9caf462e2de not found: ID does not exist" Jul 19 16:44:42.982167 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.982108 2583 scope.go:117] "RemoveContainer" containerID="eee18131bf1ac347964fca93207f238f9cbcb350e36f5949b8150c83f8fc502a" Jul 19 16:44:42.982320 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.982307 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfd2cbe6-9b0f-4751-acfc-71dbba9ca989" containerName="prometheus" Jul 19 16:44:42.982374 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.982322 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfd2cbe6-9b0f-4751-acfc-71dbba9ca989" containerName="prometheus" Jul 19 16:44:42.982374 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.982333 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f37a3556-1276-4995-9739-32f6fece238d" containerName="console" Jul 19 16:44:42.982374 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.982342 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="f37a3556-1276-4995-9739-32f6fece238d" containerName="console" Jul 19 16:44:42.982374 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.982351 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfd2cbe6-9b0f-4751-acfc-71dbba9ca989" containerName="kube-rbac-proxy-web" Jul 19 16:44:42.982374 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.982357 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfd2cbe6-9b0f-4751-acfc-71dbba9ca989" containerName="kube-rbac-proxy-web" Jul 19 16:44:42.982374 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.982363 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfd2cbe6-9b0f-4751-acfc-71dbba9ca989" containerName="init-config-reloader" Jul 19 16:44:42.982374 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.982368 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfd2cbe6-9b0f-4751-acfc-71dbba9ca989" containerName="init-config-reloader" Jul 19 16:44:42.982374 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.982363 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eee18131bf1ac347964fca93207f238f9cbcb350e36f5949b8150c83f8fc502a"} err="failed to get container status \"eee18131bf1ac347964fca93207f238f9cbcb350e36f5949b8150c83f8fc502a\": rpc error: code = NotFound desc = could not find container \"eee18131bf1ac347964fca93207f238f9cbcb350e36f5949b8150c83f8fc502a\": container with ID starting with eee18131bf1ac347964fca93207f238f9cbcb350e36f5949b8150c83f8fc502a not found: ID does not exist" Jul 19 16:44:42.982745 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.982388 2583 scope.go:117] "RemoveContainer" containerID="de33cb394ac35ddae3b49af5adbbc54042b36cea95e90f80384593f78760c0cd" Jul 19 16:44:42.982745 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.982373 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfd2cbe6-9b0f-4751-acfc-71dbba9ca989" containerName="thanos-sidecar" Jul 19 16:44:42.982745 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.982455 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfd2cbe6-9b0f-4751-acfc-71dbba9ca989" containerName="thanos-sidecar" Jul 19 16:44:42.982745 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.982478 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfd2cbe6-9b0f-4751-acfc-71dbba9ca989" containerName="kube-rbac-proxy" Jul 19 16:44:42.982745 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.982487 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfd2cbe6-9b0f-4751-acfc-71dbba9ca989" containerName="kube-rbac-proxy" Jul 19 16:44:42.982745 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.982500 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfd2cbe6-9b0f-4751-acfc-71dbba9ca989" containerName="config-reloader" Jul 19 16:44:42.982745 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.982509 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfd2cbe6-9b0f-4751-acfc-71dbba9ca989" containerName="config-reloader" Jul 19 16:44:42.982745 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.982522 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfd2cbe6-9b0f-4751-acfc-71dbba9ca989" containerName="kube-rbac-proxy-thanos" Jul 19 16:44:42.982745 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.982530 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfd2cbe6-9b0f-4751-acfc-71dbba9ca989" containerName="kube-rbac-proxy-thanos" Jul 19 16:44:42.982745 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.982618 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="dfd2cbe6-9b0f-4751-acfc-71dbba9ca989" containerName="kube-rbac-proxy-web" Jul 19 16:44:42.982745 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.982640 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="dfd2cbe6-9b0f-4751-acfc-71dbba9ca989" containerName="kube-rbac-proxy" Jul 19 16:44:42.982745 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.982651 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="f37a3556-1276-4995-9739-32f6fece238d" containerName="console" Jul 19 16:44:42.982745 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.982663 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="dfd2cbe6-9b0f-4751-acfc-71dbba9ca989" containerName="thanos-sidecar" Jul 19 16:44:42.982745 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.982671 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="dfd2cbe6-9b0f-4751-acfc-71dbba9ca989" containerName="config-reloader" Jul 19 16:44:42.982745 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.982682 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="dfd2cbe6-9b0f-4751-acfc-71dbba9ca989" containerName="prometheus" Jul 19 16:44:42.982745 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.982690 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="dfd2cbe6-9b0f-4751-acfc-71dbba9ca989" containerName="kube-rbac-proxy-thanos" Jul 19 16:44:42.982745 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.982615 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de33cb394ac35ddae3b49af5adbbc54042b36cea95e90f80384593f78760c0cd"} err="failed to get container status \"de33cb394ac35ddae3b49af5adbbc54042b36cea95e90f80384593f78760c0cd\": rpc error: code = NotFound desc = could not find container \"de33cb394ac35ddae3b49af5adbbc54042b36cea95e90f80384593f78760c0cd\": container with ID starting with de33cb394ac35ddae3b49af5adbbc54042b36cea95e90f80384593f78760c0cd not found: ID does not exist" Jul 19 16:44:42.982745 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.982705 2583 scope.go:117] "RemoveContainer" containerID="9085817976b3674ad0d8a53c31787102a9bc306864ce29bcd2b5c53ca9c1abee" Jul 19 16:44:42.983569 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.982881 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9085817976b3674ad0d8a53c31787102a9bc306864ce29bcd2b5c53ca9c1abee"} err="failed to get container status \"9085817976b3674ad0d8a53c31787102a9bc306864ce29bcd2b5c53ca9c1abee\": rpc error: code = NotFound desc = could not find container \"9085817976b3674ad0d8a53c31787102a9bc306864ce29bcd2b5c53ca9c1abee\": container with ID starting with 9085817976b3674ad0d8a53c31787102a9bc306864ce29bcd2b5c53ca9c1abee not found: ID does not exist" Jul 19 16:44:42.983569 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.982895 2583 scope.go:117] "RemoveContainer" containerID="40fe6c40615e212265c60ba92bb2c1b4be67450e23241ab1188c09d426246789" Jul 19 16:44:42.983569 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.983090 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40fe6c40615e212265c60ba92bb2c1b4be67450e23241ab1188c09d426246789"} err="failed to get container status \"40fe6c40615e212265c60ba92bb2c1b4be67450e23241ab1188c09d426246789\": rpc error: code = NotFound desc = could not find container \"40fe6c40615e212265c60ba92bb2c1b4be67450e23241ab1188c09d426246789\": container with ID starting with 40fe6c40615e212265c60ba92bb2c1b4be67450e23241ab1188c09d426246789 not found: ID does not exist" Jul 19 16:44:42.983569 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.983108 2583 scope.go:117] "RemoveContainer" containerID="6e5c17437f4c971a597369d9d0ada4949a9910e2a2925bce3614d3a3375bd246" Jul 19 16:44:42.983569 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.983313 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e5c17437f4c971a597369d9d0ada4949a9910e2a2925bce3614d3a3375bd246"} err="failed to get container status \"6e5c17437f4c971a597369d9d0ada4949a9910e2a2925bce3614d3a3375bd246\": rpc error: code = NotFound desc = could not find container \"6e5c17437f4c971a597369d9d0ada4949a9910e2a2925bce3614d3a3375bd246\": container with ID starting with 6e5c17437f4c971a597369d9d0ada4949a9910e2a2925bce3614d3a3375bd246 not found: ID does not exist" Jul 19 16:44:42.983569 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.983330 2583 scope.go:117] "RemoveContainer" containerID="6c7c4400137af6e29fc20ba2fb5391c3980a8d132729168366dd2f3f41859a8a" Jul 19 16:44:42.983569 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.983514 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c7c4400137af6e29fc20ba2fb5391c3980a8d132729168366dd2f3f41859a8a"} err="failed to get container status \"6c7c4400137af6e29fc20ba2fb5391c3980a8d132729168366dd2f3f41859a8a\": rpc error: code = NotFound desc = could not find container \"6c7c4400137af6e29fc20ba2fb5391c3980a8d132729168366dd2f3f41859a8a\": container with ID starting with 6c7c4400137af6e29fc20ba2fb5391c3980a8d132729168366dd2f3f41859a8a not found: ID does not exist" Jul 19 16:44:42.983569 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.983531 2583 scope.go:117] "RemoveContainer" containerID="2c14a34aa62649d407b0b5e8bfa6e7db398bcf4924e27ab63614c9caf462e2de" Jul 19 16:44:42.983925 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.983726 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c14a34aa62649d407b0b5e8bfa6e7db398bcf4924e27ab63614c9caf462e2de"} err="failed to get container status \"2c14a34aa62649d407b0b5e8bfa6e7db398bcf4924e27ab63614c9caf462e2de\": rpc error: code = NotFound desc = could not find container \"2c14a34aa62649d407b0b5e8bfa6e7db398bcf4924e27ab63614c9caf462e2de\": container with ID starting with 2c14a34aa62649d407b0b5e8bfa6e7db398bcf4924e27ab63614c9caf462e2de not found: ID does not exist" Jul 19 16:44:42.983925 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.983739 2583 scope.go:117] "RemoveContainer" containerID="eee18131bf1ac347964fca93207f238f9cbcb350e36f5949b8150c83f8fc502a" Jul 19 16:44:42.983925 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.983897 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eee18131bf1ac347964fca93207f238f9cbcb350e36f5949b8150c83f8fc502a"} err="failed to get container status \"eee18131bf1ac347964fca93207f238f9cbcb350e36f5949b8150c83f8fc502a\": rpc error: code = NotFound desc = could not find container \"eee18131bf1ac347964fca93207f238f9cbcb350e36f5949b8150c83f8fc502a\": container with ID starting with eee18131bf1ac347964fca93207f238f9cbcb350e36f5949b8150c83f8fc502a not found: ID does not exist" Jul 19 16:44:42.983925 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.983909 2583 scope.go:117] "RemoveContainer" containerID="de33cb394ac35ddae3b49af5adbbc54042b36cea95e90f80384593f78760c0cd" Jul 19 16:44:42.984259 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.984134 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de33cb394ac35ddae3b49af5adbbc54042b36cea95e90f80384593f78760c0cd"} err="failed to get container status \"de33cb394ac35ddae3b49af5adbbc54042b36cea95e90f80384593f78760c0cd\": rpc error: code = NotFound desc = could not find container \"de33cb394ac35ddae3b49af5adbbc54042b36cea95e90f80384593f78760c0cd\": container with ID starting with de33cb394ac35ddae3b49af5adbbc54042b36cea95e90f80384593f78760c0cd not found: ID does not exist" Jul 19 16:44:42.984259 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.984157 2583 scope.go:117] "RemoveContainer" containerID="9085817976b3674ad0d8a53c31787102a9bc306864ce29bcd2b5c53ca9c1abee" Jul 19 16:44:42.984415 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.984389 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9085817976b3674ad0d8a53c31787102a9bc306864ce29bcd2b5c53ca9c1abee"} err="failed to get container status \"9085817976b3674ad0d8a53c31787102a9bc306864ce29bcd2b5c53ca9c1abee\": rpc error: code = NotFound desc = could not find container \"9085817976b3674ad0d8a53c31787102a9bc306864ce29bcd2b5c53ca9c1abee\": container with ID starting with 9085817976b3674ad0d8a53c31787102a9bc306864ce29bcd2b5c53ca9c1abee not found: ID does not exist" Jul 19 16:44:42.984471 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.984417 2583 scope.go:117] "RemoveContainer" containerID="40fe6c40615e212265c60ba92bb2c1b4be67450e23241ab1188c09d426246789" Jul 19 16:44:42.984698 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.984681 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40fe6c40615e212265c60ba92bb2c1b4be67450e23241ab1188c09d426246789"} err="failed to get container status \"40fe6c40615e212265c60ba92bb2c1b4be67450e23241ab1188c09d426246789\": rpc error: code = NotFound desc = could not find container \"40fe6c40615e212265c60ba92bb2c1b4be67450e23241ab1188c09d426246789\": container with ID starting with 40fe6c40615e212265c60ba92bb2c1b4be67450e23241ab1188c09d426246789 not found: ID does not exist" Jul 19 16:44:42.984744 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.984698 2583 scope.go:117] "RemoveContainer" containerID="6e5c17437f4c971a597369d9d0ada4949a9910e2a2925bce3614d3a3375bd246" Jul 19 16:44:42.984878 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.984863 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e5c17437f4c971a597369d9d0ada4949a9910e2a2925bce3614d3a3375bd246"} err="failed to get container status \"6e5c17437f4c971a597369d9d0ada4949a9910e2a2925bce3614d3a3375bd246\": rpc error: code = NotFound desc = could not find container \"6e5c17437f4c971a597369d9d0ada4949a9910e2a2925bce3614d3a3375bd246\": container with ID starting with 6e5c17437f4c971a597369d9d0ada4949a9910e2a2925bce3614d3a3375bd246 not found: ID does not exist" Jul 19 16:44:42.984919 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.984878 2583 scope.go:117] "RemoveContainer" containerID="6c7c4400137af6e29fc20ba2fb5391c3980a8d132729168366dd2f3f41859a8a" Jul 19 16:44:42.985084 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.985050 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c7c4400137af6e29fc20ba2fb5391c3980a8d132729168366dd2f3f41859a8a"} err="failed to get container status \"6c7c4400137af6e29fc20ba2fb5391c3980a8d132729168366dd2f3f41859a8a\": rpc error: code = NotFound desc = could not find container \"6c7c4400137af6e29fc20ba2fb5391c3980a8d132729168366dd2f3f41859a8a\": container with ID starting with 6c7c4400137af6e29fc20ba2fb5391c3980a8d132729168366dd2f3f41859a8a not found: ID does not exist" Jul 19 16:44:42.985155 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.985086 2583 scope.go:117] "RemoveContainer" containerID="2c14a34aa62649d407b0b5e8bfa6e7db398bcf4924e27ab63614c9caf462e2de" Jul 19 16:44:42.985354 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.985337 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c14a34aa62649d407b0b5e8bfa6e7db398bcf4924e27ab63614c9caf462e2de"} err="failed to get container status \"2c14a34aa62649d407b0b5e8bfa6e7db398bcf4924e27ab63614c9caf462e2de\": rpc error: code = NotFound desc = could not find container \"2c14a34aa62649d407b0b5e8bfa6e7db398bcf4924e27ab63614c9caf462e2de\": container with ID starting with 2c14a34aa62649d407b0b5e8bfa6e7db398bcf4924e27ab63614c9caf462e2de not found: ID does not exist" Jul 19 16:44:42.985354 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.985356 2583 scope.go:117] "RemoveContainer" containerID="eee18131bf1ac347964fca93207f238f9cbcb350e36f5949b8150c83f8fc502a" Jul 19 16:44:42.985604 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.985583 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eee18131bf1ac347964fca93207f238f9cbcb350e36f5949b8150c83f8fc502a"} err="failed to get container status \"eee18131bf1ac347964fca93207f238f9cbcb350e36f5949b8150c83f8fc502a\": rpc error: code = NotFound desc = could not find container \"eee18131bf1ac347964fca93207f238f9cbcb350e36f5949b8150c83f8fc502a\": container with ID starting with eee18131bf1ac347964fca93207f238f9cbcb350e36f5949b8150c83f8fc502a not found: ID does not exist" Jul 19 16:44:42.985651 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.985606 2583 scope.go:117] "RemoveContainer" containerID="de33cb394ac35ddae3b49af5adbbc54042b36cea95e90f80384593f78760c0cd" Jul 19 16:44:42.985795 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.985775 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de33cb394ac35ddae3b49af5adbbc54042b36cea95e90f80384593f78760c0cd"} err="failed to get container status \"de33cb394ac35ddae3b49af5adbbc54042b36cea95e90f80384593f78760c0cd\": rpc error: code = NotFound desc = could not find container \"de33cb394ac35ddae3b49af5adbbc54042b36cea95e90f80384593f78760c0cd\": container with ID starting with de33cb394ac35ddae3b49af5adbbc54042b36cea95e90f80384593f78760c0cd not found: ID does not exist" Jul 19 16:44:42.985837 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.985795 2583 scope.go:117] "RemoveContainer" containerID="9085817976b3674ad0d8a53c31787102a9bc306864ce29bcd2b5c53ca9c1abee" Jul 19 16:44:42.985988 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.985971 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9085817976b3674ad0d8a53c31787102a9bc306864ce29bcd2b5c53ca9c1abee"} err="failed to get container status \"9085817976b3674ad0d8a53c31787102a9bc306864ce29bcd2b5c53ca9c1abee\": rpc error: code = NotFound desc = could not find container \"9085817976b3674ad0d8a53c31787102a9bc306864ce29bcd2b5c53ca9c1abee\": container with ID starting with 9085817976b3674ad0d8a53c31787102a9bc306864ce29bcd2b5c53ca9c1abee not found: ID does not exist" Jul 19 16:44:42.986029 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.985989 2583 scope.go:117] "RemoveContainer" containerID="40fe6c40615e212265c60ba92bb2c1b4be67450e23241ab1188c09d426246789" Jul 19 16:44:42.986241 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.986220 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40fe6c40615e212265c60ba92bb2c1b4be67450e23241ab1188c09d426246789"} err="failed to get container status \"40fe6c40615e212265c60ba92bb2c1b4be67450e23241ab1188c09d426246789\": rpc error: code = NotFound desc = could not find container \"40fe6c40615e212265c60ba92bb2c1b4be67450e23241ab1188c09d426246789\": container with ID starting with 40fe6c40615e212265c60ba92bb2c1b4be67450e23241ab1188c09d426246789 not found: ID does not exist" Jul 19 16:44:42.986241 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.986239 2583 scope.go:117] "RemoveContainer" containerID="6e5c17437f4c971a597369d9d0ada4949a9910e2a2925bce3614d3a3375bd246" Jul 19 16:44:42.986466 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.986444 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e5c17437f4c971a597369d9d0ada4949a9910e2a2925bce3614d3a3375bd246"} err="failed to get container status \"6e5c17437f4c971a597369d9d0ada4949a9910e2a2925bce3614d3a3375bd246\": rpc error: code = NotFound desc = could not find container \"6e5c17437f4c971a597369d9d0ada4949a9910e2a2925bce3614d3a3375bd246\": container with ID starting with 6e5c17437f4c971a597369d9d0ada4949a9910e2a2925bce3614d3a3375bd246 not found: ID does not exist" Jul 19 16:44:42.986466 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.986464 2583 scope.go:117] "RemoveContainer" containerID="6c7c4400137af6e29fc20ba2fb5391c3980a8d132729168366dd2f3f41859a8a" Jul 19 16:44:42.986762 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.986728 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c7c4400137af6e29fc20ba2fb5391c3980a8d132729168366dd2f3f41859a8a"} err="failed to get container status \"6c7c4400137af6e29fc20ba2fb5391c3980a8d132729168366dd2f3f41859a8a\": rpc error: code = NotFound desc = could not find container \"6c7c4400137af6e29fc20ba2fb5391c3980a8d132729168366dd2f3f41859a8a\": container with ID starting with 6c7c4400137af6e29fc20ba2fb5391c3980a8d132729168366dd2f3f41859a8a not found: ID does not exist" Jul 19 16:44:42.988536 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.988517 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:42.990733 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.990717 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 19 16:44:42.990733 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.990727 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 19 16:44:42.991014 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.990998 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 19 16:44:42.991189 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.991169 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 19 16:44:42.991271 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.991191 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 19 16:44:42.991271 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.991241 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7ek68k7g6neh9\"" Jul 19 16:44:42.991271 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.991173 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 19 16:44:42.991691 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.991674 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 19 16:44:42.991779 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.991739 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 19 16:44:42.992106 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.992089 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 19 16:44:42.992178 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.992088 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 19 16:44:42.992378 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.992361 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 19 16:44:42.992607 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.992586 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-b5l9d\"" Jul 19 16:44:42.992755 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.992615 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 19 16:44:42.992755 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.992670 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 19 16:44:42.993659 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.993641 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 19 16:44:42.996858 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:42.996837 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 19 16:44:43.001331 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.001310 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 19 16:44:43.068095 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.068041 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgxcd\" (UniqueName: \"kubernetes.io/projected/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-kube-api-access-xgxcd\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.068265 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.068103 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.068265 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.068128 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.068265 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.068149 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-web-config\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.068265 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.068167 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.068265 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.068189 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.068265 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.068243 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-config\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.068442 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.068277 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.068442 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.068301 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.068442 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.068328 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.068442 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.068352 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.068442 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.068393 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.068442 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.068413 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.068609 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.068443 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.068609 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.068473 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.068609 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.068486 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-config-out\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.068609 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.068506 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.068609 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.068529 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.068609 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.068543 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.068609 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.068561 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.169200 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.169127 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.169200 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.169160 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.169200 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.169182 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.169717 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.169207 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgxcd\" (UniqueName: \"kubernetes.io/projected/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-kube-api-access-xgxcd\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.169717 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.169232 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.169717 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.169255 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.169717 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.169457 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-web-config\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.169717 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.169577 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.169717 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.169631 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.170052 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.169965 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.170241 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.170212 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.170363 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.170269 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-config\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.170363 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.170299 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.170363 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.170303 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.170363 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.170330 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.170588 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.170380 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.170588 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.170429 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.170588 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.170469 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.170588 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.170499 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.170588 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.170543 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.170852 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.170594 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.170852 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.170619 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-config-out\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.170852 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.170643 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.172152 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.171248 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.172258 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.172186 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.172417 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.172397 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.172493 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.172424 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.172493 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.172427 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-web-config\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.172663 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.172631 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.173128 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.173102 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.173219 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.173123 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.173356 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.173334 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-config\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.173893 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.173851 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.174758 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.174687 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.174858 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.174837 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-config-out\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.174915 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.174882 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.175622 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.175591 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.175622 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.175612 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.175779 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.175736 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.176886 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.176868 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgxcd\" (UniqueName: \"kubernetes.io/projected/0e0a7170-1745-4ae0-8cc1-4713f1cc3d61-kube-api-access-xgxcd\") pod \"prometheus-k8s-0\" (UID: \"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.298689 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.298646 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:44:43.435893 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.435822 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 19 16:44:43.439217 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:44:43.439192 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e0a7170_1745_4ae0_8cc1_4713f1cc3d61.slice/crio-a8f499970782f47c45ada6804d408133aae9318fb1ac86b3dccdacd19aa003e2 WatchSource:0}: Error finding container a8f499970782f47c45ada6804d408133aae9318fb1ac86b3dccdacd19aa003e2: Status 404 returned error can't find the container with id a8f499970782f47c45ada6804d408133aae9318fb1ac86b3dccdacd19aa003e2 Jul 19 16:44:43.601135 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.601095 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dfd2cbe6-9b0f-4751-acfc-71dbba9ca989" path="/var/lib/kubelet/pods/dfd2cbe6-9b0f-4751-acfc-71dbba9ca989/volumes" Jul 19 16:44:43.932343 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.932312 2583 generic.go:356] "Generic (PLEG): container finished" podID="0e0a7170-1745-4ae0-8cc1-4713f1cc3d61" containerID="207e9256c8f91beedd6a7609d72e8ef4c59a2ec51f5d866ec0cc8f066f92a646" exitCode=0 Jul 19 16:44:43.932518 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.932368 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61","Type":"ContainerDied","Data":"207e9256c8f91beedd6a7609d72e8ef4c59a2ec51f5d866ec0cc8f066f92a646"} Jul 19 16:44:43.932518 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:43.932415 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61","Type":"ContainerStarted","Data":"a8f499970782f47c45ada6804d408133aae9318fb1ac86b3dccdacd19aa003e2"} Jul 19 16:44:44.941003 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:44.940961 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61","Type":"ContainerStarted","Data":"9c27e26b6186d47d878533fc20b746373af72c1d9b06fd4e61a8663c69dc496d"} Jul 19 16:44:44.941507 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:44.941102 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61","Type":"ContainerStarted","Data":"260d74b376fd000271bd7af24d6fc1b3a75a4872b79fe85e59a6ac02a9483c99"} Jul 19 16:44:44.941507 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:44.941152 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61","Type":"ContainerStarted","Data":"e5a82759a8c88a86b31bdf40248d3bb81b2ebf8138f4f40c9ac398a63a730842"} Jul 19 16:44:44.941507 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:44.941187 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61","Type":"ContainerStarted","Data":"faaebe3f2103ae35da40d46176340c72c80e9da2f5bbef34a369252b52ae39ab"} Jul 19 16:44:44.941507 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:44.941223 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61","Type":"ContainerStarted","Data":"64e7d55f43e7d8faa6e5a3dc996d1056605b2ffdbe4ea037ae76976e93e18f47"} Jul 19 16:44:44.941507 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:44.941262 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0e0a7170-1745-4ae0-8cc1-4713f1cc3d61","Type":"ContainerStarted","Data":"93d288bfa5bdb2a91baef361250ac4f6450ab8650bddace7fe12fbff0e05fe56"} Jul 19 16:44:44.970405 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:44.970350 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.9515117010000003 podStartE2EDuration="2.970333891s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.82219ms" isStatefulPod=false podCreationTimestamp="2026-07-19 16:44:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 16:44:44.968171296 +0000 UTC m=+379.954746779" watchObservedRunningTime="2026-07-19 16:44:44.970333891 +0000 UTC m=+379.956909374" Jul 19 16:44:48.299735 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:44:48.299702 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:45:43.299151 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:45:43.299110 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:45:43.314485 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:45:43.314455 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:45:44.175098 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:45:44.175045 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 19 16:47:31.835422 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:47:31.835341 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-twr52"] Jul 19 16:47:31.838141 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:47:31.838121 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-twr52" Jul 19 16:47:31.840352 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:47:31.840332 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 19 16:47:31.840483 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:47:31.840464 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 19 16:47:31.841080 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:47:31.841065 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-rn2xf\"" Jul 19 16:47:31.848795 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:47:31.848774 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-twr52"] Jul 19 16:47:31.857225 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:47:31.857203 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jz27w\" (UniqueName: \"kubernetes.io/projected/e5e510b0-8c2f-4c2d-b6f7-5de76d221c13-kube-api-access-jz27w\") pod \"cert-manager-55b4d954d5-twr52\" (UID: \"e5e510b0-8c2f-4c2d-b6f7-5de76d221c13\") " pod="cert-manager/cert-manager-55b4d954d5-twr52" Jul 19 16:47:31.857347 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:47:31.857276 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e5e510b0-8c2f-4c2d-b6f7-5de76d221c13-bound-sa-token\") pod \"cert-manager-55b4d954d5-twr52\" (UID: \"e5e510b0-8c2f-4c2d-b6f7-5de76d221c13\") " pod="cert-manager/cert-manager-55b4d954d5-twr52" Jul 19 16:47:31.958720 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:47:31.958679 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jz27w\" (UniqueName: \"kubernetes.io/projected/e5e510b0-8c2f-4c2d-b6f7-5de76d221c13-kube-api-access-jz27w\") pod \"cert-manager-55b4d954d5-twr52\" (UID: \"e5e510b0-8c2f-4c2d-b6f7-5de76d221c13\") " pod="cert-manager/cert-manager-55b4d954d5-twr52" Jul 19 16:47:31.958918 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:47:31.958755 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e5e510b0-8c2f-4c2d-b6f7-5de76d221c13-bound-sa-token\") pod \"cert-manager-55b4d954d5-twr52\" (UID: \"e5e510b0-8c2f-4c2d-b6f7-5de76d221c13\") " pod="cert-manager/cert-manager-55b4d954d5-twr52" Jul 19 16:47:31.966715 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:47:31.966680 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e5e510b0-8c2f-4c2d-b6f7-5de76d221c13-bound-sa-token\") pod \"cert-manager-55b4d954d5-twr52\" (UID: \"e5e510b0-8c2f-4c2d-b6f7-5de76d221c13\") " pod="cert-manager/cert-manager-55b4d954d5-twr52" Jul 19 16:47:31.966850 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:47:31.966833 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jz27w\" (UniqueName: \"kubernetes.io/projected/e5e510b0-8c2f-4c2d-b6f7-5de76d221c13-kube-api-access-jz27w\") pod \"cert-manager-55b4d954d5-twr52\" (UID: \"e5e510b0-8c2f-4c2d-b6f7-5de76d221c13\") " pod="cert-manager/cert-manager-55b4d954d5-twr52" Jul 19 16:47:32.147241 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:47:32.147145 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-twr52" Jul 19 16:47:32.276175 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:47:32.276143 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-twr52"] Jul 19 16:47:32.279200 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:47:32.279174 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode5e510b0_8c2f_4c2d_b6f7_5de76d221c13.slice/crio-ecafdec4b13d0ae83a6e38501d47df8d17161b2c8cce56de548036ffb93574db WatchSource:0}: Error finding container ecafdec4b13d0ae83a6e38501d47df8d17161b2c8cce56de548036ffb93574db: Status 404 returned error can't find the container with id ecafdec4b13d0ae83a6e38501d47df8d17161b2c8cce56de548036ffb93574db Jul 19 16:47:32.580259 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:47:32.580227 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-twr52" event={"ID":"e5e510b0-8c2f-4c2d-b6f7-5de76d221c13","Type":"ContainerStarted","Data":"ecafdec4b13d0ae83a6e38501d47df8d17161b2c8cce56de548036ffb93574db"} Jul 19 16:47:35.594641 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:47:35.594610 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-twr52" event={"ID":"e5e510b0-8c2f-4c2d-b6f7-5de76d221c13","Type":"ContainerStarted","Data":"7b2bbf65b7d4e728f05006857bcdbe391a946da12c0ec930edec325926c13bfd"} Jul 19 16:47:35.609601 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:47:35.609547 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-twr52" podStartSLOduration=1.7325744950000002 podStartE2EDuration="4.609531478s" totalImagesPullingTime="2.876956983s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 16:47:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 16:47:35.608385554 +0000 UTC m=+550.594961037" watchObservedRunningTime="2026-07-19 16:47:35.609531478 +0000 UTC m=+550.596106962" Jul 19 16:48:00.155834 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:00.155791 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5dffb557f4-rndm6"] Jul 19 16:48:00.159494 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:00.159469 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5dffb557f4-rndm6" Jul 19 16:48:00.162214 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:00.162190 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 19 16:48:00.163890 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:00.163872 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 19 16:48:00.164170 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:00.164149 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 19 16:48:00.164566 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:00.164550 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 19 16:48:00.164647 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:00.164556 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 19 16:48:00.164829 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:00.164812 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-q7m8d\"" Jul 19 16:48:00.183647 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:00.183619 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5dffb557f4-rndm6"] Jul 19 16:48:00.304272 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:00.304230 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9b779d11-4e6f-4da7-85cb-5c2aac527bd1-manager-config\") pod \"lws-controller-manager-5dffb557f4-rndm6\" (UID: \"9b779d11-4e6f-4da7-85cb-5c2aac527bd1\") " pod="openshift-lws-operator/lws-controller-manager-5dffb557f4-rndm6" Jul 19 16:48:00.304480 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:00.304331 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9b779d11-4e6f-4da7-85cb-5c2aac527bd1-metrics-cert\") pod \"lws-controller-manager-5dffb557f4-rndm6\" (UID: \"9b779d11-4e6f-4da7-85cb-5c2aac527bd1\") " pod="openshift-lws-operator/lws-controller-manager-5dffb557f4-rndm6" Jul 19 16:48:00.304480 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:00.304372 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7l4cs\" (UniqueName: \"kubernetes.io/projected/9b779d11-4e6f-4da7-85cb-5c2aac527bd1-kube-api-access-7l4cs\") pod \"lws-controller-manager-5dffb557f4-rndm6\" (UID: \"9b779d11-4e6f-4da7-85cb-5c2aac527bd1\") " pod="openshift-lws-operator/lws-controller-manager-5dffb557f4-rndm6" Jul 19 16:48:00.304480 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:00.304429 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9b779d11-4e6f-4da7-85cb-5c2aac527bd1-cert\") pod \"lws-controller-manager-5dffb557f4-rndm6\" (UID: \"9b779d11-4e6f-4da7-85cb-5c2aac527bd1\") " pod="openshift-lws-operator/lws-controller-manager-5dffb557f4-rndm6" Jul 19 16:48:00.405334 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:00.405292 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9b779d11-4e6f-4da7-85cb-5c2aac527bd1-metrics-cert\") pod \"lws-controller-manager-5dffb557f4-rndm6\" (UID: \"9b779d11-4e6f-4da7-85cb-5c2aac527bd1\") " pod="openshift-lws-operator/lws-controller-manager-5dffb557f4-rndm6" Jul 19 16:48:00.405503 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:00.405352 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7l4cs\" (UniqueName: \"kubernetes.io/projected/9b779d11-4e6f-4da7-85cb-5c2aac527bd1-kube-api-access-7l4cs\") pod \"lws-controller-manager-5dffb557f4-rndm6\" (UID: \"9b779d11-4e6f-4da7-85cb-5c2aac527bd1\") " pod="openshift-lws-operator/lws-controller-manager-5dffb557f4-rndm6" Jul 19 16:48:00.405503 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:00.405401 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9b779d11-4e6f-4da7-85cb-5c2aac527bd1-cert\") pod \"lws-controller-manager-5dffb557f4-rndm6\" (UID: \"9b779d11-4e6f-4da7-85cb-5c2aac527bd1\") " pod="openshift-lws-operator/lws-controller-manager-5dffb557f4-rndm6" Jul 19 16:48:00.405503 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:00.405432 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9b779d11-4e6f-4da7-85cb-5c2aac527bd1-manager-config\") pod \"lws-controller-manager-5dffb557f4-rndm6\" (UID: \"9b779d11-4e6f-4da7-85cb-5c2aac527bd1\") " pod="openshift-lws-operator/lws-controller-manager-5dffb557f4-rndm6" Jul 19 16:48:00.406232 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:00.406170 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9b779d11-4e6f-4da7-85cb-5c2aac527bd1-manager-config\") pod \"lws-controller-manager-5dffb557f4-rndm6\" (UID: \"9b779d11-4e6f-4da7-85cb-5c2aac527bd1\") " pod="openshift-lws-operator/lws-controller-manager-5dffb557f4-rndm6" Jul 19 16:48:00.407858 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:00.407834 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9b779d11-4e6f-4da7-85cb-5c2aac527bd1-cert\") pod \"lws-controller-manager-5dffb557f4-rndm6\" (UID: \"9b779d11-4e6f-4da7-85cb-5c2aac527bd1\") " pod="openshift-lws-operator/lws-controller-manager-5dffb557f4-rndm6" Jul 19 16:48:00.408030 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:00.408008 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9b779d11-4e6f-4da7-85cb-5c2aac527bd1-metrics-cert\") pod \"lws-controller-manager-5dffb557f4-rndm6\" (UID: \"9b779d11-4e6f-4da7-85cb-5c2aac527bd1\") " pod="openshift-lws-operator/lws-controller-manager-5dffb557f4-rndm6" Jul 19 16:48:00.414991 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:00.414953 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7l4cs\" (UniqueName: \"kubernetes.io/projected/9b779d11-4e6f-4da7-85cb-5c2aac527bd1-kube-api-access-7l4cs\") pod \"lws-controller-manager-5dffb557f4-rndm6\" (UID: \"9b779d11-4e6f-4da7-85cb-5c2aac527bd1\") " pod="openshift-lws-operator/lws-controller-manager-5dffb557f4-rndm6" Jul 19 16:48:00.468817 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:00.468774 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5dffb557f4-rndm6" Jul 19 16:48:00.596661 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:00.596633 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5dffb557f4-rndm6"] Jul 19 16:48:00.599142 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:48:00.599115 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b779d11_4e6f_4da7_85cb_5c2aac527bd1.slice/crio-5c33b64503eb1051ccee85737ed045702770e0f2be72eaede66dd1ddb94dcc90 WatchSource:0}: Error finding container 5c33b64503eb1051ccee85737ed045702770e0f2be72eaede66dd1ddb94dcc90: Status 404 returned error can't find the container with id 5c33b64503eb1051ccee85737ed045702770e0f2be72eaede66dd1ddb94dcc90 Jul 19 16:48:00.691498 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:00.691412 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5dffb557f4-rndm6" event={"ID":"9b779d11-4e6f-4da7-85cb-5c2aac527bd1","Type":"ContainerStarted","Data":"5c33b64503eb1051ccee85737ed045702770e0f2be72eaede66dd1ddb94dcc90"} Jul 19 16:48:04.943652 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:04.943605 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5f7db45576-hndzz"] Jul 19 16:48:04.947027 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:04.947001 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5f7db45576-hndzz" Jul 19 16:48:04.949971 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:04.949947 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-9w89c\"" Jul 19 16:48:04.950150 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:04.950127 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 19 16:48:04.950343 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:04.950322 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 19 16:48:04.950547 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:04.950526 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 19 16:48:04.950642 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:04.950623 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 19 16:48:04.962210 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:04.962184 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5f7db45576-hndzz"] Jul 19 16:48:05.049079 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:05.048746 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fab2e64a-f91c-4227-bfef-7f77d0538adb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5f7db45576-hndzz\" (UID: \"fab2e64a-f91c-4227-bfef-7f77d0538adb\") " pod="opendatahub/opendatahub-operator-controller-manager-5f7db45576-hndzz" Jul 19 16:48:05.049079 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:05.048810 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fab2e64a-f91c-4227-bfef-7f77d0538adb-webhook-cert\") pod \"opendatahub-operator-controller-manager-5f7db45576-hndzz\" (UID: \"fab2e64a-f91c-4227-bfef-7f77d0538adb\") " pod="opendatahub/opendatahub-operator-controller-manager-5f7db45576-hndzz" Jul 19 16:48:05.049079 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:05.048863 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/fab2e64a-f91c-4227-bfef-7f77d0538adb-manifests\") pod \"opendatahub-operator-controller-manager-5f7db45576-hndzz\" (UID: \"fab2e64a-f91c-4227-bfef-7f77d0538adb\") " pod="opendatahub/opendatahub-operator-controller-manager-5f7db45576-hndzz" Jul 19 16:48:05.049079 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:05.048934 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tszpn\" (UniqueName: \"kubernetes.io/projected/fab2e64a-f91c-4227-bfef-7f77d0538adb-kube-api-access-tszpn\") pod \"opendatahub-operator-controller-manager-5f7db45576-hndzz\" (UID: \"fab2e64a-f91c-4227-bfef-7f77d0538adb\") " pod="opendatahub/opendatahub-operator-controller-manager-5f7db45576-hndzz" Jul 19 16:48:05.049079 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:05.048968 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fab2e64a-f91c-4227-bfef-7f77d0538adb-tmp\") pod \"opendatahub-operator-controller-manager-5f7db45576-hndzz\" (UID: \"fab2e64a-f91c-4227-bfef-7f77d0538adb\") " pod="opendatahub/opendatahub-operator-controller-manager-5f7db45576-hndzz" Jul 19 16:48:05.150176 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:05.150139 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tszpn\" (UniqueName: \"kubernetes.io/projected/fab2e64a-f91c-4227-bfef-7f77d0538adb-kube-api-access-tszpn\") pod \"opendatahub-operator-controller-manager-5f7db45576-hndzz\" (UID: \"fab2e64a-f91c-4227-bfef-7f77d0538adb\") " pod="opendatahub/opendatahub-operator-controller-manager-5f7db45576-hndzz" Jul 19 16:48:05.150387 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:05.150183 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fab2e64a-f91c-4227-bfef-7f77d0538adb-tmp\") pod \"opendatahub-operator-controller-manager-5f7db45576-hndzz\" (UID: \"fab2e64a-f91c-4227-bfef-7f77d0538adb\") " pod="opendatahub/opendatahub-operator-controller-manager-5f7db45576-hndzz" Jul 19 16:48:05.150387 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:05.150231 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fab2e64a-f91c-4227-bfef-7f77d0538adb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5f7db45576-hndzz\" (UID: \"fab2e64a-f91c-4227-bfef-7f77d0538adb\") " pod="opendatahub/opendatahub-operator-controller-manager-5f7db45576-hndzz" Jul 19 16:48:05.150387 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:05.150271 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fab2e64a-f91c-4227-bfef-7f77d0538adb-webhook-cert\") pod \"opendatahub-operator-controller-manager-5f7db45576-hndzz\" (UID: \"fab2e64a-f91c-4227-bfef-7f77d0538adb\") " pod="opendatahub/opendatahub-operator-controller-manager-5f7db45576-hndzz" Jul 19 16:48:05.150541 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:05.150429 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/fab2e64a-f91c-4227-bfef-7f77d0538adb-manifests\") pod \"opendatahub-operator-controller-manager-5f7db45576-hndzz\" (UID: \"fab2e64a-f91c-4227-bfef-7f77d0538adb\") " pod="opendatahub/opendatahub-operator-controller-manager-5f7db45576-hndzz" Jul 19 16:48:05.150919 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:05.150892 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/fab2e64a-f91c-4227-bfef-7f77d0538adb-manifests\") pod \"opendatahub-operator-controller-manager-5f7db45576-hndzz\" (UID: \"fab2e64a-f91c-4227-bfef-7f77d0538adb\") " pod="opendatahub/opendatahub-operator-controller-manager-5f7db45576-hndzz" Jul 19 16:48:05.152827 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:05.152807 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fab2e64a-f91c-4227-bfef-7f77d0538adb-tmp\") pod \"opendatahub-operator-controller-manager-5f7db45576-hndzz\" (UID: \"fab2e64a-f91c-4227-bfef-7f77d0538adb\") " pod="opendatahub/opendatahub-operator-controller-manager-5f7db45576-hndzz" Jul 19 16:48:05.153197 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:05.153179 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fab2e64a-f91c-4227-bfef-7f77d0538adb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5f7db45576-hndzz\" (UID: \"fab2e64a-f91c-4227-bfef-7f77d0538adb\") " pod="opendatahub/opendatahub-operator-controller-manager-5f7db45576-hndzz" Jul 19 16:48:05.153306 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:05.153285 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fab2e64a-f91c-4227-bfef-7f77d0538adb-webhook-cert\") pod \"opendatahub-operator-controller-manager-5f7db45576-hndzz\" (UID: \"fab2e64a-f91c-4227-bfef-7f77d0538adb\") " pod="opendatahub/opendatahub-operator-controller-manager-5f7db45576-hndzz" Jul 19 16:48:05.157739 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:05.157715 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tszpn\" (UniqueName: \"kubernetes.io/projected/fab2e64a-f91c-4227-bfef-7f77d0538adb-kube-api-access-tszpn\") pod \"opendatahub-operator-controller-manager-5f7db45576-hndzz\" (UID: \"fab2e64a-f91c-4227-bfef-7f77d0538adb\") " pod="opendatahub/opendatahub-operator-controller-manager-5f7db45576-hndzz" Jul 19 16:48:05.263323 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:05.263235 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5f7db45576-hndzz" Jul 19 16:48:05.405811 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:05.405740 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5f7db45576-hndzz"] Jul 19 16:48:05.410549 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:48:05.410518 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfab2e64a_f91c_4227_bfef_7f77d0538adb.slice/crio-542462a095d0f80080d3c5d99d4f3c09a0cb909cf9b27b5d93e9b47aafca0687 WatchSource:0}: Error finding container 542462a095d0f80080d3c5d99d4f3c09a0cb909cf9b27b5d93e9b47aafca0687: Status 404 returned error can't find the container with id 542462a095d0f80080d3c5d99d4f3c09a0cb909cf9b27b5d93e9b47aafca0687 Jul 19 16:48:05.721529 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:05.721496 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5f7db45576-hndzz" event={"ID":"fab2e64a-f91c-4227-bfef-7f77d0538adb","Type":"ContainerStarted","Data":"542462a095d0f80080d3c5d99d4f3c09a0cb909cf9b27b5d93e9b47aafca0687"} Jul 19 16:48:08.738755 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:08.738719 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5f7db45576-hndzz" event={"ID":"fab2e64a-f91c-4227-bfef-7f77d0538adb","Type":"ContainerStarted","Data":"5b71088ccb0e8fb3ee9ae3e2c8b58cc1f11c511194f8ac837fa2bb8270427554"} Jul 19 16:48:09.747551 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:09.747464 2583 generic.go:356] "Generic (PLEG): container finished" podID="fab2e64a-f91c-4227-bfef-7f77d0538adb" containerID="5b71088ccb0e8fb3ee9ae3e2c8b58cc1f11c511194f8ac837fa2bb8270427554" exitCode=0 Jul 19 16:48:09.747923 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:09.747556 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5f7db45576-hndzz" event={"ID":"fab2e64a-f91c-4227-bfef-7f77d0538adb","Type":"ContainerDied","Data":"5b71088ccb0e8fb3ee9ae3e2c8b58cc1f11c511194f8ac837fa2bb8270427554"} Jul 19 16:48:10.753497 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:10.753460 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5f7db45576-hndzz" event={"ID":"fab2e64a-f91c-4227-bfef-7f77d0538adb","Type":"ContainerStarted","Data":"85ccd255503e7bada6192e824a7b46ca48eb2c7c1a02d5cdcc44b6df760b62ca"} Jul 19 16:48:10.753886 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:10.753621 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5f7db45576-hndzz" Jul 19 16:48:10.775202 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:10.775149 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5f7db45576-hndzz" podStartSLOduration=2.4559010949999998 podStartE2EDuration="6.775133315s" totalImagesPullingTime="3.264069813s" totalInitContainerRuntime="1.055162407s" isStatefulPod=false podCreationTimestamp="2026-07-19 16:48:04 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 16:48:10.773293665 +0000 UTC m=+585.759869150" watchObservedRunningTime="2026-07-19 16:48:10.775133315 +0000 UTC m=+585.761708799" Jul 19 16:48:21.760546 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:21.760511 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5f7db45576-hndzz" Jul 19 16:48:21.802526 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:21.802488 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5dffb557f4-rndm6" event={"ID":"9b779d11-4e6f-4da7-85cb-5c2aac527bd1","Type":"ContainerStarted","Data":"a065d5240df117432b36471567abb8eb8278158a0ea86d3a7d3f8f8caf57b731"} Jul 19 16:48:21.802725 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:21.802607 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5dffb557f4-rndm6" Jul 19 16:48:32.269531 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:32.269471 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5dffb557f4-rndm6" podStartSLOduration=11.818783643 podStartE2EDuration="32.269450826s" totalImagesPullingTime="20.450667183s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 16:48:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 16:48:21.819326665 +0000 UTC m=+596.805902149" watchObservedRunningTime="2026-07-19 16:48:32.269450826 +0000 UTC m=+607.256026312" Jul 19 16:48:32.271398 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:32.271373 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kvhsx"] Jul 19 16:48:32.273607 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:32.273587 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kvhsx" Jul 19 16:48:32.276107 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:32.276085 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 19 16:48:32.276287 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:32.276267 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-r5b8j\"" Jul 19 16:48:32.288382 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:32.288359 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kvhsx"] Jul 19 16:48:32.377435 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:32.377404 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2833b8cd-b6e8-454f-88ea-fd29daf550db-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4kvhsx\" (UID: \"2833b8cd-b6e8-454f-88ea-fd29daf550db\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kvhsx" Jul 19 16:48:32.377435 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:32.377438 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m547z\" (UniqueName: \"kubernetes.io/projected/2833b8cd-b6e8-454f-88ea-fd29daf550db-kube-api-access-m547z\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4kvhsx\" (UID: \"2833b8cd-b6e8-454f-88ea-fd29daf550db\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kvhsx" Jul 19 16:48:32.377657 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:32.377465 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2833b8cd-b6e8-454f-88ea-fd29daf550db-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4kvhsx\" (UID: \"2833b8cd-b6e8-454f-88ea-fd29daf550db\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kvhsx" Jul 19 16:48:32.377657 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:32.377519 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2833b8cd-b6e8-454f-88ea-fd29daf550db-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4kvhsx\" (UID: \"2833b8cd-b6e8-454f-88ea-fd29daf550db\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kvhsx" Jul 19 16:48:32.377657 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:32.377536 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2833b8cd-b6e8-454f-88ea-fd29daf550db-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4kvhsx\" (UID: \"2833b8cd-b6e8-454f-88ea-fd29daf550db\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kvhsx" Jul 19 16:48:32.377657 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:32.377633 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2833b8cd-b6e8-454f-88ea-fd29daf550db-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4kvhsx\" (UID: \"2833b8cd-b6e8-454f-88ea-fd29daf550db\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kvhsx" Jul 19 16:48:32.377827 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:32.377674 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2833b8cd-b6e8-454f-88ea-fd29daf550db-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4kvhsx\" (UID: \"2833b8cd-b6e8-454f-88ea-fd29daf550db\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kvhsx" Jul 19 16:48:32.377827 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:32.377701 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2833b8cd-b6e8-454f-88ea-fd29daf550db-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4kvhsx\" (UID: \"2833b8cd-b6e8-454f-88ea-fd29daf550db\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kvhsx" Jul 19 16:48:32.377827 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:32.377759 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2833b8cd-b6e8-454f-88ea-fd29daf550db-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4kvhsx\" (UID: \"2833b8cd-b6e8-454f-88ea-fd29daf550db\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kvhsx" Jul 19 16:48:32.478871 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:32.478837 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2833b8cd-b6e8-454f-88ea-fd29daf550db-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4kvhsx\" (UID: \"2833b8cd-b6e8-454f-88ea-fd29daf550db\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kvhsx" Jul 19 16:48:32.479090 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:32.478891 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2833b8cd-b6e8-454f-88ea-fd29daf550db-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4kvhsx\" (UID: \"2833b8cd-b6e8-454f-88ea-fd29daf550db\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kvhsx" Jul 19 16:48:32.479090 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:32.478911 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m547z\" (UniqueName: \"kubernetes.io/projected/2833b8cd-b6e8-454f-88ea-fd29daf550db-kube-api-access-m547z\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4kvhsx\" (UID: \"2833b8cd-b6e8-454f-88ea-fd29daf550db\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kvhsx" Jul 19 16:48:32.479090 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:32.478937 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2833b8cd-b6e8-454f-88ea-fd29daf550db-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4kvhsx\" (UID: \"2833b8cd-b6e8-454f-88ea-fd29daf550db\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kvhsx" Jul 19 16:48:32.479090 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:32.478966 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2833b8cd-b6e8-454f-88ea-fd29daf550db-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4kvhsx\" (UID: \"2833b8cd-b6e8-454f-88ea-fd29daf550db\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kvhsx" Jul 19 16:48:32.479090 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:32.478987 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2833b8cd-b6e8-454f-88ea-fd29daf550db-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4kvhsx\" (UID: \"2833b8cd-b6e8-454f-88ea-fd29daf550db\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kvhsx" Jul 19 16:48:32.479090 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:32.479024 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2833b8cd-b6e8-454f-88ea-fd29daf550db-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4kvhsx\" (UID: \"2833b8cd-b6e8-454f-88ea-fd29daf550db\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kvhsx" Jul 19 16:48:32.479090 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:32.479051 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2833b8cd-b6e8-454f-88ea-fd29daf550db-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4kvhsx\" (UID: \"2833b8cd-b6e8-454f-88ea-fd29daf550db\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kvhsx" Jul 19 16:48:32.479459 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:32.479127 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2833b8cd-b6e8-454f-88ea-fd29daf550db-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4kvhsx\" (UID: \"2833b8cd-b6e8-454f-88ea-fd29daf550db\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kvhsx" Jul 19 16:48:32.479517 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:32.479483 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2833b8cd-b6e8-454f-88ea-fd29daf550db-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4kvhsx\" (UID: \"2833b8cd-b6e8-454f-88ea-fd29daf550db\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kvhsx" Jul 19 16:48:32.479597 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:32.479566 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2833b8cd-b6e8-454f-88ea-fd29daf550db-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4kvhsx\" (UID: \"2833b8cd-b6e8-454f-88ea-fd29daf550db\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kvhsx" Jul 19 16:48:32.479800 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:32.479775 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2833b8cd-b6e8-454f-88ea-fd29daf550db-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4kvhsx\" (UID: \"2833b8cd-b6e8-454f-88ea-fd29daf550db\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kvhsx" Jul 19 16:48:32.479800 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:32.479790 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2833b8cd-b6e8-454f-88ea-fd29daf550db-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4kvhsx\" (UID: \"2833b8cd-b6e8-454f-88ea-fd29daf550db\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kvhsx" Jul 19 16:48:32.479951 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:32.479919 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2833b8cd-b6e8-454f-88ea-fd29daf550db-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4kvhsx\" (UID: \"2833b8cd-b6e8-454f-88ea-fd29daf550db\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kvhsx" Jul 19 16:48:32.482090 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:32.482038 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2833b8cd-b6e8-454f-88ea-fd29daf550db-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4kvhsx\" (UID: \"2833b8cd-b6e8-454f-88ea-fd29daf550db\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kvhsx" Jul 19 16:48:32.482298 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:32.482274 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2833b8cd-b6e8-454f-88ea-fd29daf550db-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4kvhsx\" (UID: \"2833b8cd-b6e8-454f-88ea-fd29daf550db\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kvhsx" Jul 19 16:48:32.502628 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:32.502598 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m547z\" (UniqueName: \"kubernetes.io/projected/2833b8cd-b6e8-454f-88ea-fd29daf550db-kube-api-access-m547z\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4kvhsx\" (UID: \"2833b8cd-b6e8-454f-88ea-fd29daf550db\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kvhsx" Jul 19 16:48:32.508772 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:32.508747 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2833b8cd-b6e8-454f-88ea-fd29daf550db-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4kvhsx\" (UID: \"2833b8cd-b6e8-454f-88ea-fd29daf550db\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kvhsx" Jul 19 16:48:32.583485 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:32.583451 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kvhsx" Jul 19 16:48:32.724341 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:32.724299 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kvhsx"] Jul 19 16:48:32.727892 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:48:32.727855 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2833b8cd_b6e8_454f_88ea_fd29daf550db.slice/crio-c4bd7495397ce6b8e73d7a968a834cda2dd1e237704404f914482bbab8f864ef WatchSource:0}: Error finding container c4bd7495397ce6b8e73d7a968a834cda2dd1e237704404f914482bbab8f864ef: Status 404 returned error can't find the container with id c4bd7495397ce6b8e73d7a968a834cda2dd1e237704404f914482bbab8f864ef Jul 19 16:48:32.809024 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:32.808992 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5dffb557f4-rndm6" Jul 19 16:48:32.851537 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:32.851424 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kvhsx" event={"ID":"2833b8cd-b6e8-454f-88ea-fd29daf550db","Type":"ContainerStarted","Data":"c4bd7495397ce6b8e73d7a968a834cda2dd1e237704404f914482bbab8f864ef"} Jul 19 16:48:34.181951 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:34.181914 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fvj5h"] Jul 19 16:48:34.184690 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:34.184659 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fvj5h" Jul 19 16:48:34.197052 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:34.195941 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 19 16:48:34.197052 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:34.196106 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-m45md\"" Jul 19 16:48:34.197052 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:34.196245 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 19 16:48:34.197341 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:34.197265 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 19 16:48:34.199522 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:34.199489 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fvj5h"] Jul 19 16:48:34.296810 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:34.296775 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/53fb7f67-967e-46d9-baed-fbc03e25e04b-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-fvj5h\" (UID: \"53fb7f67-967e-46d9-baed-fbc03e25e04b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fvj5h" Jul 19 16:48:34.297001 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:34.296816 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/53fb7f67-967e-46d9-baed-fbc03e25e04b-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-fvj5h\" (UID: \"53fb7f67-967e-46d9-baed-fbc03e25e04b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fvj5h" Jul 19 16:48:34.297001 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:34.296849 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6nr7\" (UniqueName: \"kubernetes.io/projected/53fb7f67-967e-46d9-baed-fbc03e25e04b-kube-api-access-j6nr7\") pod \"istiod-openshift-gateway-94bb8fbfd-fvj5h\" (UID: \"53fb7f67-967e-46d9-baed-fbc03e25e04b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fvj5h" Jul 19 16:48:34.297001 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:34.296890 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/53fb7f67-967e-46d9-baed-fbc03e25e04b-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-fvj5h\" (UID: \"53fb7f67-967e-46d9-baed-fbc03e25e04b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fvj5h" Jul 19 16:48:34.297001 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:34.296946 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/53fb7f67-967e-46d9-baed-fbc03e25e04b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-fvj5h\" (UID: \"53fb7f67-967e-46d9-baed-fbc03e25e04b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fvj5h" Jul 19 16:48:34.297001 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:34.296996 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/53fb7f67-967e-46d9-baed-fbc03e25e04b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-fvj5h\" (UID: \"53fb7f67-967e-46d9-baed-fbc03e25e04b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fvj5h" Jul 19 16:48:34.297259 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:34.297017 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/53fb7f67-967e-46d9-baed-fbc03e25e04b-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-fvj5h\" (UID: \"53fb7f67-967e-46d9-baed-fbc03e25e04b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fvj5h" Jul 19 16:48:34.398282 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:34.398243 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/53fb7f67-967e-46d9-baed-fbc03e25e04b-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-fvj5h\" (UID: \"53fb7f67-967e-46d9-baed-fbc03e25e04b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fvj5h" Jul 19 16:48:34.398474 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:34.398300 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/53fb7f67-967e-46d9-baed-fbc03e25e04b-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-fvj5h\" (UID: \"53fb7f67-967e-46d9-baed-fbc03e25e04b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fvj5h" Jul 19 16:48:34.398544 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:34.398478 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6nr7\" (UniqueName: \"kubernetes.io/projected/53fb7f67-967e-46d9-baed-fbc03e25e04b-kube-api-access-j6nr7\") pod \"istiod-openshift-gateway-94bb8fbfd-fvj5h\" (UID: \"53fb7f67-967e-46d9-baed-fbc03e25e04b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fvj5h" Jul 19 16:48:34.398608 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:34.398557 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/53fb7f67-967e-46d9-baed-fbc03e25e04b-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-fvj5h\" (UID: \"53fb7f67-967e-46d9-baed-fbc03e25e04b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fvj5h" Jul 19 16:48:34.398658 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:34.398619 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/53fb7f67-967e-46d9-baed-fbc03e25e04b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-fvj5h\" (UID: \"53fb7f67-967e-46d9-baed-fbc03e25e04b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fvj5h" Jul 19 16:48:34.399220 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:34.398761 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/53fb7f67-967e-46d9-baed-fbc03e25e04b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-fvj5h\" (UID: \"53fb7f67-967e-46d9-baed-fbc03e25e04b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fvj5h" Jul 19 16:48:34.399220 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:34.398826 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/53fb7f67-967e-46d9-baed-fbc03e25e04b-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-fvj5h\" (UID: \"53fb7f67-967e-46d9-baed-fbc03e25e04b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fvj5h" Jul 19 16:48:34.399553 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:34.399528 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/53fb7f67-967e-46d9-baed-fbc03e25e04b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-fvj5h\" (UID: \"53fb7f67-967e-46d9-baed-fbc03e25e04b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fvj5h" Jul 19 16:48:34.401139 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:34.401117 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/53fb7f67-967e-46d9-baed-fbc03e25e04b-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-fvj5h\" (UID: \"53fb7f67-967e-46d9-baed-fbc03e25e04b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fvj5h" Jul 19 16:48:34.401407 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:34.401381 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/53fb7f67-967e-46d9-baed-fbc03e25e04b-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-fvj5h\" (UID: \"53fb7f67-967e-46d9-baed-fbc03e25e04b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fvj5h" Jul 19 16:48:34.401499 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:34.401450 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/53fb7f67-967e-46d9-baed-fbc03e25e04b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-fvj5h\" (UID: \"53fb7f67-967e-46d9-baed-fbc03e25e04b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fvj5h" Jul 19 16:48:34.401558 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:34.401513 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/53fb7f67-967e-46d9-baed-fbc03e25e04b-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-fvj5h\" (UID: \"53fb7f67-967e-46d9-baed-fbc03e25e04b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fvj5h" Jul 19 16:48:34.414512 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:34.414482 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6nr7\" (UniqueName: \"kubernetes.io/projected/53fb7f67-967e-46d9-baed-fbc03e25e04b-kube-api-access-j6nr7\") pod \"istiod-openshift-gateway-94bb8fbfd-fvj5h\" (UID: \"53fb7f67-967e-46d9-baed-fbc03e25e04b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fvj5h" Jul 19 16:48:34.419575 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:34.419542 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/53fb7f67-967e-46d9-baed-fbc03e25e04b-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-fvj5h\" (UID: \"53fb7f67-967e-46d9-baed-fbc03e25e04b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fvj5h" Jul 19 16:48:34.505539 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:34.505460 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fvj5h" Jul 19 16:48:34.678108 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:34.678077 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fvj5h"] Jul 19 16:48:35.668177 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:48:35.668139 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53fb7f67_967e_46d9_baed_fbc03e25e04b.slice/crio-0ec3f09eebb035e7679e657ca1d8623ffafe3de64fb03efe8f24c5ad074141f6 WatchSource:0}: Error finding container 0ec3f09eebb035e7679e657ca1d8623ffafe3de64fb03efe8f24c5ad074141f6: Status 404 returned error can't find the container with id 0ec3f09eebb035e7679e657ca1d8623ffafe3de64fb03efe8f24c5ad074141f6 Jul 19 16:48:35.728241 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:35.728195 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 16:48:35.728363 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:35.728276 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 16:48:35.728363 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:35.728306 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 16:48:35.872408 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:35.872361 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kvhsx" event={"ID":"2833b8cd-b6e8-454f-88ea-fd29daf550db","Type":"ContainerStarted","Data":"fc0748db953ec21b6ca9970ceaeab20ce52fc4452359bdd2cdc0f6d0cc2291c4"} Jul 19 16:48:35.873741 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:35.873708 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fvj5h" event={"ID":"53fb7f67-967e-46d9-baed-fbc03e25e04b","Type":"ContainerStarted","Data":"0ec3f09eebb035e7679e657ca1d8623ffafe3de64fb03efe8f24c5ad074141f6"} Jul 19 16:48:35.894979 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:35.894925 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kvhsx" podStartSLOduration=0.896884247 podStartE2EDuration="3.894909027s" totalImagesPullingTime="2.99802478s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 16:48:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 16:48:35.893743319 +0000 UTC m=+610.880318804" watchObservedRunningTime="2026-07-19 16:48:35.894909027 +0000 UTC m=+610.881484511" Jul 19 16:48:36.583803 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:36.583765 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kvhsx" Jul 19 16:48:36.589639 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:36.589607 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kvhsx" Jul 19 16:48:36.879255 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:36.879172 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kvhsx" Jul 19 16:48:36.880235 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:36.880214 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kvhsx" Jul 19 16:48:38.044083 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:38.044022 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 16:48:38.044353 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:38.044115 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 16:48:38.892284 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:38.892245 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fvj5h" event={"ID":"53fb7f67-967e-46d9-baed-fbc03e25e04b","Type":"ContainerStarted","Data":"9251fe8641dceb239d4c39417d75a56eab3bc62fdab092ec9611584eeb158864"} Jul 19 16:48:38.892618 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:38.892592 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fvj5h" Jul 19 16:48:38.933226 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:38.933173 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fvj5h" podStartSLOduration=2.560064456 podStartE2EDuration="4.933154371s" totalImagesPullingTime="2.373089915s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 16:48:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 16:48:38.930697203 +0000 UTC m=+613.917272685" watchObservedRunningTime="2026-07-19 16:48:38.933154371 +0000 UTC m=+613.919729854" Jul 19 16:48:39.898868 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:48:39.898840 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fvj5h" Jul 19 16:50:46.629476 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:50:46.629397 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5frw9"] Jul 19 16:50:46.633125 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:50:46.633047 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5frw9" Jul 19 16:50:46.635439 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:50:46.635416 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-gf29f\"" Jul 19 16:50:46.645206 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:50:46.645182 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5frw9"] Jul 19 16:50:46.674663 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:50:46.674629 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9660736b-a005-469d-af64-0088155863a5-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-5frw9\" (UID: \"9660736b-a005-469d-af64-0088155863a5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5frw9" Jul 19 16:50:46.674826 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:50:46.674699 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9660736b-a005-469d-af64-0088155863a5-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-5frw9\" (UID: \"9660736b-a005-469d-af64-0088155863a5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5frw9" Jul 19 16:50:46.674826 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:50:46.674808 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9660736b-a005-469d-af64-0088155863a5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-5frw9\" (UID: \"9660736b-a005-469d-af64-0088155863a5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5frw9" Jul 19 16:50:46.674960 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:50:46.674850 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xsrrc\" (UniqueName: \"kubernetes.io/projected/9660736b-a005-469d-af64-0088155863a5-kube-api-access-xsrrc\") pod \"maas-default-gateway-openshift-default-56654dcf7-5frw9\" (UID: \"9660736b-a005-469d-af64-0088155863a5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5frw9" Jul 19 16:50:46.675027 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:50:46.675002 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9660736b-a005-469d-af64-0088155863a5-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-5frw9\" (UID: \"9660736b-a005-469d-af64-0088155863a5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5frw9" Jul 19 16:50:46.675109 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:50:46.675048 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9660736b-a005-469d-af64-0088155863a5-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-5frw9\" (UID: \"9660736b-a005-469d-af64-0088155863a5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5frw9" Jul 19 16:50:46.675161 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:50:46.675149 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9660736b-a005-469d-af64-0088155863a5-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-5frw9\" (UID: \"9660736b-a005-469d-af64-0088155863a5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5frw9" Jul 19 16:50:46.675207 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:50:46.675191 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9660736b-a005-469d-af64-0088155863a5-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-5frw9\" (UID: \"9660736b-a005-469d-af64-0088155863a5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5frw9" Jul 19 16:50:46.675264 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:50:46.675247 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9660736b-a005-469d-af64-0088155863a5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-5frw9\" (UID: \"9660736b-a005-469d-af64-0088155863a5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5frw9" Jul 19 16:50:46.776219 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:50:46.776184 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9660736b-a005-469d-af64-0088155863a5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-5frw9\" (UID: \"9660736b-a005-469d-af64-0088155863a5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5frw9" Jul 19 16:50:46.776219 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:50:46.776225 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9660736b-a005-469d-af64-0088155863a5-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-5frw9\" (UID: \"9660736b-a005-469d-af64-0088155863a5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5frw9" Jul 19 16:50:46.776429 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:50:46.776352 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9660736b-a005-469d-af64-0088155863a5-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-5frw9\" (UID: \"9660736b-a005-469d-af64-0088155863a5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5frw9" Jul 19 16:50:46.776429 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:50:46.776413 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9660736b-a005-469d-af64-0088155863a5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-5frw9\" (UID: \"9660736b-a005-469d-af64-0088155863a5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5frw9" Jul 19 16:50:46.776519 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:50:46.776436 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xsrrc\" (UniqueName: \"kubernetes.io/projected/9660736b-a005-469d-af64-0088155863a5-kube-api-access-xsrrc\") pod \"maas-default-gateway-openshift-default-56654dcf7-5frw9\" (UID: \"9660736b-a005-469d-af64-0088155863a5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5frw9" Jul 19 16:50:46.776519 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:50:46.776476 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9660736b-a005-469d-af64-0088155863a5-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-5frw9\" (UID: \"9660736b-a005-469d-af64-0088155863a5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5frw9" Jul 19 16:50:46.776519 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:50:46.776507 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9660736b-a005-469d-af64-0088155863a5-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-5frw9\" (UID: \"9660736b-a005-469d-af64-0088155863a5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5frw9" Jul 19 16:50:46.776655 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:50:46.776543 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9660736b-a005-469d-af64-0088155863a5-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-5frw9\" (UID: \"9660736b-a005-469d-af64-0088155863a5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5frw9" Jul 19 16:50:46.776655 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:50:46.776575 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9660736b-a005-469d-af64-0088155863a5-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-5frw9\" (UID: \"9660736b-a005-469d-af64-0088155863a5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5frw9" Jul 19 16:50:46.776767 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:50:46.776656 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9660736b-a005-469d-af64-0088155863a5-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-5frw9\" (UID: \"9660736b-a005-469d-af64-0088155863a5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5frw9" Jul 19 16:50:46.776902 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:50:46.776880 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9660736b-a005-469d-af64-0088155863a5-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-5frw9\" (UID: \"9660736b-a005-469d-af64-0088155863a5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5frw9" Jul 19 16:50:46.776984 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:50:46.776937 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9660736b-a005-469d-af64-0088155863a5-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-5frw9\" (UID: \"9660736b-a005-469d-af64-0088155863a5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5frw9" Jul 19 16:50:46.777087 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:50:46.777051 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9660736b-a005-469d-af64-0088155863a5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-5frw9\" (UID: \"9660736b-a005-469d-af64-0088155863a5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5frw9" Jul 19 16:50:46.777157 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:50:46.777050 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9660736b-a005-469d-af64-0088155863a5-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-5frw9\" (UID: \"9660736b-a005-469d-af64-0088155863a5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5frw9" Jul 19 16:50:46.778922 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:50:46.778896 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9660736b-a005-469d-af64-0088155863a5-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-5frw9\" (UID: \"9660736b-a005-469d-af64-0088155863a5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5frw9" Jul 19 16:50:46.779150 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:50:46.779132 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9660736b-a005-469d-af64-0088155863a5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-5frw9\" (UID: \"9660736b-a005-469d-af64-0088155863a5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5frw9" Jul 19 16:50:46.795277 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:50:46.795247 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9660736b-a005-469d-af64-0088155863a5-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-5frw9\" (UID: \"9660736b-a005-469d-af64-0088155863a5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5frw9" Jul 19 16:50:46.801723 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:50:46.801698 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xsrrc\" (UniqueName: \"kubernetes.io/projected/9660736b-a005-469d-af64-0088155863a5-kube-api-access-xsrrc\") pod \"maas-default-gateway-openshift-default-56654dcf7-5frw9\" (UID: \"9660736b-a005-469d-af64-0088155863a5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5frw9" Jul 19 16:50:46.944264 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:50:46.944167 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5frw9" Jul 19 16:50:47.088495 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:50:47.088471 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5frw9"] Jul 19 16:50:47.090794 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:50:47.090765 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9660736b_a005_469d_af64_0088155863a5.slice/crio-23d2a4625d1112445337a8d2639e0ecdc6d44aeaece77f0ce2ce5d831b495a66 WatchSource:0}: Error finding container 23d2a4625d1112445337a8d2639e0ecdc6d44aeaece77f0ce2ce5d831b495a66: Status 404 returned error can't find the container with id 23d2a4625d1112445337a8d2639e0ecdc6d44aeaece77f0ce2ce5d831b495a66 Jul 19 16:50:47.092913 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:50:47.092882 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 16:50:47.092974 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:50:47.092951 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 16:50:47.093014 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:50:47.092979 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 16:50:47.451893 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:50:47.451856 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5frw9" event={"ID":"9660736b-a005-469d-af64-0088155863a5","Type":"ContainerStarted","Data":"59b6ef2965c1fe202dcf52028810a454ccc442638594794ce8ed7e5a5829d6d8"} Jul 19 16:50:47.452088 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:50:47.451939 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5frw9" event={"ID":"9660736b-a005-469d-af64-0088155863a5","Type":"ContainerStarted","Data":"23d2a4625d1112445337a8d2639e0ecdc6d44aeaece77f0ce2ce5d831b495a66"} Jul 19 16:50:47.471078 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:50:47.471013 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5frw9" podStartSLOduration=1.470998039 podStartE2EDuration="1.470998039s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 16:50:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 16:50:47.468857273 +0000 UTC m=+742.455432756" watchObservedRunningTime="2026-07-19 16:50:47.470998039 +0000 UTC m=+742.457573523" Jul 19 16:50:47.945107 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:50:47.945047 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5frw9" Jul 19 16:50:48.950608 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:50:48.950578 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5frw9" Jul 19 16:50:49.461025 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:50:49.460991 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5frw9" Jul 19 16:50:49.462022 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:50:49.462001 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5frw9" Jul 19 16:52:14.842262 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:52:14.842228 2583 generic.go:356] "Generic (PLEG): container finished" podID="9660736b-a005-469d-af64-0088155863a5" containerID="59b6ef2965c1fe202dcf52028810a454ccc442638594794ce8ed7e5a5829d6d8" exitCode=0 Jul 19 16:52:14.842687 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:52:14.842302 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5frw9" event={"ID":"9660736b-a005-469d-af64-0088155863a5","Type":"ContainerDied","Data":"59b6ef2965c1fe202dcf52028810a454ccc442638594794ce8ed7e5a5829d6d8"} Jul 19 16:52:14.842743 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:52:14.842725 2583 scope.go:117] "RemoveContainer" containerID="59b6ef2965c1fe202dcf52028810a454ccc442638594794ce8ed7e5a5829d6d8" Jul 19 16:52:14.843734 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:52:14.843696 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 16:52:14.843849 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:52:14.843780 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 16:52:14.843849 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:52:14.843829 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 16:52:15.852634 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:52:15.852599 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5frw9" event={"ID":"9660736b-a005-469d-af64-0088155863a5","Type":"ContainerStarted","Data":"9e3035907caa72a37a07333d9efcc65fd3333962672fe3d9c7c07dac632969ca"} Jul 19 16:52:15.944874 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:52:15.944829 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5frw9" Jul 19 16:52:15.946244 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:52:15.946214 2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-5frw9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused" start-of-body= Jul 19 16:52:15.946510 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:52:15.946471 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5frw9" podUID="9660736b-a005-469d-af64-0088155863a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused" Jul 19 16:52:16.945452 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:52:16.945405 2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-5frw9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused" start-of-body= Jul 19 16:52:16.945931 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:52:16.945479 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5frw9" podUID="9660736b-a005-469d-af64-0088155863a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused" Jul 19 16:52:17.945474 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:52:17.945437 2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-5frw9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused" start-of-body= Jul 19 16:52:17.946076 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:52:17.946025 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5frw9" podUID="9660736b-a005-469d-af64-0088155863a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused" Jul 19 16:52:18.945200 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:52:18.945158 2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-5frw9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused" start-of-body= Jul 19 16:52:18.945504 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:52:18.945469 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5frw9" podUID="9660736b-a005-469d-af64-0088155863a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused" Jul 19 16:52:19.461992 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:52:19.461952 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5frw9" Jul 19 16:52:19.948033 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:52:19.948007 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5frw9" Jul 19 16:52:19.949035 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:52:19.949016 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-5frw9" Jul 19 16:52:37.026001 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:52:37.025967 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-qd65d"] Jul 19 16:52:37.028451 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:52:37.028431 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qd65d" Jul 19 16:52:37.030641 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:52:37.030617 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 19 16:52:37.030641 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:52:37.030630 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 19 16:52:37.030800 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:52:37.030627 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 19 16:52:37.031329 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:52:37.031312 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-kp5v5\"" Jul 19 16:52:37.043337 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:52:37.043300 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-qd65d"] Jul 19 16:52:37.147931 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:52:37.147894 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0854ed41-d942-4cf3-b58f-0d396128a995-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qd65d\" (UID: \"0854ed41-d942-4cf3-b58f-0d396128a995\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qd65d" Jul 19 16:52:37.148130 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:52:37.147950 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0854ed41-d942-4cf3-b58f-0d396128a995-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qd65d\" (UID: \"0854ed41-d942-4cf3-b58f-0d396128a995\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qd65d" Jul 19 16:52:37.148130 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:52:37.147970 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0854ed41-d942-4cf3-b58f-0d396128a995-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qd65d\" (UID: \"0854ed41-d942-4cf3-b58f-0d396128a995\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qd65d" Jul 19 16:52:37.148130 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:52:37.148051 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0854ed41-d942-4cf3-b58f-0d396128a995-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qd65d\" (UID: \"0854ed41-d942-4cf3-b58f-0d396128a995\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qd65d" Jul 19 16:52:37.148236 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:52:37.148126 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0854ed41-d942-4cf3-b58f-0d396128a995-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qd65d\" (UID: \"0854ed41-d942-4cf3-b58f-0d396128a995\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qd65d" Jul 19 16:52:37.148236 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:52:37.148163 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-585q2\" (UniqueName: \"kubernetes.io/projected/0854ed41-d942-4cf3-b58f-0d396128a995-kube-api-access-585q2\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qd65d\" (UID: \"0854ed41-d942-4cf3-b58f-0d396128a995\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qd65d" Jul 19 16:52:37.248990 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:52:37.248948 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0854ed41-d942-4cf3-b58f-0d396128a995-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qd65d\" (UID: \"0854ed41-d942-4cf3-b58f-0d396128a995\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qd65d" Jul 19 16:52:37.248990 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:52:37.248990 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0854ed41-d942-4cf3-b58f-0d396128a995-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qd65d\" (UID: \"0854ed41-d942-4cf3-b58f-0d396128a995\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qd65d" Jul 19 16:52:37.249267 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:52:37.249025 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0854ed41-d942-4cf3-b58f-0d396128a995-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qd65d\" (UID: \"0854ed41-d942-4cf3-b58f-0d396128a995\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qd65d" Jul 19 16:52:37.249267 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:52:37.249145 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0854ed41-d942-4cf3-b58f-0d396128a995-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qd65d\" (UID: \"0854ed41-d942-4cf3-b58f-0d396128a995\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qd65d" Jul 19 16:52:37.249267 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:52:37.249185 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-585q2\" (UniqueName: \"kubernetes.io/projected/0854ed41-d942-4cf3-b58f-0d396128a995-kube-api-access-585q2\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qd65d\" (UID: \"0854ed41-d942-4cf3-b58f-0d396128a995\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qd65d" Jul 19 16:52:37.249267 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:52:37.249244 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0854ed41-d942-4cf3-b58f-0d396128a995-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qd65d\" (UID: \"0854ed41-d942-4cf3-b58f-0d396128a995\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qd65d" Jul 19 16:52:37.249457 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:52:37.249416 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0854ed41-d942-4cf3-b58f-0d396128a995-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qd65d\" (UID: \"0854ed41-d942-4cf3-b58f-0d396128a995\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qd65d" Jul 19 16:52:37.249457 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:52:37.249433 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0854ed41-d942-4cf3-b58f-0d396128a995-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qd65d\" (UID: \"0854ed41-d942-4cf3-b58f-0d396128a995\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qd65d" Jul 19 16:52:37.249608 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:52:37.249588 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0854ed41-d942-4cf3-b58f-0d396128a995-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qd65d\" (UID: \"0854ed41-d942-4cf3-b58f-0d396128a995\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qd65d" Jul 19 16:52:37.251427 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:52:37.251395 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0854ed41-d942-4cf3-b58f-0d396128a995-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qd65d\" (UID: \"0854ed41-d942-4cf3-b58f-0d396128a995\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qd65d" Jul 19 16:52:37.251532 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:52:37.251500 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0854ed41-d942-4cf3-b58f-0d396128a995-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qd65d\" (UID: \"0854ed41-d942-4cf3-b58f-0d396128a995\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qd65d" Jul 19 16:52:37.256603 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:52:37.256586 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-585q2\" (UniqueName: \"kubernetes.io/projected/0854ed41-d942-4cf3-b58f-0d396128a995-kube-api-access-585q2\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qd65d\" (UID: \"0854ed41-d942-4cf3-b58f-0d396128a995\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qd65d" Jul 19 16:52:37.343930 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:52:37.343898 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qd65d" Jul 19 16:52:37.472209 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:52:37.472149 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-qd65d"] Jul 19 16:52:37.474452 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:52:37.474423 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0854ed41_d942_4cf3_b58f_0d396128a995.slice/crio-0ef7925d9bd63656515da1af9db76acf51ab70fe70367770855ad455011cf31f WatchSource:0}: Error finding container 0ef7925d9bd63656515da1af9db76acf51ab70fe70367770855ad455011cf31f: Status 404 returned error can't find the container with id 0ef7925d9bd63656515da1af9db76acf51ab70fe70367770855ad455011cf31f Jul 19 16:52:37.475686 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:52:37.475668 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 19 16:52:37.952965 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:52:37.952922 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qd65d" event={"ID":"0854ed41-d942-4cf3-b58f-0d396128a995","Type":"ContainerStarted","Data":"0ef7925d9bd63656515da1af9db76acf51ab70fe70367770855ad455011cf31f"} Jul 19 16:52:38.959414 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:52:38.959382 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qd65d" event={"ID":"0854ed41-d942-4cf3-b58f-0d396128a995","Type":"ContainerStarted","Data":"a683fa3cb8a4c84df5cfb895bd628660ae888be8901e2985e4205189f32fdd46"} Jul 19 16:52:38.976186 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:52:38.976136 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qd65d" podStartSLOduration=1.047728663 podStartE2EDuration="1.976120767s" totalImagesPullingTime="928.392104ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 16:52:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 16:52:38.974833204 +0000 UTC m=+853.961408690" watchObservedRunningTime="2026-07-19 16:52:38.976120767 +0000 UTC m=+853.962696250" Jul 19 16:52:47.344842 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:52:47.344797 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qd65d" Jul 19 16:52:47.345352 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:52:47.344872 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qd65d" Jul 19 16:52:47.357995 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:52:47.357966 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qd65d" Jul 19 16:52:48.009781 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:52:48.009753 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qd65d" Jul 19 16:53:04.317979 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:53:04.317941 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzcg"] Jul 19 16:53:04.321642 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:53:04.321620 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzcg" Jul 19 16:53:04.323970 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:53:04.323946 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 19 16:53:04.333459 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:53:04.333428 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzcg"] Jul 19 16:53:04.400723 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:53:04.400672 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c43c4dbe-e471-45fe-8dd5-465b5f2fc774-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzcg\" (UID: \"c43c4dbe-e471-45fe-8dd5-465b5f2fc774\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzcg" Jul 19 16:53:04.400723 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:53:04.400720 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgjjl\" (UniqueName: \"kubernetes.io/projected/c43c4dbe-e471-45fe-8dd5-465b5f2fc774-kube-api-access-sgjjl\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzcg\" (UID: \"c43c4dbe-e471-45fe-8dd5-465b5f2fc774\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzcg" Jul 19 16:53:04.400925 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:53:04.400786 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c43c4dbe-e471-45fe-8dd5-465b5f2fc774-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzcg\" (UID: \"c43c4dbe-e471-45fe-8dd5-465b5f2fc774\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzcg" Jul 19 16:53:04.400925 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:53:04.400809 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c43c4dbe-e471-45fe-8dd5-465b5f2fc774-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzcg\" (UID: \"c43c4dbe-e471-45fe-8dd5-465b5f2fc774\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzcg" Jul 19 16:53:04.400925 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:53:04.400896 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c43c4dbe-e471-45fe-8dd5-465b5f2fc774-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzcg\" (UID: \"c43c4dbe-e471-45fe-8dd5-465b5f2fc774\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzcg" Jul 19 16:53:04.401023 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:53:04.400965 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c43c4dbe-e471-45fe-8dd5-465b5f2fc774-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzcg\" (UID: \"c43c4dbe-e471-45fe-8dd5-465b5f2fc774\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzcg" Jul 19 16:53:04.501677 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:53:04.501642 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c43c4dbe-e471-45fe-8dd5-465b5f2fc774-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzcg\" (UID: \"c43c4dbe-e471-45fe-8dd5-465b5f2fc774\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzcg" Jul 19 16:53:04.501677 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:53:04.501676 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgjjl\" (UniqueName: \"kubernetes.io/projected/c43c4dbe-e471-45fe-8dd5-465b5f2fc774-kube-api-access-sgjjl\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzcg\" (UID: \"c43c4dbe-e471-45fe-8dd5-465b5f2fc774\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzcg" Jul 19 16:53:04.501899 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:53:04.501807 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c43c4dbe-e471-45fe-8dd5-465b5f2fc774-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzcg\" (UID: \"c43c4dbe-e471-45fe-8dd5-465b5f2fc774\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzcg" Jul 19 16:53:04.501899 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:53:04.501843 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c43c4dbe-e471-45fe-8dd5-465b5f2fc774-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzcg\" (UID: \"c43c4dbe-e471-45fe-8dd5-465b5f2fc774\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzcg" Jul 19 16:53:04.501899 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:53:04.501884 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c43c4dbe-e471-45fe-8dd5-465b5f2fc774-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzcg\" (UID: \"c43c4dbe-e471-45fe-8dd5-465b5f2fc774\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzcg" Jul 19 16:53:04.502048 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:53:04.501943 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c43c4dbe-e471-45fe-8dd5-465b5f2fc774-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzcg\" (UID: \"c43c4dbe-e471-45fe-8dd5-465b5f2fc774\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzcg" Jul 19 16:53:04.502048 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:53:04.501993 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c43c4dbe-e471-45fe-8dd5-465b5f2fc774-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzcg\" (UID: \"c43c4dbe-e471-45fe-8dd5-465b5f2fc774\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzcg" Jul 19 16:53:04.502232 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:53:04.502211 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c43c4dbe-e471-45fe-8dd5-465b5f2fc774-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzcg\" (UID: \"c43c4dbe-e471-45fe-8dd5-465b5f2fc774\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzcg" Jul 19 16:53:04.502324 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:53:04.502296 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c43c4dbe-e471-45fe-8dd5-465b5f2fc774-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzcg\" (UID: \"c43c4dbe-e471-45fe-8dd5-465b5f2fc774\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzcg" Jul 19 16:53:04.504136 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:53:04.504117 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c43c4dbe-e471-45fe-8dd5-465b5f2fc774-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzcg\" (UID: \"c43c4dbe-e471-45fe-8dd5-465b5f2fc774\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzcg" Jul 19 16:53:04.504271 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:53:04.504255 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c43c4dbe-e471-45fe-8dd5-465b5f2fc774-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzcg\" (UID: \"c43c4dbe-e471-45fe-8dd5-465b5f2fc774\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzcg" Jul 19 16:53:04.509341 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:53:04.509314 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgjjl\" (UniqueName: \"kubernetes.io/projected/c43c4dbe-e471-45fe-8dd5-465b5f2fc774-kube-api-access-sgjjl\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzcg\" (UID: \"c43c4dbe-e471-45fe-8dd5-465b5f2fc774\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzcg" Jul 19 16:53:04.633515 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:53:04.633471 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzcg" Jul 19 16:53:04.770640 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:53:04.770613 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzcg"] Jul 19 16:53:04.772948 ip-10-0-134-203 kubenswrapper[2583]: W0719 16:53:04.772918 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc43c4dbe_e471_45fe_8dd5_465b5f2fc774.slice/crio-1dff63a52c270e914b5006334cb085f7a00719bee4631127103a315c7143a694 WatchSource:0}: Error finding container 1dff63a52c270e914b5006334cb085f7a00719bee4631127103a315c7143a694: Status 404 returned error can't find the container with id 1dff63a52c270e914b5006334cb085f7a00719bee4631127103a315c7143a694 Jul 19 16:53:05.085492 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:53:05.085460 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzcg" event={"ID":"c43c4dbe-e471-45fe-8dd5-465b5f2fc774","Type":"ContainerStarted","Data":"cabf85cd6c1300bfdc9a7d3e50fc26f2a50c2317c9ea45e77a9ab1ff12a77303"} Jul 19 16:53:05.085649 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:53:05.085515 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzcg" event={"ID":"c43c4dbe-e471-45fe-8dd5-465b5f2fc774","Type":"ContainerStarted","Data":"1dff63a52c270e914b5006334cb085f7a00719bee4631127103a315c7143a694"} Jul 19 16:53:05.102588 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:53:05.102536 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzcg" podStartSLOduration=0.929406963 podStartE2EDuration="1.102519904s" totalImagesPullingTime="173.112941ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 16:53:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 16:53:05.101422414 +0000 UTC m=+880.087997902" watchObservedRunningTime="2026-07-19 16:53:05.102519904 +0000 UTC m=+880.089095388" Jul 19 16:53:14.634554 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:53:14.634519 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzcg" Jul 19 16:53:14.634991 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:53:14.634579 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzcg" Jul 19 16:53:14.647459 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:53:14.647432 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzcg" Jul 19 16:53:15.144172 ip-10-0-134-203 kubenswrapper[2583]: I0719 16:53:15.144145 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzcg" Jul 19 17:09:50.445848 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:09:50.445762 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6"] Jul 19 17:09:50.449253 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:09:50.449233 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6" Jul 19 17:09:50.451425 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:09:50.451400 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-debb71c1-openshift-default-dockercfg-2zzqd\"" Jul 19 17:09:50.467778 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:09:50.467744 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6"] Jul 19 17:09:50.573889 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:09:50.573840 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4bbb120-ff91-4a9b-9cee-926912ae6eac-istio-envoy\") pod \"e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6\" (UID: \"f4bbb120-ff91-4a9b-9cee-926912ae6eac\") " pod="openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6" Jul 19 17:09:50.573889 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:09:50.573887 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4bbb120-ff91-4a9b-9cee-926912ae6eac-istio-token\") pod \"e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6\" (UID: \"f4bbb120-ff91-4a9b-9cee-926912ae6eac\") " pod="openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6" Jul 19 17:09:50.574163 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:09:50.573908 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fg7q4\" (UniqueName: \"kubernetes.io/projected/f4bbb120-ff91-4a9b-9cee-926912ae6eac-kube-api-access-fg7q4\") pod \"e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6\" (UID: \"f4bbb120-ff91-4a9b-9cee-926912ae6eac\") " pod="openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6" Jul 19 17:09:50.574163 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:09:50.574009 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4bbb120-ff91-4a9b-9cee-926912ae6eac-credential-socket\") pod \"e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6\" (UID: \"f4bbb120-ff91-4a9b-9cee-926912ae6eac\") " pod="openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6" Jul 19 17:09:50.574163 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:09:50.574104 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4bbb120-ff91-4a9b-9cee-926912ae6eac-workload-socket\") pod \"e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6\" (UID: \"f4bbb120-ff91-4a9b-9cee-926912ae6eac\") " pod="openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6" Jul 19 17:09:50.574163 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:09:50.574127 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4bbb120-ff91-4a9b-9cee-926912ae6eac-workload-certs\") pod \"e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6\" (UID: \"f4bbb120-ff91-4a9b-9cee-926912ae6eac\") " pod="openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6" Jul 19 17:09:50.574163 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:09:50.574157 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4bbb120-ff91-4a9b-9cee-926912ae6eac-istiod-ca-cert\") pod \"e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6\" (UID: \"f4bbb120-ff91-4a9b-9cee-926912ae6eac\") " pod="openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6" Jul 19 17:09:50.574318 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:09:50.574227 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4bbb120-ff91-4a9b-9cee-926912ae6eac-istio-podinfo\") pod \"e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6\" (UID: \"f4bbb120-ff91-4a9b-9cee-926912ae6eac\") " pod="openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6" Jul 19 17:09:50.574318 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:09:50.574267 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4bbb120-ff91-4a9b-9cee-926912ae6eac-istio-data\") pod \"e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6\" (UID: \"f4bbb120-ff91-4a9b-9cee-926912ae6eac\") " pod="openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6" Jul 19 17:09:50.675677 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:09:50.675640 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4bbb120-ff91-4a9b-9cee-926912ae6eac-credential-socket\") pod \"e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6\" (UID: \"f4bbb120-ff91-4a9b-9cee-926912ae6eac\") " pod="openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6" Jul 19 17:09:50.675869 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:09:50.675701 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4bbb120-ff91-4a9b-9cee-926912ae6eac-workload-socket\") pod \"e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6\" (UID: \"f4bbb120-ff91-4a9b-9cee-926912ae6eac\") " pod="openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6" Jul 19 17:09:50.675869 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:09:50.675812 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4bbb120-ff91-4a9b-9cee-926912ae6eac-workload-certs\") pod \"e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6\" (UID: \"f4bbb120-ff91-4a9b-9cee-926912ae6eac\") " pod="openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6" Jul 19 17:09:50.675869 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:09:50.675861 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4bbb120-ff91-4a9b-9cee-926912ae6eac-istiod-ca-cert\") pod \"e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6\" (UID: \"f4bbb120-ff91-4a9b-9cee-926912ae6eac\") " pod="openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6" Jul 19 17:09:50.676031 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:09:50.675886 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4bbb120-ff91-4a9b-9cee-926912ae6eac-istio-podinfo\") pod \"e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6\" (UID: \"f4bbb120-ff91-4a9b-9cee-926912ae6eac\") " pod="openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6" Jul 19 17:09:50.676127 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:09:50.676027 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4bbb120-ff91-4a9b-9cee-926912ae6eac-istio-data\") pod \"e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6\" (UID: \"f4bbb120-ff91-4a9b-9cee-926912ae6eac\") " pod="openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6" Jul 19 17:09:50.676198 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:09:50.676124 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4bbb120-ff91-4a9b-9cee-926912ae6eac-credential-socket\") pod \"e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6\" (UID: \"f4bbb120-ff91-4a9b-9cee-926912ae6eac\") " pod="openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6" Jul 19 17:09:50.676198 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:09:50.676135 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4bbb120-ff91-4a9b-9cee-926912ae6eac-istio-envoy\") pod \"e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6\" (UID: \"f4bbb120-ff91-4a9b-9cee-926912ae6eac\") " pod="openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6" Jul 19 17:09:50.676198 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:09:50.676179 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4bbb120-ff91-4a9b-9cee-926912ae6eac-istio-token\") pod \"e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6\" (UID: \"f4bbb120-ff91-4a9b-9cee-926912ae6eac\") " pod="openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6" Jul 19 17:09:50.676198 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:09:50.676184 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4bbb120-ff91-4a9b-9cee-926912ae6eac-workload-socket\") pod \"e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6\" (UID: \"f4bbb120-ff91-4a9b-9cee-926912ae6eac\") " pod="openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6" Jul 19 17:09:50.676405 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:09:50.676219 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fg7q4\" (UniqueName: \"kubernetes.io/projected/f4bbb120-ff91-4a9b-9cee-926912ae6eac-kube-api-access-fg7q4\") pod \"e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6\" (UID: \"f4bbb120-ff91-4a9b-9cee-926912ae6eac\") " pod="openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6" Jul 19 17:09:50.676405 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:09:50.676225 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4bbb120-ff91-4a9b-9cee-926912ae6eac-workload-certs\") pod \"e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6\" (UID: \"f4bbb120-ff91-4a9b-9cee-926912ae6eac\") " pod="openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6" Jul 19 17:09:50.676514 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:09:50.676398 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4bbb120-ff91-4a9b-9cee-926912ae6eac-istio-data\") pod \"e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6\" (UID: \"f4bbb120-ff91-4a9b-9cee-926912ae6eac\") " pod="openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6" Jul 19 17:09:50.676928 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:09:50.676903 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4bbb120-ff91-4a9b-9cee-926912ae6eac-istiod-ca-cert\") pod \"e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6\" (UID: \"f4bbb120-ff91-4a9b-9cee-926912ae6eac\") " pod="openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6" Jul 19 17:09:50.678371 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:09:50.678350 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4bbb120-ff91-4a9b-9cee-926912ae6eac-istio-envoy\") pod \"e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6\" (UID: \"f4bbb120-ff91-4a9b-9cee-926912ae6eac\") " pod="openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6" Jul 19 17:09:50.678574 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:09:50.678558 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4bbb120-ff91-4a9b-9cee-926912ae6eac-istio-podinfo\") pod \"e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6\" (UID: \"f4bbb120-ff91-4a9b-9cee-926912ae6eac\") " pod="openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6" Jul 19 17:09:50.683109 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:09:50.683043 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4bbb120-ff91-4a9b-9cee-926912ae6eac-istio-token\") pod \"e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6\" (UID: \"f4bbb120-ff91-4a9b-9cee-926912ae6eac\") " pod="openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6" Jul 19 17:09:50.683336 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:09:50.683317 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fg7q4\" (UniqueName: \"kubernetes.io/projected/f4bbb120-ff91-4a9b-9cee-926912ae6eac-kube-api-access-fg7q4\") pod \"e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6\" (UID: \"f4bbb120-ff91-4a9b-9cee-926912ae6eac\") " pod="openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6" Jul 19 17:09:50.759412 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:09:50.759317 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6" Jul 19 17:09:50.905357 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:09:50.905330 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6"] Jul 19 17:09:50.907987 ip-10-0-134-203 kubenswrapper[2583]: W0719 17:09:50.907959 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4bbb120_ff91_4a9b_9cee_926912ae6eac.slice/crio-7708b9b384d62ced4167c267d96f24e0383392b1d4c499842e771204889edb42 WatchSource:0}: Error finding container 7708b9b384d62ced4167c267d96f24e0383392b1d4c499842e771204889edb42: Status 404 returned error can't find the container with id 7708b9b384d62ced4167c267d96f24e0383392b1d4c499842e771204889edb42 Jul 19 17:09:50.910150 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:09:50.910111 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:09:50.910247 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:09:50.910189 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:09:50.910247 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:09:50.910219 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:09:51.704551 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:09:51.704511 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6" event={"ID":"f4bbb120-ff91-4a9b-9cee-926912ae6eac","Type":"ContainerStarted","Data":"4340d75b060a4e2a28bdc29444e233f5cd51b51a47ba1481239af39ebe4b06e6"} Jul 19 17:09:51.704966 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:09:51.704566 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6" event={"ID":"f4bbb120-ff91-4a9b-9cee-926912ae6eac","Type":"ContainerStarted","Data":"7708b9b384d62ced4167c267d96f24e0383392b1d4c499842e771204889edb42"} Jul 19 17:09:51.722955 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:09:51.722896 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6" podStartSLOduration=1.7228790680000001 podStartE2EDuration="1.722879068s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 17:09:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 17:09:51.72156109 +0000 UTC m=+1886.708136575" watchObservedRunningTime="2026-07-19 17:09:51.722879068 +0000 UTC m=+1886.709454551" Jul 19 17:09:51.760479 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:09:51.760448 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6" Jul 19 17:09:51.765366 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:09:51.765337 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6" Jul 19 17:09:52.710160 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:09:52.710126 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6" Jul 19 17:09:52.711093 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:09:52.711045 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6" Jul 19 17:10:08.752269 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:08.752220 2583 patch_prober.go:28] interesting pod/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 19 17:10:08.754571 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:08.752300 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6" podUID="f4bbb120-ff91-4a9b-9cee-926912ae6eac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 19 17:10:09.791679 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:09.791588 2583 generic.go:356] "Generic (PLEG): container finished" podID="f4bbb120-ff91-4a9b-9cee-926912ae6eac" containerID="4340d75b060a4e2a28bdc29444e233f5cd51b51a47ba1481239af39ebe4b06e6" exitCode=0 Jul 19 17:10:09.791679 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:09.791661 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6" event={"ID":"f4bbb120-ff91-4a9b-9cee-926912ae6eac","Type":"ContainerDied","Data":"4340d75b060a4e2a28bdc29444e233f5cd51b51a47ba1481239af39ebe4b06e6"} Jul 19 17:10:09.792145 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:09.792100 2583 scope.go:117] "RemoveContainer" containerID="4340d75b060a4e2a28bdc29444e233f5cd51b51a47ba1481239af39ebe4b06e6" Jul 19 17:10:09.793143 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:09.793103 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:10:09.793268 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:09.793221 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:10:09.793268 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:09.793256 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:10:10.799672 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:10.799638 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6" event={"ID":"f4bbb120-ff91-4a9b-9cee-926912ae6eac","Type":"ContainerStarted","Data":"c7e0d59c72fe1d8d696dc4cf51d4de49d63fe3287ead834814ed97ee8597d355"} Jul 19 17:10:11.759625 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:11.759591 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6" Jul 19 17:10:11.759827 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:11.759774 2583 patch_prober.go:28] interesting pod/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body= Jul 19 17:10:11.759906 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:11.759830 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6" podUID="f4bbb120-ff91-4a9b-9cee-926912ae6eac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" Jul 19 17:10:12.759758 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:12.759717 2583 patch_prober.go:28] interesting pod/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body= Jul 19 17:10:12.760178 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:12.759774 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6" podUID="f4bbb120-ff91-4a9b-9cee-926912ae6eac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" Jul 19 17:10:13.760259 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:13.760219 2583 patch_prober.go:28] interesting pod/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body= Jul 19 17:10:13.760695 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:13.760280 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6" podUID="f4bbb120-ff91-4a9b-9cee-926912ae6eac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" Jul 19 17:10:14.760238 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:14.760198 2583 patch_prober.go:28] interesting pod/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body= Jul 19 17:10:14.760432 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:14.760274 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6" podUID="f4bbb120-ff91-4a9b-9cee-926912ae6eac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" Jul 19 17:10:15.764013 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:15.763979 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6" Jul 19 17:10:15.764419 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:15.764299 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6" Jul 19 17:10:15.765159 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:15.765131 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6" Jul 19 17:10:33.810665 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:33.810627 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6"] Jul 19 17:10:33.811174 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:33.810850 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6" podUID="f4bbb120-ff91-4a9b-9cee-926912ae6eac" containerName="istio-proxy" containerID="cri-o://c7e0d59c72fe1d8d696dc4cf51d4de49d63fe3287ead834814ed97ee8597d355" gracePeriod=30 Jul 19 17:10:34.815294 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:34.815261 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr"] Jul 19 17:10:34.818140 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:34.818119 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr" Jul 19 17:10:34.820344 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:34.820319 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-5f384e8f-gw-openshift-default-dockercfg-xhqcj\"" Jul 19 17:10:34.830324 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:34.830293 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr"] Jul 19 17:10:34.896202 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:34.896166 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db1eda5e-b48f-446b-b90f-886b9259439f-istio-podinfo\") pod \"e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr\" (UID: \"db1eda5e-b48f-446b-b90f-886b9259439f\") " pod="openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr" Jul 19 17:10:34.896202 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:34.896205 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db1eda5e-b48f-446b-b90f-886b9259439f-istio-envoy\") pod \"e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr\" (UID: \"db1eda5e-b48f-446b-b90f-886b9259439f\") " pod="openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr" Jul 19 17:10:34.896444 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:34.896304 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db1eda5e-b48f-446b-b90f-886b9259439f-istio-token\") pod \"e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr\" (UID: \"db1eda5e-b48f-446b-b90f-886b9259439f\") " pod="openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr" Jul 19 17:10:34.896444 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:34.896358 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db1eda5e-b48f-446b-b90f-886b9259439f-credential-socket\") pod \"e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr\" (UID: \"db1eda5e-b48f-446b-b90f-886b9259439f\") " pod="openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr" Jul 19 17:10:34.896444 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:34.896385 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db1eda5e-b48f-446b-b90f-886b9259439f-workload-certs\") pod \"e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr\" (UID: \"db1eda5e-b48f-446b-b90f-886b9259439f\") " pod="openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr" Jul 19 17:10:34.896444 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:34.896409 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s257n\" (UniqueName: \"kubernetes.io/projected/db1eda5e-b48f-446b-b90f-886b9259439f-kube-api-access-s257n\") pod \"e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr\" (UID: \"db1eda5e-b48f-446b-b90f-886b9259439f\") " pod="openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr" Jul 19 17:10:34.896625 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:34.896448 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db1eda5e-b48f-446b-b90f-886b9259439f-istiod-ca-cert\") pod \"e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr\" (UID: \"db1eda5e-b48f-446b-b90f-886b9259439f\") " pod="openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr" Jul 19 17:10:34.896625 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:34.896497 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db1eda5e-b48f-446b-b90f-886b9259439f-workload-socket\") pod \"e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr\" (UID: \"db1eda5e-b48f-446b-b90f-886b9259439f\") " pod="openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr" Jul 19 17:10:34.896625 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:34.896516 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db1eda5e-b48f-446b-b90f-886b9259439f-istio-data\") pod \"e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr\" (UID: \"db1eda5e-b48f-446b-b90f-886b9259439f\") " pod="openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr" Jul 19 17:10:34.997608 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:34.997569 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db1eda5e-b48f-446b-b90f-886b9259439f-istio-podinfo\") pod \"e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr\" (UID: \"db1eda5e-b48f-446b-b90f-886b9259439f\") " pod="openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr" Jul 19 17:10:34.997608 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:34.997621 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db1eda5e-b48f-446b-b90f-886b9259439f-istio-envoy\") pod \"e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr\" (UID: \"db1eda5e-b48f-446b-b90f-886b9259439f\") " pod="openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr" Jul 19 17:10:34.997893 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:34.997677 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db1eda5e-b48f-446b-b90f-886b9259439f-istio-token\") pod \"e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr\" (UID: \"db1eda5e-b48f-446b-b90f-886b9259439f\") " pod="openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr" Jul 19 17:10:34.997893 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:34.997718 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db1eda5e-b48f-446b-b90f-886b9259439f-credential-socket\") pod \"e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr\" (UID: \"db1eda5e-b48f-446b-b90f-886b9259439f\") " pod="openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr" Jul 19 17:10:34.997893 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:34.997768 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db1eda5e-b48f-446b-b90f-886b9259439f-workload-certs\") pod \"e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr\" (UID: \"db1eda5e-b48f-446b-b90f-886b9259439f\") " pod="openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr" Jul 19 17:10:34.997893 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:34.997810 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s257n\" (UniqueName: \"kubernetes.io/projected/db1eda5e-b48f-446b-b90f-886b9259439f-kube-api-access-s257n\") pod \"e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr\" (UID: \"db1eda5e-b48f-446b-b90f-886b9259439f\") " pod="openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr" Jul 19 17:10:34.997893 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:34.997841 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db1eda5e-b48f-446b-b90f-886b9259439f-istiod-ca-cert\") pod \"e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr\" (UID: \"db1eda5e-b48f-446b-b90f-886b9259439f\") " pod="openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr" Jul 19 17:10:34.997893 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:34.997877 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db1eda5e-b48f-446b-b90f-886b9259439f-workload-socket\") pod \"e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr\" (UID: \"db1eda5e-b48f-446b-b90f-886b9259439f\") " pod="openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr" Jul 19 17:10:34.998236 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:34.997904 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db1eda5e-b48f-446b-b90f-886b9259439f-istio-data\") pod \"e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr\" (UID: \"db1eda5e-b48f-446b-b90f-886b9259439f\") " pod="openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr" Jul 19 17:10:34.998290 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:34.998233 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db1eda5e-b48f-446b-b90f-886b9259439f-credential-socket\") pod \"e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr\" (UID: \"db1eda5e-b48f-446b-b90f-886b9259439f\") " pod="openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr" Jul 19 17:10:34.998355 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:34.998280 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db1eda5e-b48f-446b-b90f-886b9259439f-workload-certs\") pod \"e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr\" (UID: \"db1eda5e-b48f-446b-b90f-886b9259439f\") " pod="openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr" Jul 19 17:10:34.998442 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:34.998418 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db1eda5e-b48f-446b-b90f-886b9259439f-istio-data\") pod \"e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr\" (UID: \"db1eda5e-b48f-446b-b90f-886b9259439f\") " pod="openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr" Jul 19 17:10:34.998494 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:34.998468 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db1eda5e-b48f-446b-b90f-886b9259439f-workload-socket\") pod \"e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr\" (UID: \"db1eda5e-b48f-446b-b90f-886b9259439f\") " pod="openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr" Jul 19 17:10:34.998772 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:34.998749 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db1eda5e-b48f-446b-b90f-886b9259439f-istiod-ca-cert\") pod \"e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr\" (UID: \"db1eda5e-b48f-446b-b90f-886b9259439f\") " pod="openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr" Jul 19 17:10:35.000107 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:35.000089 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db1eda5e-b48f-446b-b90f-886b9259439f-istio-envoy\") pod \"e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr\" (UID: \"db1eda5e-b48f-446b-b90f-886b9259439f\") " pod="openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr" Jul 19 17:10:35.000251 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:35.000236 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db1eda5e-b48f-446b-b90f-886b9259439f-istio-podinfo\") pod \"e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr\" (UID: \"db1eda5e-b48f-446b-b90f-886b9259439f\") " pod="openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr" Jul 19 17:10:35.005459 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:35.005436 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db1eda5e-b48f-446b-b90f-886b9259439f-istio-token\") pod \"e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr\" (UID: \"db1eda5e-b48f-446b-b90f-886b9259439f\") " pod="openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr" Jul 19 17:10:35.005664 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:35.005646 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s257n\" (UniqueName: \"kubernetes.io/projected/db1eda5e-b48f-446b-b90f-886b9259439f-kube-api-access-s257n\") pod \"e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr\" (UID: \"db1eda5e-b48f-446b-b90f-886b9259439f\") " pod="openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr" Jul 19 17:10:35.129517 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:35.129480 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr" Jul 19 17:10:35.274523 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:35.274481 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr"] Jul 19 17:10:35.277281 ip-10-0-134-203 kubenswrapper[2583]: W0719 17:10:35.277250 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb1eda5e_b48f_446b_b90f_886b9259439f.slice/crio-3e1c263e62a8ab621f7ceeec0e5e8eef35ddea25abb2a1f94cd696381ad414ca WatchSource:0}: Error finding container 3e1c263e62a8ab621f7ceeec0e5e8eef35ddea25abb2a1f94cd696381ad414ca: Status 404 returned error can't find the container with id 3e1c263e62a8ab621f7ceeec0e5e8eef35ddea25abb2a1f94cd696381ad414ca Jul 19 17:10:35.279968 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:35.279934 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:10:35.280027 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:35.280004 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:10:35.280102 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:35.280031 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:10:35.927800 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:35.927765 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr" event={"ID":"db1eda5e-b48f-446b-b90f-886b9259439f","Type":"ContainerStarted","Data":"eede9aa39d05ff5bb31410d83ccb76998a8100196ee06d81368035aabb997a15"} Jul 19 17:10:35.928267 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:35.927816 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr" event={"ID":"db1eda5e-b48f-446b-b90f-886b9259439f","Type":"ContainerStarted","Data":"3e1c263e62a8ab621f7ceeec0e5e8eef35ddea25abb2a1f94cd696381ad414ca"} Jul 19 17:10:35.946709 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:35.946652 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr" podStartSLOduration=1.946634389 podStartE2EDuration="1.946634389s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 17:10:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 17:10:35.945193046 +0000 UTC m=+1930.931768556" watchObservedRunningTime="2026-07-19 17:10:35.946634389 +0000 UTC m=+1930.933209881" Jul 19 17:10:36.130481 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:36.130440 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr" Jul 19 17:10:36.132036 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:36.132012 2583 patch_prober.go:28] interesting pod/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body= Jul 19 17:10:36.132105 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:36.132074 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr" podUID="db1eda5e-b48f-446b-b90f-886b9259439f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" Jul 19 17:10:37.135115 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:37.135047 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr" Jul 19 17:10:37.937936 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:37.937903 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr" Jul 19 17:10:37.938871 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:37.938850 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr" Jul 19 17:10:38.944217 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:38.944181 2583 generic.go:356] "Generic (PLEG): container finished" podID="f4bbb120-ff91-4a9b-9cee-926912ae6eac" containerID="c7e0d59c72fe1d8d696dc4cf51d4de49d63fe3287ead834814ed97ee8597d355" exitCode=0 Jul 19 17:10:38.944217 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:38.944215 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6" event={"ID":"f4bbb120-ff91-4a9b-9cee-926912ae6eac","Type":"ContainerDied","Data":"c7e0d59c72fe1d8d696dc4cf51d4de49d63fe3287ead834814ed97ee8597d355"} Jul 19 17:10:38.944715 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:38.944293 2583 scope.go:117] "RemoveContainer" containerID="4340d75b060a4e2a28bdc29444e233f5cd51b51a47ba1481239af39ebe4b06e6" Jul 19 17:10:39.072113 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:39.072027 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6" Jul 19 17:10:39.242625 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:39.242591 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4bbb120-ff91-4a9b-9cee-926912ae6eac-istiod-ca-cert\") pod \"f4bbb120-ff91-4a9b-9cee-926912ae6eac\" (UID: \"f4bbb120-ff91-4a9b-9cee-926912ae6eac\") " Jul 19 17:10:39.242625 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:39.242633 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4bbb120-ff91-4a9b-9cee-926912ae6eac-istio-envoy\") pod \"f4bbb120-ff91-4a9b-9cee-926912ae6eac\" (UID: \"f4bbb120-ff91-4a9b-9cee-926912ae6eac\") " Jul 19 17:10:39.242888 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:39.242654 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4bbb120-ff91-4a9b-9cee-926912ae6eac-workload-certs\") pod \"f4bbb120-ff91-4a9b-9cee-926912ae6eac\" (UID: \"f4bbb120-ff91-4a9b-9cee-926912ae6eac\") " Jul 19 17:10:39.242888 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:39.242673 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4bbb120-ff91-4a9b-9cee-926912ae6eac-istio-data\") pod \"f4bbb120-ff91-4a9b-9cee-926912ae6eac\" (UID: \"f4bbb120-ff91-4a9b-9cee-926912ae6eac\") " Jul 19 17:10:39.242888 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:39.242695 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4bbb120-ff91-4a9b-9cee-926912ae6eac-workload-socket\") pod \"f4bbb120-ff91-4a9b-9cee-926912ae6eac\" (UID: \"f4bbb120-ff91-4a9b-9cee-926912ae6eac\") " Jul 19 17:10:39.242888 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:39.242760 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4bbb120-ff91-4a9b-9cee-926912ae6eac-istio-token\") pod \"f4bbb120-ff91-4a9b-9cee-926912ae6eac\" (UID: \"f4bbb120-ff91-4a9b-9cee-926912ae6eac\") " Jul 19 17:10:39.242888 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:39.242854 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4bbb120-ff91-4a9b-9cee-926912ae6eac-istio-podinfo\") pod \"f4bbb120-ff91-4a9b-9cee-926912ae6eac\" (UID: \"f4bbb120-ff91-4a9b-9cee-926912ae6eac\") " Jul 19 17:10:39.242888 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:39.242887 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fg7q4\" (UniqueName: \"kubernetes.io/projected/f4bbb120-ff91-4a9b-9cee-926912ae6eac-kube-api-access-fg7q4\") pod \"f4bbb120-ff91-4a9b-9cee-926912ae6eac\" (UID: \"f4bbb120-ff91-4a9b-9cee-926912ae6eac\") " Jul 19 17:10:39.243236 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:39.242910 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4bbb120-ff91-4a9b-9cee-926912ae6eac-credential-socket\") pod \"f4bbb120-ff91-4a9b-9cee-926912ae6eac\" (UID: \"f4bbb120-ff91-4a9b-9cee-926912ae6eac\") " Jul 19 17:10:39.243236 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:39.242978 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4bbb120-ff91-4a9b-9cee-926912ae6eac-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f4bbb120-ff91-4a9b-9cee-926912ae6eac" (UID: "f4bbb120-ff91-4a9b-9cee-926912ae6eac"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:10:39.243236 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:39.243173 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4bbb120-ff91-4a9b-9cee-926912ae6eac-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f4bbb120-ff91-4a9b-9cee-926912ae6eac" (UID: "f4bbb120-ff91-4a9b-9cee-926912ae6eac"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 19 17:10:39.243379 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:39.243311 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4bbb120-ff91-4a9b-9cee-926912ae6eac-workload-certs\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:10:39.243379 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:39.243329 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4bbb120-ff91-4a9b-9cee-926912ae6eac-istiod-ca-cert\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:10:39.243484 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:39.243431 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4bbb120-ff91-4a9b-9cee-926912ae6eac-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f4bbb120-ff91-4a9b-9cee-926912ae6eac" (UID: "f4bbb120-ff91-4a9b-9cee-926912ae6eac"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:10:39.243611 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:39.243568 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4bbb120-ff91-4a9b-9cee-926912ae6eac-istio-data" (OuterVolumeSpecName: "istio-data") pod "f4bbb120-ff91-4a9b-9cee-926912ae6eac" (UID: "f4bbb120-ff91-4a9b-9cee-926912ae6eac"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:10:39.243717 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:39.243631 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4bbb120-ff91-4a9b-9cee-926912ae6eac-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f4bbb120-ff91-4a9b-9cee-926912ae6eac" (UID: "f4bbb120-ff91-4a9b-9cee-926912ae6eac"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:10:39.245199 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:39.245173 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4bbb120-ff91-4a9b-9cee-926912ae6eac-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f4bbb120-ff91-4a9b-9cee-926912ae6eac" (UID: "f4bbb120-ff91-4a9b-9cee-926912ae6eac"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:10:39.245280 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:39.245247 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4bbb120-ff91-4a9b-9cee-926912ae6eac-kube-api-access-fg7q4" (OuterVolumeSpecName: "kube-api-access-fg7q4") pod "f4bbb120-ff91-4a9b-9cee-926912ae6eac" (UID: "f4bbb120-ff91-4a9b-9cee-926912ae6eac"). InnerVolumeSpecName "kube-api-access-fg7q4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 19 17:10:39.245433 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:39.245411 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4bbb120-ff91-4a9b-9cee-926912ae6eac-istio-token" (OuterVolumeSpecName: "istio-token") pod "f4bbb120-ff91-4a9b-9cee-926912ae6eac" (UID: "f4bbb120-ff91-4a9b-9cee-926912ae6eac"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 19 17:10:39.245501 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:39.245427 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f4bbb120-ff91-4a9b-9cee-926912ae6eac-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f4bbb120-ff91-4a9b-9cee-926912ae6eac" (UID: "f4bbb120-ff91-4a9b-9cee-926912ae6eac"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 19 17:10:39.345011 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:39.344966 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4bbb120-ff91-4a9b-9cee-926912ae6eac-istio-token\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:10:39.345011 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:39.345005 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4bbb120-ff91-4a9b-9cee-926912ae6eac-istio-podinfo\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:10:39.345011 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:39.345015 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fg7q4\" (UniqueName: \"kubernetes.io/projected/f4bbb120-ff91-4a9b-9cee-926912ae6eac-kube-api-access-fg7q4\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:10:39.345332 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:39.345027 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4bbb120-ff91-4a9b-9cee-926912ae6eac-credential-socket\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:10:39.345332 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:39.345036 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4bbb120-ff91-4a9b-9cee-926912ae6eac-istio-envoy\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:10:39.345332 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:39.345045 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4bbb120-ff91-4a9b-9cee-926912ae6eac-istio-data\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:10:39.345332 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:39.345052 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4bbb120-ff91-4a9b-9cee-926912ae6eac-workload-socket\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:10:39.950268 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:39.950176 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6" event={"ID":"f4bbb120-ff91-4a9b-9cee-926912ae6eac","Type":"ContainerDied","Data":"7708b9b384d62ced4167c267d96f24e0383392b1d4c499842e771204889edb42"} Jul 19 17:10:39.950268 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:39.950241 2583 scope.go:117] "RemoveContainer" containerID="c7e0d59c72fe1d8d696dc4cf51d4de49d63fe3287ead834814ed97ee8597d355" Jul 19 17:10:39.950808 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:39.950197 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6" Jul 19 17:10:39.966729 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:39.966692 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6"] Jul 19 17:10:39.970354 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:39.970319 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-debb71c1-openshift-default-865bd9bbcc-7tqw6"] Jul 19 17:10:41.601424 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:41.601387 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4bbb120-ff91-4a9b-9cee-926912ae6eac" path="/var/lib/kubelet/pods/f4bbb120-ff91-4a9b-9cee-926912ae6eac/volumes" Jul 19 17:10:52.009469 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:52.009430 2583 generic.go:356] "Generic (PLEG): container finished" podID="db1eda5e-b48f-446b-b90f-886b9259439f" containerID="eede9aa39d05ff5bb31410d83ccb76998a8100196ee06d81368035aabb997a15" exitCode=0 Jul 19 17:10:52.010018 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:52.009492 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr" event={"ID":"db1eda5e-b48f-446b-b90f-886b9259439f","Type":"ContainerDied","Data":"eede9aa39d05ff5bb31410d83ccb76998a8100196ee06d81368035aabb997a15"} Jul 19 17:10:52.010018 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:52.010000 2583 scope.go:117] "RemoveContainer" containerID="eede9aa39d05ff5bb31410d83ccb76998a8100196ee06d81368035aabb997a15" Jul 19 17:10:52.011093 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:52.011045 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:10:52.011196 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:52.011138 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:10:52.011196 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:52.011171 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:10:52.938297 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:52.938240 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr" Jul 19 17:10:53.016329 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:53.016294 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr" event={"ID":"db1eda5e-b48f-446b-b90f-886b9259439f","Type":"ContainerStarted","Data":"98d18f25a116693fadc7d8e27808a08c06b2a9cf6f35c71c5b266aab376ce666"} Jul 19 17:10:53.130482 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:53.130436 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr" Jul 19 17:10:53.132421 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:53.131931 2583 patch_prober.go:28] interesting pod/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body= Jul 19 17:10:53.132421 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:53.131992 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr" podUID="db1eda5e-b48f-446b-b90f-886b9259439f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" Jul 19 17:10:54.130457 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:54.130408 2583 patch_prober.go:28] interesting pod/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body= Jul 19 17:10:54.130998 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:54.130489 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr" podUID="db1eda5e-b48f-446b-b90f-886b9259439f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" Jul 19 17:10:55.130314 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:55.130273 2583 patch_prober.go:28] interesting pod/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body= Jul 19 17:10:55.130613 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:55.130579 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr" podUID="db1eda5e-b48f-446b-b90f-886b9259439f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" Jul 19 17:10:56.130482 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:56.130438 2583 patch_prober.go:28] interesting pod/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body= Jul 19 17:10:56.130813 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:56.130771 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr" podUID="db1eda5e-b48f-446b-b90f-886b9259439f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" Jul 19 17:10:57.131545 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:57.131503 2583 patch_prober.go:28] interesting pod/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body= Jul 19 17:10:57.132160 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:57.132126 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr" podUID="db1eda5e-b48f-446b-b90f-886b9259439f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" Jul 19 17:10:58.134072 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:58.134028 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr" Jul 19 17:10:58.134517 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:58.134355 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr" Jul 19 17:10:58.135037 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:10:58.135019 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr" Jul 19 17:11:17.629465 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:17.629363 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr"] Jul 19 17:11:17.630213 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:17.629632 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr" podUID="db1eda5e-b48f-446b-b90f-886b9259439f" containerName="istio-proxy" containerID="cri-o://98d18f25a116693fadc7d8e27808a08c06b2a9cf6f35c71c5b266aab376ce666" gracePeriod=30 Jul 19 17:11:18.440585 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:18.440540 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp"] Jul 19 17:11:18.440936 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:18.440920 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4bbb120-ff91-4a9b-9cee-926912ae6eac" containerName="istio-proxy" Jul 19 17:11:18.440936 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:18.440936 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4bbb120-ff91-4a9b-9cee-926912ae6eac" containerName="istio-proxy" Jul 19 17:11:18.441043 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:18.440946 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4bbb120-ff91-4a9b-9cee-926912ae6eac" containerName="istio-proxy" Jul 19 17:11:18.441043 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:18.440951 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4bbb120-ff91-4a9b-9cee-926912ae6eac" containerName="istio-proxy" Jul 19 17:11:18.441043 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:18.441023 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4bbb120-ff91-4a9b-9cee-926912ae6eac" containerName="istio-proxy" Jul 19 17:11:18.441043 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:18.441031 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4bbb120-ff91-4a9b-9cee-926912ae6eac" containerName="istio-proxy" Jul 19 17:11:18.443329 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:18.443313 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp" Jul 19 17:11:18.445484 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:18.445460 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-77bade34-openshift-default-dockercfg-wcdk4\"" Jul 19 17:11:18.453929 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:18.453901 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp"] Jul 19 17:11:18.542321 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:18.542283 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmtv7\" (UniqueName: \"kubernetes.io/projected/4cf296b2-f860-4181-a609-3d202761fef8-kube-api-access-xmtv7\") pod \"e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp\" (UID: \"4cf296b2-f860-4181-a609-3d202761fef8\") " pod="openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp" Jul 19 17:11:18.542527 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:18.542358 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4cf296b2-f860-4181-a609-3d202761fef8-istio-token\") pod \"e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp\" (UID: \"4cf296b2-f860-4181-a609-3d202761fef8\") " pod="openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp" Jul 19 17:11:18.542527 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:18.542396 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4cf296b2-f860-4181-a609-3d202761fef8-credential-socket\") pod \"e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp\" (UID: \"4cf296b2-f860-4181-a609-3d202761fef8\") " pod="openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp" Jul 19 17:11:18.542527 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:18.542417 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4cf296b2-f860-4181-a609-3d202761fef8-istio-podinfo\") pod \"e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp\" (UID: \"4cf296b2-f860-4181-a609-3d202761fef8\") " pod="openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp" Jul 19 17:11:18.542527 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:18.542452 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4cf296b2-f860-4181-a609-3d202761fef8-workload-certs\") pod \"e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp\" (UID: \"4cf296b2-f860-4181-a609-3d202761fef8\") " pod="openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp" Jul 19 17:11:18.542527 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:18.542470 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4cf296b2-f860-4181-a609-3d202761fef8-istio-envoy\") pod \"e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp\" (UID: \"4cf296b2-f860-4181-a609-3d202761fef8\") " pod="openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp" Jul 19 17:11:18.542527 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:18.542493 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4cf296b2-f860-4181-a609-3d202761fef8-workload-socket\") pod \"e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp\" (UID: \"4cf296b2-f860-4181-a609-3d202761fef8\") " pod="openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp" Jul 19 17:11:18.542527 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:18.542516 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4cf296b2-f860-4181-a609-3d202761fef8-istiod-ca-cert\") pod \"e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp\" (UID: \"4cf296b2-f860-4181-a609-3d202761fef8\") " pod="openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp" Jul 19 17:11:18.542889 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:18.542543 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4cf296b2-f860-4181-a609-3d202761fef8-istio-data\") pod \"e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp\" (UID: \"4cf296b2-f860-4181-a609-3d202761fef8\") " pod="openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp" Jul 19 17:11:18.643847 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:18.643805 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4cf296b2-f860-4181-a609-3d202761fef8-istio-data\") pod \"e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp\" (UID: \"4cf296b2-f860-4181-a609-3d202761fef8\") " pod="openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp" Jul 19 17:11:18.644380 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:18.643871 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmtv7\" (UniqueName: \"kubernetes.io/projected/4cf296b2-f860-4181-a609-3d202761fef8-kube-api-access-xmtv7\") pod \"e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp\" (UID: \"4cf296b2-f860-4181-a609-3d202761fef8\") " pod="openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp" Jul 19 17:11:18.644380 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:18.643911 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4cf296b2-f860-4181-a609-3d202761fef8-istio-token\") pod \"e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp\" (UID: \"4cf296b2-f860-4181-a609-3d202761fef8\") " pod="openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp" Jul 19 17:11:18.644380 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:18.643941 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4cf296b2-f860-4181-a609-3d202761fef8-credential-socket\") pod \"e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp\" (UID: \"4cf296b2-f860-4181-a609-3d202761fef8\") " pod="openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp" Jul 19 17:11:18.644380 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:18.644087 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4cf296b2-f860-4181-a609-3d202761fef8-istio-podinfo\") pod \"e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp\" (UID: \"4cf296b2-f860-4181-a609-3d202761fef8\") " pod="openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp" Jul 19 17:11:18.644380 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:18.644166 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4cf296b2-f860-4181-a609-3d202761fef8-workload-certs\") pod \"e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp\" (UID: \"4cf296b2-f860-4181-a609-3d202761fef8\") " pod="openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp" Jul 19 17:11:18.644380 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:18.644256 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4cf296b2-f860-4181-a609-3d202761fef8-istio-data\") pod \"e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp\" (UID: \"4cf296b2-f860-4181-a609-3d202761fef8\") " pod="openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp" Jul 19 17:11:18.644380 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:18.644299 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4cf296b2-f860-4181-a609-3d202761fef8-istio-envoy\") pod \"e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp\" (UID: \"4cf296b2-f860-4181-a609-3d202761fef8\") " pod="openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp" Jul 19 17:11:18.644380 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:18.644330 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4cf296b2-f860-4181-a609-3d202761fef8-workload-socket\") pod \"e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp\" (UID: \"4cf296b2-f860-4181-a609-3d202761fef8\") " pod="openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp" Jul 19 17:11:18.644380 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:18.644364 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4cf296b2-f860-4181-a609-3d202761fef8-istiod-ca-cert\") pod \"e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp\" (UID: \"4cf296b2-f860-4181-a609-3d202761fef8\") " pod="openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp" Jul 19 17:11:18.644862 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:18.644366 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4cf296b2-f860-4181-a609-3d202761fef8-credential-socket\") pod \"e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp\" (UID: \"4cf296b2-f860-4181-a609-3d202761fef8\") " pod="openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp" Jul 19 17:11:18.644862 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:18.644477 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4cf296b2-f860-4181-a609-3d202761fef8-workload-certs\") pod \"e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp\" (UID: \"4cf296b2-f860-4181-a609-3d202761fef8\") " pod="openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp" Jul 19 17:11:18.644862 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:18.644560 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4cf296b2-f860-4181-a609-3d202761fef8-workload-socket\") pod \"e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp\" (UID: \"4cf296b2-f860-4181-a609-3d202761fef8\") " pod="openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp" Jul 19 17:11:18.644966 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:18.644940 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4cf296b2-f860-4181-a609-3d202761fef8-istiod-ca-cert\") pod \"e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp\" (UID: \"4cf296b2-f860-4181-a609-3d202761fef8\") " pod="openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp" Jul 19 17:11:18.646624 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:18.646604 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4cf296b2-f860-4181-a609-3d202761fef8-istio-envoy\") pod \"e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp\" (UID: \"4cf296b2-f860-4181-a609-3d202761fef8\") " pod="openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp" Jul 19 17:11:18.646790 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:18.646770 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4cf296b2-f860-4181-a609-3d202761fef8-istio-podinfo\") pod \"e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp\" (UID: \"4cf296b2-f860-4181-a609-3d202761fef8\") " pod="openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp" Jul 19 17:11:18.651686 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:18.651660 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmtv7\" (UniqueName: \"kubernetes.io/projected/4cf296b2-f860-4181-a609-3d202761fef8-kube-api-access-xmtv7\") pod \"e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp\" (UID: \"4cf296b2-f860-4181-a609-3d202761fef8\") " pod="openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp" Jul 19 17:11:18.651792 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:18.651659 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4cf296b2-f860-4181-a609-3d202761fef8-istio-token\") pod \"e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp\" (UID: \"4cf296b2-f860-4181-a609-3d202761fef8\") " pod="openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp" Jul 19 17:11:18.754886 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:18.754793 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp" Jul 19 17:11:18.894299 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:18.894269 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp"] Jul 19 17:11:18.896224 ip-10-0-134-203 kubenswrapper[2583]: W0719 17:11:18.896190 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cf296b2_f860_4181_a609_3d202761fef8.slice/crio-8a5e33e2cd05f47bb13b1c85f5a5a708fd345c3bdf6612b27184e29244aa2c65 WatchSource:0}: Error finding container 8a5e33e2cd05f47bb13b1c85f5a5a708fd345c3bdf6612b27184e29244aa2c65: Status 404 returned error can't find the container with id 8a5e33e2cd05f47bb13b1c85f5a5a708fd345c3bdf6612b27184e29244aa2c65 Jul 19 17:11:18.898514 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:18.898479 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:11:18.898615 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:18.898547 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:11:18.898615 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:18.898578 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:11:19.152477 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:19.152435 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp" event={"ID":"4cf296b2-f860-4181-a609-3d202761fef8","Type":"ContainerStarted","Data":"83abdd82afba445f44e3541432b7508b39161658139155bd9abdcb11785edef2"} Jul 19 17:11:19.152632 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:19.152506 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp" event={"ID":"4cf296b2-f860-4181-a609-3d202761fef8","Type":"ContainerStarted","Data":"8a5e33e2cd05f47bb13b1c85f5a5a708fd345c3bdf6612b27184e29244aa2c65"} Jul 19 17:11:19.171258 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:19.171194 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp" podStartSLOduration=1.171172372 podStartE2EDuration="1.171172372s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 17:11:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 17:11:19.168985631 +0000 UTC m=+1974.155561116" watchObservedRunningTime="2026-07-19 17:11:19.171172372 +0000 UTC m=+1974.157747860" Jul 19 17:11:19.755909 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:19.755863 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp" Jul 19 17:11:19.761095 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:19.761037 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp" Jul 19 17:11:20.157879 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:20.157838 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp" Jul 19 17:11:20.158941 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:20.158918 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp" Jul 19 17:11:22.897353 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:22.897285 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr" Jul 19 17:11:23.088575 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:23.088533 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db1eda5e-b48f-446b-b90f-886b9259439f-istio-envoy\") pod \"db1eda5e-b48f-446b-b90f-886b9259439f\" (UID: \"db1eda5e-b48f-446b-b90f-886b9259439f\") " Jul 19 17:11:23.088575 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:23.088574 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db1eda5e-b48f-446b-b90f-886b9259439f-credential-socket\") pod \"db1eda5e-b48f-446b-b90f-886b9259439f\" (UID: \"db1eda5e-b48f-446b-b90f-886b9259439f\") " Jul 19 17:11:23.088852 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:23.088604 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db1eda5e-b48f-446b-b90f-886b9259439f-istio-podinfo\") pod \"db1eda5e-b48f-446b-b90f-886b9259439f\" (UID: \"db1eda5e-b48f-446b-b90f-886b9259439f\") " Jul 19 17:11:23.088852 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:23.088620 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db1eda5e-b48f-446b-b90f-886b9259439f-istio-token\") pod \"db1eda5e-b48f-446b-b90f-886b9259439f\" (UID: \"db1eda5e-b48f-446b-b90f-886b9259439f\") " Jul 19 17:11:23.088852 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:23.088755 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db1eda5e-b48f-446b-b90f-886b9259439f-workload-socket\") pod \"db1eda5e-b48f-446b-b90f-886b9259439f\" (UID: \"db1eda5e-b48f-446b-b90f-886b9259439f\") " Jul 19 17:11:23.088852 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:23.088799 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db1eda5e-b48f-446b-b90f-886b9259439f-istiod-ca-cert\") pod \"db1eda5e-b48f-446b-b90f-886b9259439f\" (UID: \"db1eda5e-b48f-446b-b90f-886b9259439f\") " Jul 19 17:11:23.088852 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:23.088829 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db1eda5e-b48f-446b-b90f-886b9259439f-workload-certs\") pod \"db1eda5e-b48f-446b-b90f-886b9259439f\" (UID: \"db1eda5e-b48f-446b-b90f-886b9259439f\") " Jul 19 17:11:23.089147 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:23.088889 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s257n\" (UniqueName: \"kubernetes.io/projected/db1eda5e-b48f-446b-b90f-886b9259439f-kube-api-access-s257n\") pod \"db1eda5e-b48f-446b-b90f-886b9259439f\" (UID: \"db1eda5e-b48f-446b-b90f-886b9259439f\") " Jul 19 17:11:23.089147 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:23.088978 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db1eda5e-b48f-446b-b90f-886b9259439f-istio-data\") pod \"db1eda5e-b48f-446b-b90f-886b9259439f\" (UID: \"db1eda5e-b48f-446b-b90f-886b9259439f\") " Jul 19 17:11:23.089147 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:23.088997 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db1eda5e-b48f-446b-b90f-886b9259439f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "db1eda5e-b48f-446b-b90f-886b9259439f" (UID: "db1eda5e-b48f-446b-b90f-886b9259439f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:11:23.089147 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:23.089120 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db1eda5e-b48f-446b-b90f-886b9259439f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "db1eda5e-b48f-446b-b90f-886b9259439f" (UID: "db1eda5e-b48f-446b-b90f-886b9259439f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:11:23.089385 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:23.089225 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db1eda5e-b48f-446b-b90f-886b9259439f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "db1eda5e-b48f-446b-b90f-886b9259439f" (UID: "db1eda5e-b48f-446b-b90f-886b9259439f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 19 17:11:23.089504 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:23.089461 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db1eda5e-b48f-446b-b90f-886b9259439f-workload-socket\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:11:23.089504 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:23.089497 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db1eda5e-b48f-446b-b90f-886b9259439f-istiod-ca-cert\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:11:23.089504 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:23.089497 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db1eda5e-b48f-446b-b90f-886b9259439f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "db1eda5e-b48f-446b-b90f-886b9259439f" (UID: "db1eda5e-b48f-446b-b90f-886b9259439f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:11:23.089766 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:23.089512 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db1eda5e-b48f-446b-b90f-886b9259439f-workload-certs\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:11:23.090129 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:23.090105 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db1eda5e-b48f-446b-b90f-886b9259439f-istio-data" (OuterVolumeSpecName: "istio-data") pod "db1eda5e-b48f-446b-b90f-886b9259439f" (UID: "db1eda5e-b48f-446b-b90f-886b9259439f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:11:23.091169 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:23.091146 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db1eda5e-b48f-446b-b90f-886b9259439f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "db1eda5e-b48f-446b-b90f-886b9259439f" (UID: "db1eda5e-b48f-446b-b90f-886b9259439f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:11:23.091251 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:23.091232 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db1eda5e-b48f-446b-b90f-886b9259439f-istio-token" (OuterVolumeSpecName: "istio-token") pod "db1eda5e-b48f-446b-b90f-886b9259439f" (UID: "db1eda5e-b48f-446b-b90f-886b9259439f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 19 17:11:23.091367 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:23.091347 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/db1eda5e-b48f-446b-b90f-886b9259439f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "db1eda5e-b48f-446b-b90f-886b9259439f" (UID: "db1eda5e-b48f-446b-b90f-886b9259439f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 19 17:11:23.091419 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:23.091359 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db1eda5e-b48f-446b-b90f-886b9259439f-kube-api-access-s257n" (OuterVolumeSpecName: "kube-api-access-s257n") pod "db1eda5e-b48f-446b-b90f-886b9259439f" (UID: "db1eda5e-b48f-446b-b90f-886b9259439f"). InnerVolumeSpecName "kube-api-access-s257n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 19 17:11:23.173343 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:23.173253 2583 generic.go:356] "Generic (PLEG): container finished" podID="db1eda5e-b48f-446b-b90f-886b9259439f" containerID="98d18f25a116693fadc7d8e27808a08c06b2a9cf6f35c71c5b266aab376ce666" exitCode=0 Jul 19 17:11:23.173343 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:23.173301 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr" Jul 19 17:11:23.173343 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:23.173310 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr" event={"ID":"db1eda5e-b48f-446b-b90f-886b9259439f","Type":"ContainerDied","Data":"98d18f25a116693fadc7d8e27808a08c06b2a9cf6f35c71c5b266aab376ce666"} Jul 19 17:11:23.173597 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:23.173351 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr" event={"ID":"db1eda5e-b48f-446b-b90f-886b9259439f","Type":"ContainerDied","Data":"3e1c263e62a8ab621f7ceeec0e5e8eef35ddea25abb2a1f94cd696381ad414ca"} Jul 19 17:11:23.173597 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:23.173374 2583 scope.go:117] "RemoveContainer" containerID="98d18f25a116693fadc7d8e27808a08c06b2a9cf6f35c71c5b266aab376ce666" Jul 19 17:11:23.182609 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:23.182592 2583 scope.go:117] "RemoveContainer" containerID="eede9aa39d05ff5bb31410d83ccb76998a8100196ee06d81368035aabb997a15" Jul 19 17:11:23.190535 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:23.190453 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s257n\" (UniqueName: \"kubernetes.io/projected/db1eda5e-b48f-446b-b90f-886b9259439f-kube-api-access-s257n\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:11:23.190535 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:23.190475 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db1eda5e-b48f-446b-b90f-886b9259439f-istio-data\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:11:23.190535 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:23.190487 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db1eda5e-b48f-446b-b90f-886b9259439f-istio-envoy\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:11:23.190535 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:23.190495 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db1eda5e-b48f-446b-b90f-886b9259439f-credential-socket\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:11:23.190535 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:23.190504 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db1eda5e-b48f-446b-b90f-886b9259439f-istio-podinfo\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:11:23.190535 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:23.190513 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db1eda5e-b48f-446b-b90f-886b9259439f-istio-token\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:11:23.190804 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:23.190550 2583 scope.go:117] "RemoveContainer" containerID="98d18f25a116693fadc7d8e27808a08c06b2a9cf6f35c71c5b266aab376ce666" Jul 19 17:11:23.190861 ip-10-0-134-203 kubenswrapper[2583]: E0719 17:11:23.190842 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98d18f25a116693fadc7d8e27808a08c06b2a9cf6f35c71c5b266aab376ce666\": container with ID starting with 98d18f25a116693fadc7d8e27808a08c06b2a9cf6f35c71c5b266aab376ce666 not found: ID does not exist" containerID="98d18f25a116693fadc7d8e27808a08c06b2a9cf6f35c71c5b266aab376ce666" Jul 19 17:11:23.190897 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:23.190868 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98d18f25a116693fadc7d8e27808a08c06b2a9cf6f35c71c5b266aab376ce666"} err="failed to get container status \"98d18f25a116693fadc7d8e27808a08c06b2a9cf6f35c71c5b266aab376ce666\": rpc error: code = NotFound desc = could not find container \"98d18f25a116693fadc7d8e27808a08c06b2a9cf6f35c71c5b266aab376ce666\": container with ID starting with 98d18f25a116693fadc7d8e27808a08c06b2a9cf6f35c71c5b266aab376ce666 not found: ID does not exist" Jul 19 17:11:23.190897 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:23.190883 2583 scope.go:117] "RemoveContainer" containerID="eede9aa39d05ff5bb31410d83ccb76998a8100196ee06d81368035aabb997a15" Jul 19 17:11:23.191377 ip-10-0-134-203 kubenswrapper[2583]: E0719 17:11:23.191305 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eede9aa39d05ff5bb31410d83ccb76998a8100196ee06d81368035aabb997a15\": container with ID starting with eede9aa39d05ff5bb31410d83ccb76998a8100196ee06d81368035aabb997a15 not found: ID does not exist" containerID="eede9aa39d05ff5bb31410d83ccb76998a8100196ee06d81368035aabb997a15" Jul 19 17:11:23.191499 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:23.191364 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eede9aa39d05ff5bb31410d83ccb76998a8100196ee06d81368035aabb997a15"} err="failed to get container status \"eede9aa39d05ff5bb31410d83ccb76998a8100196ee06d81368035aabb997a15\": rpc error: code = NotFound desc = could not find container \"eede9aa39d05ff5bb31410d83ccb76998a8100196ee06d81368035aabb997a15\": container with ID starting with eede9aa39d05ff5bb31410d83ccb76998a8100196ee06d81368035aabb997a15 not found: ID does not exist" Jul 19 17:11:23.193522 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:23.193499 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr"] Jul 19 17:11:23.197927 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:23.197904 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-5f384e8f-gw-openshift-default-7d5bd68c55-bfhqr"] Jul 19 17:11:23.602611 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:23.602576 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db1eda5e-b48f-446b-b90f-886b9259439f" path="/var/lib/kubelet/pods/db1eda5e-b48f-446b-b90f-886b9259439f/volumes" Jul 19 17:11:35.159522 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:35.159437 2583 patch_prober.go:28] interesting pod/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body= Jul 19 17:11:35.159522 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:35.159510 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp" podUID="4cf296b2-f860-4181-a609-3d202761fef8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" Jul 19 17:11:35.231927 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:35.231838 2583 generic.go:356] "Generic (PLEG): container finished" podID="4cf296b2-f860-4181-a609-3d202761fef8" containerID="83abdd82afba445f44e3541432b7508b39161658139155bd9abdcb11785edef2" exitCode=0 Jul 19 17:11:35.231927 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:35.231900 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp" event={"ID":"4cf296b2-f860-4181-a609-3d202761fef8","Type":"ContainerDied","Data":"83abdd82afba445f44e3541432b7508b39161658139155bd9abdcb11785edef2"} Jul 19 17:11:35.232331 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:35.232316 2583 scope.go:117] "RemoveContainer" containerID="83abdd82afba445f44e3541432b7508b39161658139155bd9abdcb11785edef2" Jul 19 17:11:35.233362 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:35.233333 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:11:35.233429 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:35.233405 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:11:35.233480 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:35.233436 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:11:36.241402 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:36.241355 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp" event={"ID":"4cf296b2-f860-4181-a609-3d202761fef8","Type":"ContainerStarted","Data":"602ccaeb86bc56c237f6ee77d1c6761d3a0a2a83892297b33b74546387a34ff7"} Jul 19 17:11:36.755360 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:36.755309 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp" Jul 19 17:11:36.755584 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:36.755542 2583 patch_prober.go:28] interesting pod/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body= Jul 19 17:11:36.755654 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:36.755605 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp" podUID="4cf296b2-f860-4181-a609-3d202761fef8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" Jul 19 17:11:37.755582 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:37.755542 2583 patch_prober.go:28] interesting pod/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body= Jul 19 17:11:37.755979 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:37.755619 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp" podUID="4cf296b2-f860-4181-a609-3d202761fef8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" Jul 19 17:11:38.755842 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:38.755800 2583 patch_prober.go:28] interesting pod/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body= Jul 19 17:11:38.756242 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:38.755869 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp" podUID="4cf296b2-f860-4181-a609-3d202761fef8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" Jul 19 17:11:39.755986 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:39.755946 2583 patch_prober.go:28] interesting pod/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body= Jul 19 17:11:39.756480 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:39.756008 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp" podUID="4cf296b2-f860-4181-a609-3d202761fef8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" Jul 19 17:11:40.758808 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:40.758777 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp" Jul 19 17:11:40.759293 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:40.759159 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp" Jul 19 17:11:40.759817 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:11:40.759798 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp" Jul 19 17:12:02.647026 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:02.646987 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp"] Jul 19 17:12:02.647609 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:02.647247 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp" podUID="4cf296b2-f860-4181-a609-3d202761fef8" containerName="istio-proxy" containerID="cri-o://602ccaeb86bc56c237f6ee77d1c6761d3a0a2a83892297b33b74546387a34ff7" gracePeriod=30 Jul 19 17:12:03.379412 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:03.379377 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj"] Jul 19 17:12:03.379770 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:03.379749 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db1eda5e-b48f-446b-b90f-886b9259439f" containerName="istio-proxy" Jul 19 17:12:03.379770 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:03.379765 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="db1eda5e-b48f-446b-b90f-886b9259439f" containerName="istio-proxy" Jul 19 17:12:03.379965 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:03.379781 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db1eda5e-b48f-446b-b90f-886b9259439f" containerName="istio-proxy" Jul 19 17:12:03.379965 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:03.379789 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="db1eda5e-b48f-446b-b90f-886b9259439f" containerName="istio-proxy" Jul 19 17:12:03.379965 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:03.379869 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="db1eda5e-b48f-446b-b90f-886b9259439f" containerName="istio-proxy" Jul 19 17:12:03.382032 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:03.382010 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj" Jul 19 17:12:03.384309 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:03.384282 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-352a195c-openshift-default-dockercfg-wzgwv\"" Jul 19 17:12:03.395560 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:03.395532 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj"] Jul 19 17:12:03.483922 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:03.483882 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8557b4d5-630f-4808-8a46-ee3e9337c3eb-istio-envoy\") pod \"e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj\" (UID: \"8557b4d5-630f-4808-8a46-ee3e9337c3eb\") " pod="openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj" Jul 19 17:12:03.484139 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:03.483998 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8557b4d5-630f-4808-8a46-ee3e9337c3eb-istio-token\") pod \"e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj\" (UID: \"8557b4d5-630f-4808-8a46-ee3e9337c3eb\") " pod="openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj" Jul 19 17:12:03.484139 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:03.484031 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8557b4d5-630f-4808-8a46-ee3e9337c3eb-credential-socket\") pod \"e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj\" (UID: \"8557b4d5-630f-4808-8a46-ee3e9337c3eb\") " pod="openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj" Jul 19 17:12:03.484139 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:03.484079 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jxxj\" (UniqueName: \"kubernetes.io/projected/8557b4d5-630f-4808-8a46-ee3e9337c3eb-kube-api-access-6jxxj\") pod \"e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj\" (UID: \"8557b4d5-630f-4808-8a46-ee3e9337c3eb\") " pod="openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj" Jul 19 17:12:03.484332 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:03.484139 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8557b4d5-630f-4808-8a46-ee3e9337c3eb-workload-certs\") pod \"e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj\" (UID: \"8557b4d5-630f-4808-8a46-ee3e9337c3eb\") " pod="openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj" Jul 19 17:12:03.484332 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:03.484171 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8557b4d5-630f-4808-8a46-ee3e9337c3eb-istiod-ca-cert\") pod \"e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj\" (UID: \"8557b4d5-630f-4808-8a46-ee3e9337c3eb\") " pod="openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj" Jul 19 17:12:03.484332 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:03.484203 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8557b4d5-630f-4808-8a46-ee3e9337c3eb-istio-data\") pod \"e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj\" (UID: \"8557b4d5-630f-4808-8a46-ee3e9337c3eb\") " pod="openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj" Jul 19 17:12:03.484332 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:03.484242 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8557b4d5-630f-4808-8a46-ee3e9337c3eb-workload-socket\") pod \"e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj\" (UID: \"8557b4d5-630f-4808-8a46-ee3e9337c3eb\") " pod="openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj" Jul 19 17:12:03.484332 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:03.484270 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8557b4d5-630f-4808-8a46-ee3e9337c3eb-istio-podinfo\") pod \"e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj\" (UID: \"8557b4d5-630f-4808-8a46-ee3e9337c3eb\") " pod="openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj" Jul 19 17:12:03.584808 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:03.584772 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8557b4d5-630f-4808-8a46-ee3e9337c3eb-workload-socket\") pod \"e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj\" (UID: \"8557b4d5-630f-4808-8a46-ee3e9337c3eb\") " pod="openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj" Jul 19 17:12:03.584808 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:03.584809 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8557b4d5-630f-4808-8a46-ee3e9337c3eb-istio-podinfo\") pod \"e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj\" (UID: \"8557b4d5-630f-4808-8a46-ee3e9337c3eb\") " pod="openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj" Jul 19 17:12:03.585090 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:03.584840 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8557b4d5-630f-4808-8a46-ee3e9337c3eb-istio-envoy\") pod \"e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj\" (UID: \"8557b4d5-630f-4808-8a46-ee3e9337c3eb\") " pod="openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj" Jul 19 17:12:03.585090 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:03.584893 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8557b4d5-630f-4808-8a46-ee3e9337c3eb-istio-token\") pod \"e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj\" (UID: \"8557b4d5-630f-4808-8a46-ee3e9337c3eb\") " pod="openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj" Jul 19 17:12:03.585090 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:03.584927 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8557b4d5-630f-4808-8a46-ee3e9337c3eb-credential-socket\") pod \"e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj\" (UID: \"8557b4d5-630f-4808-8a46-ee3e9337c3eb\") " pod="openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj" Jul 19 17:12:03.585090 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:03.584953 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jxxj\" (UniqueName: \"kubernetes.io/projected/8557b4d5-630f-4808-8a46-ee3e9337c3eb-kube-api-access-6jxxj\") pod \"e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj\" (UID: \"8557b4d5-630f-4808-8a46-ee3e9337c3eb\") " pod="openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj" Jul 19 17:12:03.585090 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:03.585012 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8557b4d5-630f-4808-8a46-ee3e9337c3eb-workload-certs\") pod \"e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj\" (UID: \"8557b4d5-630f-4808-8a46-ee3e9337c3eb\") " pod="openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj" Jul 19 17:12:03.585541 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:03.585111 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8557b4d5-630f-4808-8a46-ee3e9337c3eb-istiod-ca-cert\") pod \"e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj\" (UID: \"8557b4d5-630f-4808-8a46-ee3e9337c3eb\") " pod="openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj" Jul 19 17:12:03.585541 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:03.585167 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8557b4d5-630f-4808-8a46-ee3e9337c3eb-istio-data\") pod \"e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj\" (UID: \"8557b4d5-630f-4808-8a46-ee3e9337c3eb\") " pod="openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj" Jul 19 17:12:03.585541 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:03.585291 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8557b4d5-630f-4808-8a46-ee3e9337c3eb-workload-socket\") pod \"e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj\" (UID: \"8557b4d5-630f-4808-8a46-ee3e9337c3eb\") " pod="openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj" Jul 19 17:12:03.585815 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:03.585537 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8557b4d5-630f-4808-8a46-ee3e9337c3eb-workload-certs\") pod \"e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj\" (UID: \"8557b4d5-630f-4808-8a46-ee3e9337c3eb\") " pod="openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj" Jul 19 17:12:03.585815 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:03.585583 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8557b4d5-630f-4808-8a46-ee3e9337c3eb-istio-data\") pod \"e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj\" (UID: \"8557b4d5-630f-4808-8a46-ee3e9337c3eb\") " pod="openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj" Jul 19 17:12:03.585815 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:03.585641 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8557b4d5-630f-4808-8a46-ee3e9337c3eb-credential-socket\") pod \"e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj\" (UID: \"8557b4d5-630f-4808-8a46-ee3e9337c3eb\") " pod="openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj" Jul 19 17:12:03.586039 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:03.586012 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8557b4d5-630f-4808-8a46-ee3e9337c3eb-istiod-ca-cert\") pod \"e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj\" (UID: \"8557b4d5-630f-4808-8a46-ee3e9337c3eb\") " pod="openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj" Jul 19 17:12:03.587276 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:03.587255 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8557b4d5-630f-4808-8a46-ee3e9337c3eb-istio-envoy\") pod \"e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj\" (UID: \"8557b4d5-630f-4808-8a46-ee3e9337c3eb\") " pod="openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj" Jul 19 17:12:03.587438 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:03.587423 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8557b4d5-630f-4808-8a46-ee3e9337c3eb-istio-podinfo\") pod \"e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj\" (UID: \"8557b4d5-630f-4808-8a46-ee3e9337c3eb\") " pod="openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj" Jul 19 17:12:03.592542 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:03.592523 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8557b4d5-630f-4808-8a46-ee3e9337c3eb-istio-token\") pod \"e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj\" (UID: \"8557b4d5-630f-4808-8a46-ee3e9337c3eb\") " pod="openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj" Jul 19 17:12:03.592927 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:03.592907 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jxxj\" (UniqueName: \"kubernetes.io/projected/8557b4d5-630f-4808-8a46-ee3e9337c3eb-kube-api-access-6jxxj\") pod \"e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj\" (UID: \"8557b4d5-630f-4808-8a46-ee3e9337c3eb\") " pod="openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj" Jul 19 17:12:03.693516 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:03.693427 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj" Jul 19 17:12:03.840753 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:03.838266 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj"] Jul 19 17:12:03.846361 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:03.846315 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:12:03.846469 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:03.846406 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:12:03.846469 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:03.846448 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:12:04.380552 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:04.380512 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj" event={"ID":"8557b4d5-630f-4808-8a46-ee3e9337c3eb","Type":"ContainerStarted","Data":"a2326887fa763e7c945aef3895335d12aaaf2fcec15294b4b8d73cc396b6711f"} Jul 19 17:12:04.380737 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:04.380573 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj" event={"ID":"8557b4d5-630f-4808-8a46-ee3e9337c3eb","Type":"ContainerStarted","Data":"dc74d590d2ace5a670d833da96825eb9bf887ab5f2e08a2b1e6550a1fdeb6461"} Jul 19 17:12:04.398728 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:04.398669 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj" podStartSLOduration=1.3986549099999999 podStartE2EDuration="1.39865491s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 17:12:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 17:12:04.39766078 +0000 UTC m=+2019.384236265" watchObservedRunningTime="2026-07-19 17:12:04.39865491 +0000 UTC m=+2019.385230421" Jul 19 17:12:04.694145 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:04.694021 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj" Jul 19 17:12:04.698793 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:04.698764 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj" Jul 19 17:12:05.385557 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:05.385528 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj" Jul 19 17:12:05.386493 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:05.386474 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj" Jul 19 17:12:07.905383 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:07.905359 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp" Jul 19 17:12:07.929802 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:07.929733 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4cf296b2-f860-4181-a609-3d202761fef8-istio-podinfo\") pod \"4cf296b2-f860-4181-a609-3d202761fef8\" (UID: \"4cf296b2-f860-4181-a609-3d202761fef8\") " Jul 19 17:12:07.929802 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:07.929800 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4cf296b2-f860-4181-a609-3d202761fef8-istio-data\") pod \"4cf296b2-f860-4181-a609-3d202761fef8\" (UID: \"4cf296b2-f860-4181-a609-3d202761fef8\") " Jul 19 17:12:07.930823 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:07.929826 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4cf296b2-f860-4181-a609-3d202761fef8-istio-envoy\") pod \"4cf296b2-f860-4181-a609-3d202761fef8\" (UID: \"4cf296b2-f860-4181-a609-3d202761fef8\") " Jul 19 17:12:07.930823 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:07.929855 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4cf296b2-f860-4181-a609-3d202761fef8-istiod-ca-cert\") pod \"4cf296b2-f860-4181-a609-3d202761fef8\" (UID: \"4cf296b2-f860-4181-a609-3d202761fef8\") " Jul 19 17:12:07.930823 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:07.929878 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4cf296b2-f860-4181-a609-3d202761fef8-workload-certs\") pod \"4cf296b2-f860-4181-a609-3d202761fef8\" (UID: \"4cf296b2-f860-4181-a609-3d202761fef8\") " Jul 19 17:12:07.930823 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:07.929924 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xmtv7\" (UniqueName: \"kubernetes.io/projected/4cf296b2-f860-4181-a609-3d202761fef8-kube-api-access-xmtv7\") pod \"4cf296b2-f860-4181-a609-3d202761fef8\" (UID: \"4cf296b2-f860-4181-a609-3d202761fef8\") " Jul 19 17:12:07.930823 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:07.929992 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4cf296b2-f860-4181-a609-3d202761fef8-credential-socket\") pod \"4cf296b2-f860-4181-a609-3d202761fef8\" (UID: \"4cf296b2-f860-4181-a609-3d202761fef8\") " Jul 19 17:12:07.930823 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:07.930027 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4cf296b2-f860-4181-a609-3d202761fef8-workload-socket\") pod \"4cf296b2-f860-4181-a609-3d202761fef8\" (UID: \"4cf296b2-f860-4181-a609-3d202761fef8\") " Jul 19 17:12:07.930823 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:07.930081 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4cf296b2-f860-4181-a609-3d202761fef8-istio-token\") pod \"4cf296b2-f860-4181-a609-3d202761fef8\" (UID: \"4cf296b2-f860-4181-a609-3d202761fef8\") " Jul 19 17:12:07.930823 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:07.930197 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cf296b2-f860-4181-a609-3d202761fef8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4cf296b2-f860-4181-a609-3d202761fef8" (UID: "4cf296b2-f860-4181-a609-3d202761fef8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:12:07.930823 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:07.930416 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4cf296b2-f860-4181-a609-3d202761fef8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4cf296b2-f860-4181-a609-3d202761fef8" (UID: "4cf296b2-f860-4181-a609-3d202761fef8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 19 17:12:07.930823 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:07.930432 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4cf296b2-f860-4181-a609-3d202761fef8-workload-certs\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:12:07.930823 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:07.930628 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cf296b2-f860-4181-a609-3d202761fef8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4cf296b2-f860-4181-a609-3d202761fef8" (UID: "4cf296b2-f860-4181-a609-3d202761fef8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:12:07.930823 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:07.930684 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cf296b2-f860-4181-a609-3d202761fef8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4cf296b2-f860-4181-a609-3d202761fef8" (UID: "4cf296b2-f860-4181-a609-3d202761fef8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:12:07.930823 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:07.930774 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cf296b2-f860-4181-a609-3d202761fef8-istio-data" (OuterVolumeSpecName: "istio-data") pod "4cf296b2-f860-4181-a609-3d202761fef8" (UID: "4cf296b2-f860-4181-a609-3d202761fef8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:12:07.932124 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:07.932093 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4cf296b2-f860-4181-a609-3d202761fef8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4cf296b2-f860-4181-a609-3d202761fef8" (UID: "4cf296b2-f860-4181-a609-3d202761fef8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 19 17:12:07.932492 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:07.932473 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cf296b2-f860-4181-a609-3d202761fef8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4cf296b2-f860-4181-a609-3d202761fef8" (UID: "4cf296b2-f860-4181-a609-3d202761fef8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:12:07.932562 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:07.932486 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4cf296b2-f860-4181-a609-3d202761fef8-istio-token" (OuterVolumeSpecName: "istio-token") pod "4cf296b2-f860-4181-a609-3d202761fef8" (UID: "4cf296b2-f860-4181-a609-3d202761fef8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 19 17:12:07.932632 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:07.932576 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4cf296b2-f860-4181-a609-3d202761fef8-kube-api-access-xmtv7" (OuterVolumeSpecName: "kube-api-access-xmtv7") pod "4cf296b2-f860-4181-a609-3d202761fef8" (UID: "4cf296b2-f860-4181-a609-3d202761fef8"). InnerVolumeSpecName "kube-api-access-xmtv7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 19 17:12:08.031326 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:08.031290 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4cf296b2-f860-4181-a609-3d202761fef8-credential-socket\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:12:08.031326 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:08.031319 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4cf296b2-f860-4181-a609-3d202761fef8-workload-socket\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:12:08.031326 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:08.031330 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4cf296b2-f860-4181-a609-3d202761fef8-istio-token\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:12:08.031326 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:08.031340 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4cf296b2-f860-4181-a609-3d202761fef8-istio-podinfo\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:12:08.031600 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:08.031348 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4cf296b2-f860-4181-a609-3d202761fef8-istio-data\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:12:08.031600 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:08.031356 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4cf296b2-f860-4181-a609-3d202761fef8-istio-envoy\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:12:08.031600 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:08.031365 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4cf296b2-f860-4181-a609-3d202761fef8-istiod-ca-cert\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:12:08.031600 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:08.031374 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xmtv7\" (UniqueName: \"kubernetes.io/projected/4cf296b2-f860-4181-a609-3d202761fef8-kube-api-access-xmtv7\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:12:08.401915 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:08.401879 2583 generic.go:356] "Generic (PLEG): container finished" podID="4cf296b2-f860-4181-a609-3d202761fef8" containerID="602ccaeb86bc56c237f6ee77d1c6761d3a0a2a83892297b33b74546387a34ff7" exitCode=0 Jul 19 17:12:08.402102 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:08.401934 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp" Jul 19 17:12:08.402102 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:08.401957 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp" event={"ID":"4cf296b2-f860-4181-a609-3d202761fef8","Type":"ContainerDied","Data":"602ccaeb86bc56c237f6ee77d1c6761d3a0a2a83892297b33b74546387a34ff7"} Jul 19 17:12:08.402102 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:08.402035 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp" event={"ID":"4cf296b2-f860-4181-a609-3d202761fef8","Type":"ContainerDied","Data":"8a5e33e2cd05f47bb13b1c85f5a5a708fd345c3bdf6612b27184e29244aa2c65"} Jul 19 17:12:08.402210 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:08.402141 2583 scope.go:117] "RemoveContainer" containerID="602ccaeb86bc56c237f6ee77d1c6761d3a0a2a83892297b33b74546387a34ff7" Jul 19 17:12:08.414584 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:08.414559 2583 scope.go:117] "RemoveContainer" containerID="83abdd82afba445f44e3541432b7508b39161658139155bd9abdcb11785edef2" Jul 19 17:12:08.422956 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:08.422937 2583 scope.go:117] "RemoveContainer" containerID="602ccaeb86bc56c237f6ee77d1c6761d3a0a2a83892297b33b74546387a34ff7" Jul 19 17:12:08.423269 ip-10-0-134-203 kubenswrapper[2583]: E0719 17:12:08.423250 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"602ccaeb86bc56c237f6ee77d1c6761d3a0a2a83892297b33b74546387a34ff7\": container with ID starting with 602ccaeb86bc56c237f6ee77d1c6761d3a0a2a83892297b33b74546387a34ff7 not found: ID does not exist" containerID="602ccaeb86bc56c237f6ee77d1c6761d3a0a2a83892297b33b74546387a34ff7" Jul 19 17:12:08.423337 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:08.423279 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"602ccaeb86bc56c237f6ee77d1c6761d3a0a2a83892297b33b74546387a34ff7"} err="failed to get container status \"602ccaeb86bc56c237f6ee77d1c6761d3a0a2a83892297b33b74546387a34ff7\": rpc error: code = NotFound desc = could not find container \"602ccaeb86bc56c237f6ee77d1c6761d3a0a2a83892297b33b74546387a34ff7\": container with ID starting with 602ccaeb86bc56c237f6ee77d1c6761d3a0a2a83892297b33b74546387a34ff7 not found: ID does not exist" Jul 19 17:12:08.423337 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:08.423298 2583 scope.go:117] "RemoveContainer" containerID="83abdd82afba445f44e3541432b7508b39161658139155bd9abdcb11785edef2" Jul 19 17:12:08.423532 ip-10-0-134-203 kubenswrapper[2583]: E0719 17:12:08.423517 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83abdd82afba445f44e3541432b7508b39161658139155bd9abdcb11785edef2\": container with ID starting with 83abdd82afba445f44e3541432b7508b39161658139155bd9abdcb11785edef2 not found: ID does not exist" containerID="83abdd82afba445f44e3541432b7508b39161658139155bd9abdcb11785edef2" Jul 19 17:12:08.423575 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:08.423536 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83abdd82afba445f44e3541432b7508b39161658139155bd9abdcb11785edef2"} err="failed to get container status \"83abdd82afba445f44e3541432b7508b39161658139155bd9abdcb11785edef2\": rpc error: code = NotFound desc = could not find container \"83abdd82afba445f44e3541432b7508b39161658139155bd9abdcb11785edef2\": container with ID starting with 83abdd82afba445f44e3541432b7508b39161658139155bd9abdcb11785edef2 not found: ID does not exist" Jul 19 17:12:08.423708 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:08.423690 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp"] Jul 19 17:12:08.429401 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:08.429378 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-77bade34-openshift-default-84d688b7df-gwrrp"] Jul 19 17:12:09.601852 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:09.601818 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4cf296b2-f860-4181-a609-3d202761fef8" path="/var/lib/kubelet/pods/4cf296b2-f860-4181-a609-3d202761fef8/volumes" Jul 19 17:12:20.670507 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:20.669750 2583 patch_prober.go:28] interesting pod/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": read tcp 10.134.0.2:47362->10.134.0.32:15021: read: connection reset by peer" start-of-body= Jul 19 17:12:20.670507 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:20.669825 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj" podUID="8557b4d5-630f-4808-8a46-ee3e9337c3eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": read tcp 10.134.0.2:47362->10.134.0.32:15021: read: connection reset by peer" Jul 19 17:12:21.478982 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:21.478947 2583 generic.go:356] "Generic (PLEG): container finished" podID="8557b4d5-630f-4808-8a46-ee3e9337c3eb" containerID="a2326887fa763e7c945aef3895335d12aaaf2fcec15294b4b8d73cc396b6711f" exitCode=0 Jul 19 17:12:21.479224 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:21.479026 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj" event={"ID":"8557b4d5-630f-4808-8a46-ee3e9337c3eb","Type":"ContainerDied","Data":"a2326887fa763e7c945aef3895335d12aaaf2fcec15294b4b8d73cc396b6711f"} Jul 19 17:12:21.479527 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:21.479508 2583 scope.go:117] "RemoveContainer" containerID="a2326887fa763e7c945aef3895335d12aaaf2fcec15294b4b8d73cc396b6711f" Jul 19 17:12:21.480582 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:21.480546 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:12:21.480695 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:21.480637 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:12:21.480769 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:21.480692 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:12:22.485865 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:22.485832 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj" event={"ID":"8557b4d5-630f-4808-8a46-ee3e9337c3eb","Type":"ContainerStarted","Data":"e8ea08b00a149551d156e691ad236142bf413eb59749ab490c7c7db3cdc5c72a"} Jul 19 17:12:22.693703 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:22.693663 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj" Jul 19 17:12:22.693947 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:22.693922 2583 patch_prober.go:28] interesting pod/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body= Jul 19 17:12:22.694002 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:22.693983 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj" podUID="8557b4d5-630f-4808-8a46-ee3e9337c3eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" Jul 19 17:12:23.694518 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:23.694482 2583 patch_prober.go:28] interesting pod/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body= Jul 19 17:12:23.694921 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:23.694541 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj" podUID="8557b4d5-630f-4808-8a46-ee3e9337c3eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" Jul 19 17:12:24.693995 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:24.693962 2583 patch_prober.go:28] interesting pod/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body= Jul 19 17:12:24.694188 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:24.694012 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj" podUID="8557b4d5-630f-4808-8a46-ee3e9337c3eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" Jul 19 17:12:25.694832 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:25.694795 2583 patch_prober.go:28] interesting pod/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body= Jul 19 17:12:25.695197 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:25.694864 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj" podUID="8557b4d5-630f-4808-8a46-ee3e9337c3eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" Jul 19 17:12:26.697746 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:26.697703 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj" Jul 19 17:12:26.698157 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:26.697993 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj" Jul 19 17:12:26.698855 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:26.698840 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj" Jul 19 17:12:45.820907 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:45.820869 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj"] Jul 19 17:12:45.821449 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:45.821123 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj" podUID="8557b4d5-630f-4808-8a46-ee3e9337c3eb" containerName="istio-proxy" containerID="cri-o://e8ea08b00a149551d156e691ad236142bf413eb59749ab490c7c7db3cdc5c72a" gracePeriod=30 Jul 19 17:12:51.081984 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:51.081960 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj" Jul 19 17:12:51.149343 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:51.149260 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8557b4d5-630f-4808-8a46-ee3e9337c3eb-istio-data\") pod \"8557b4d5-630f-4808-8a46-ee3e9337c3eb\" (UID: \"8557b4d5-630f-4808-8a46-ee3e9337c3eb\") " Jul 19 17:12:51.149488 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:51.149361 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8557b4d5-630f-4808-8a46-ee3e9337c3eb-workload-certs\") pod \"8557b4d5-630f-4808-8a46-ee3e9337c3eb\" (UID: \"8557b4d5-630f-4808-8a46-ee3e9337c3eb\") " Jul 19 17:12:51.149488 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:51.149394 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8557b4d5-630f-4808-8a46-ee3e9337c3eb-workload-socket\") pod \"8557b4d5-630f-4808-8a46-ee3e9337c3eb\" (UID: \"8557b4d5-630f-4808-8a46-ee3e9337c3eb\") " Jul 19 17:12:51.149488 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:51.149431 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6jxxj\" (UniqueName: \"kubernetes.io/projected/8557b4d5-630f-4808-8a46-ee3e9337c3eb-kube-api-access-6jxxj\") pod \"8557b4d5-630f-4808-8a46-ee3e9337c3eb\" (UID: \"8557b4d5-630f-4808-8a46-ee3e9337c3eb\") " Jul 19 17:12:51.149611 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:51.149505 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8557b4d5-630f-4808-8a46-ee3e9337c3eb-credential-socket\") pod \"8557b4d5-630f-4808-8a46-ee3e9337c3eb\" (UID: \"8557b4d5-630f-4808-8a46-ee3e9337c3eb\") " Jul 19 17:12:51.149611 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:51.149556 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8557b4d5-630f-4808-8a46-ee3e9337c3eb-istio-envoy\") pod \"8557b4d5-630f-4808-8a46-ee3e9337c3eb\" (UID: \"8557b4d5-630f-4808-8a46-ee3e9337c3eb\") " Jul 19 17:12:51.149611 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:51.149589 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8557b4d5-630f-4808-8a46-ee3e9337c3eb-istio-podinfo\") pod \"8557b4d5-630f-4808-8a46-ee3e9337c3eb\" (UID: \"8557b4d5-630f-4808-8a46-ee3e9337c3eb\") " Jul 19 17:12:51.149762 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:51.149617 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8557b4d5-630f-4808-8a46-ee3e9337c3eb-istio-token\") pod \"8557b4d5-630f-4808-8a46-ee3e9337c3eb\" (UID: \"8557b4d5-630f-4808-8a46-ee3e9337c3eb\") " Jul 19 17:12:51.149762 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:51.149647 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8557b4d5-630f-4808-8a46-ee3e9337c3eb-istiod-ca-cert\") pod \"8557b4d5-630f-4808-8a46-ee3e9337c3eb\" (UID: \"8557b4d5-630f-4808-8a46-ee3e9337c3eb\") " Jul 19 17:12:51.149762 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:51.149640 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8557b4d5-630f-4808-8a46-ee3e9337c3eb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8557b4d5-630f-4808-8a46-ee3e9337c3eb" (UID: "8557b4d5-630f-4808-8a46-ee3e9337c3eb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:12:51.149762 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:51.149654 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8557b4d5-630f-4808-8a46-ee3e9337c3eb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8557b4d5-630f-4808-8a46-ee3e9337c3eb" (UID: "8557b4d5-630f-4808-8a46-ee3e9337c3eb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:12:51.149967 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:51.149847 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8557b4d5-630f-4808-8a46-ee3e9337c3eb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8557b4d5-630f-4808-8a46-ee3e9337c3eb" (UID: "8557b4d5-630f-4808-8a46-ee3e9337c3eb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:12:51.150050 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:51.150031 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8557b4d5-630f-4808-8a46-ee3e9337c3eb-workload-certs\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:12:51.150152 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:51.150086 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8557b4d5-630f-4808-8a46-ee3e9337c3eb-workload-socket\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:12:51.150152 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:51.150107 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8557b4d5-630f-4808-8a46-ee3e9337c3eb-credential-socket\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:12:51.150243 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:51.150152 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8557b4d5-630f-4808-8a46-ee3e9337c3eb-istio-data" (OuterVolumeSpecName: "istio-data") pod "8557b4d5-630f-4808-8a46-ee3e9337c3eb" (UID: "8557b4d5-630f-4808-8a46-ee3e9337c3eb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:12:51.150243 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:51.150155 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8557b4d5-630f-4808-8a46-ee3e9337c3eb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8557b4d5-630f-4808-8a46-ee3e9337c3eb" (UID: "8557b4d5-630f-4808-8a46-ee3e9337c3eb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 19 17:12:51.152192 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:51.152152 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8557b4d5-630f-4808-8a46-ee3e9337c3eb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8557b4d5-630f-4808-8a46-ee3e9337c3eb" (UID: "8557b4d5-630f-4808-8a46-ee3e9337c3eb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 19 17:12:51.152192 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:51.152182 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8557b4d5-630f-4808-8a46-ee3e9337c3eb-kube-api-access-6jxxj" (OuterVolumeSpecName: "kube-api-access-6jxxj") pod "8557b4d5-630f-4808-8a46-ee3e9337c3eb" (UID: "8557b4d5-630f-4808-8a46-ee3e9337c3eb"). InnerVolumeSpecName "kube-api-access-6jxxj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 19 17:12:51.152355 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:51.152220 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8557b4d5-630f-4808-8a46-ee3e9337c3eb-istio-token" (OuterVolumeSpecName: "istio-token") pod "8557b4d5-630f-4808-8a46-ee3e9337c3eb" (UID: "8557b4d5-630f-4808-8a46-ee3e9337c3eb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 19 17:12:51.152355 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:51.152268 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8557b4d5-630f-4808-8a46-ee3e9337c3eb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8557b4d5-630f-4808-8a46-ee3e9337c3eb" (UID: "8557b4d5-630f-4808-8a46-ee3e9337c3eb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:12:51.251486 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:51.251448 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8557b4d5-630f-4808-8a46-ee3e9337c3eb-istio-envoy\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:12:51.251486 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:51.251478 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8557b4d5-630f-4808-8a46-ee3e9337c3eb-istio-podinfo\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:12:51.251486 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:51.251488 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8557b4d5-630f-4808-8a46-ee3e9337c3eb-istio-token\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:12:51.251486 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:51.251498 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8557b4d5-630f-4808-8a46-ee3e9337c3eb-istiod-ca-cert\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:12:51.251735 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:51.251506 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8557b4d5-630f-4808-8a46-ee3e9337c3eb-istio-data\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:12:51.251735 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:51.251514 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6jxxj\" (UniqueName: \"kubernetes.io/projected/8557b4d5-630f-4808-8a46-ee3e9337c3eb-kube-api-access-6jxxj\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:12:51.628317 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:51.628285 2583 generic.go:356] "Generic (PLEG): container finished" podID="8557b4d5-630f-4808-8a46-ee3e9337c3eb" containerID="e8ea08b00a149551d156e691ad236142bf413eb59749ab490c7c7db3cdc5c72a" exitCode=0 Jul 19 17:12:51.628457 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:51.628346 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj" Jul 19 17:12:51.628457 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:51.628360 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj" event={"ID":"8557b4d5-630f-4808-8a46-ee3e9337c3eb","Type":"ContainerDied","Data":"e8ea08b00a149551d156e691ad236142bf413eb59749ab490c7c7db3cdc5c72a"} Jul 19 17:12:51.628457 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:51.628411 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj" event={"ID":"8557b4d5-630f-4808-8a46-ee3e9337c3eb","Type":"ContainerDied","Data":"dc74d590d2ace5a670d833da96825eb9bf887ab5f2e08a2b1e6550a1fdeb6461"} Jul 19 17:12:51.628457 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:51.628433 2583 scope.go:117] "RemoveContainer" containerID="e8ea08b00a149551d156e691ad236142bf413eb59749ab490c7c7db3cdc5c72a" Jul 19 17:12:51.637047 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:51.637032 2583 scope.go:117] "RemoveContainer" containerID="a2326887fa763e7c945aef3895335d12aaaf2fcec15294b4b8d73cc396b6711f" Jul 19 17:12:51.645049 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:51.645030 2583 scope.go:117] "RemoveContainer" containerID="e8ea08b00a149551d156e691ad236142bf413eb59749ab490c7c7db3cdc5c72a" Jul 19 17:12:51.645404 ip-10-0-134-203 kubenswrapper[2583]: E0719 17:12:51.645377 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8ea08b00a149551d156e691ad236142bf413eb59749ab490c7c7db3cdc5c72a\": container with ID starting with e8ea08b00a149551d156e691ad236142bf413eb59749ab490c7c7db3cdc5c72a not found: ID does not exist" containerID="e8ea08b00a149551d156e691ad236142bf413eb59749ab490c7c7db3cdc5c72a" Jul 19 17:12:51.645510 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:51.645411 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8ea08b00a149551d156e691ad236142bf413eb59749ab490c7c7db3cdc5c72a"} err="failed to get container status \"e8ea08b00a149551d156e691ad236142bf413eb59749ab490c7c7db3cdc5c72a\": rpc error: code = NotFound desc = could not find container \"e8ea08b00a149551d156e691ad236142bf413eb59749ab490c7c7db3cdc5c72a\": container with ID starting with e8ea08b00a149551d156e691ad236142bf413eb59749ab490c7c7db3cdc5c72a not found: ID does not exist" Jul 19 17:12:51.645510 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:51.645430 2583 scope.go:117] "RemoveContainer" containerID="a2326887fa763e7c945aef3895335d12aaaf2fcec15294b4b8d73cc396b6711f" Jul 19 17:12:51.645773 ip-10-0-134-203 kubenswrapper[2583]: E0719 17:12:51.645743 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2326887fa763e7c945aef3895335d12aaaf2fcec15294b4b8d73cc396b6711f\": container with ID starting with a2326887fa763e7c945aef3895335d12aaaf2fcec15294b4b8d73cc396b6711f not found: ID does not exist" containerID="a2326887fa763e7c945aef3895335d12aaaf2fcec15294b4b8d73cc396b6711f" Jul 19 17:12:51.645889 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:51.645797 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2326887fa763e7c945aef3895335d12aaaf2fcec15294b4b8d73cc396b6711f"} err="failed to get container status \"a2326887fa763e7c945aef3895335d12aaaf2fcec15294b4b8d73cc396b6711f\": rpc error: code = NotFound desc = could not find container \"a2326887fa763e7c945aef3895335d12aaaf2fcec15294b4b8d73cc396b6711f\": container with ID starting with a2326887fa763e7c945aef3895335d12aaaf2fcec15294b4b8d73cc396b6711f not found: ID does not exist" Jul 19 17:12:51.648625 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:51.648603 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj"] Jul 19 17:12:51.651658 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:51.651638 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-352a195c-openshift-default-86b48cb775-t6mtj"] Jul 19 17:12:53.602097 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:12:53.602038 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8557b4d5-630f-4808-8a46-ee3e9337c3eb" path="/var/lib/kubelet/pods/8557b4d5-630f-4808-8a46-ee3e9337c3eb/volumes" Jul 19 17:14:26.765123 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:26.765011 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6"] Jul 19 17:14:26.765604 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:26.765418 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4cf296b2-f860-4181-a609-3d202761fef8" containerName="istio-proxy" Jul 19 17:14:26.765604 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:26.765430 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="4cf296b2-f860-4181-a609-3d202761fef8" containerName="istio-proxy" Jul 19 17:14:26.765604 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:26.765439 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4cf296b2-f860-4181-a609-3d202761fef8" containerName="istio-proxy" Jul 19 17:14:26.765604 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:26.765445 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="4cf296b2-f860-4181-a609-3d202761fef8" containerName="istio-proxy" Jul 19 17:14:26.765604 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:26.765452 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8557b4d5-630f-4808-8a46-ee3e9337c3eb" containerName="istio-proxy" Jul 19 17:14:26.765604 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:26.765458 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="8557b4d5-630f-4808-8a46-ee3e9337c3eb" containerName="istio-proxy" Jul 19 17:14:26.765604 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:26.765469 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8557b4d5-630f-4808-8a46-ee3e9337c3eb" containerName="istio-proxy" Jul 19 17:14:26.765604 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:26.765475 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="8557b4d5-630f-4808-8a46-ee3e9337c3eb" containerName="istio-proxy" Jul 19 17:14:26.765604 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:26.765525 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="db1eda5e-b48f-446b-b90f-886b9259439f" containerName="istio-proxy" Jul 19 17:14:26.765604 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:26.765532 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="4cf296b2-f860-4181-a609-3d202761fef8" containerName="istio-proxy" Jul 19 17:14:26.765604 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:26.765538 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="8557b4d5-630f-4808-8a46-ee3e9337c3eb" containerName="istio-proxy" Jul 19 17:14:26.765604 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:26.765544 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="4cf296b2-f860-4181-a609-3d202761fef8" containerName="istio-proxy" Jul 19 17:14:26.768550 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:26.768529 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6" Jul 19 17:14:26.770539 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:26.770516 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-cf60ac3a-openshift-default-dockercfg-gzqvf\"" Jul 19 17:14:26.785655 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:26.785631 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6"] Jul 19 17:14:26.856726 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:26.856690 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3254e35b-879b-4a29-84a7-41050fc425a2-istio-podinfo\") pod \"e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6\" (UID: \"3254e35b-879b-4a29-84a7-41050fc425a2\") " pod="openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6" Jul 19 17:14:26.856726 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:26.856737 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3254e35b-879b-4a29-84a7-41050fc425a2-credential-socket\") pod \"e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6\" (UID: \"3254e35b-879b-4a29-84a7-41050fc425a2\") " pod="openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6" Jul 19 17:14:26.856992 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:26.856836 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3254e35b-879b-4a29-84a7-41050fc425a2-workload-certs\") pod \"e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6\" (UID: \"3254e35b-879b-4a29-84a7-41050fc425a2\") " pod="openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6" Jul 19 17:14:26.856992 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:26.856898 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3254e35b-879b-4a29-84a7-41050fc425a2-istio-envoy\") pod \"e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6\" (UID: \"3254e35b-879b-4a29-84a7-41050fc425a2\") " pod="openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6" Jul 19 17:14:26.856992 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:26.856933 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3254e35b-879b-4a29-84a7-41050fc425a2-istio-data\") pod \"e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6\" (UID: \"3254e35b-879b-4a29-84a7-41050fc425a2\") " pod="openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6" Jul 19 17:14:26.857148 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:26.856988 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhx8j\" (UniqueName: \"kubernetes.io/projected/3254e35b-879b-4a29-84a7-41050fc425a2-kube-api-access-jhx8j\") pod \"e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6\" (UID: \"3254e35b-879b-4a29-84a7-41050fc425a2\") " pod="openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6" Jul 19 17:14:26.857148 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:26.857073 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3254e35b-879b-4a29-84a7-41050fc425a2-workload-socket\") pod \"e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6\" (UID: \"3254e35b-879b-4a29-84a7-41050fc425a2\") " pod="openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6" Jul 19 17:14:26.857148 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:26.857132 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3254e35b-879b-4a29-84a7-41050fc425a2-istio-token\") pod \"e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6\" (UID: \"3254e35b-879b-4a29-84a7-41050fc425a2\") " pod="openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6" Jul 19 17:14:26.857294 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:26.857194 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3254e35b-879b-4a29-84a7-41050fc425a2-istiod-ca-cert\") pod \"e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6\" (UID: \"3254e35b-879b-4a29-84a7-41050fc425a2\") " pod="openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6" Jul 19 17:14:26.958544 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:26.958505 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3254e35b-879b-4a29-84a7-41050fc425a2-istio-podinfo\") pod \"e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6\" (UID: \"3254e35b-879b-4a29-84a7-41050fc425a2\") " pod="openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6" Jul 19 17:14:26.958727 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:26.958563 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3254e35b-879b-4a29-84a7-41050fc425a2-credential-socket\") pod \"e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6\" (UID: \"3254e35b-879b-4a29-84a7-41050fc425a2\") " pod="openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6" Jul 19 17:14:26.958727 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:26.958615 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3254e35b-879b-4a29-84a7-41050fc425a2-workload-certs\") pod \"e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6\" (UID: \"3254e35b-879b-4a29-84a7-41050fc425a2\") " pod="openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6" Jul 19 17:14:26.958727 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:26.958661 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3254e35b-879b-4a29-84a7-41050fc425a2-istio-envoy\") pod \"e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6\" (UID: \"3254e35b-879b-4a29-84a7-41050fc425a2\") " pod="openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6" Jul 19 17:14:26.958727 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:26.958695 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3254e35b-879b-4a29-84a7-41050fc425a2-istio-data\") pod \"e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6\" (UID: \"3254e35b-879b-4a29-84a7-41050fc425a2\") " pod="openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6" Jul 19 17:14:26.958727 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:26.958719 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhx8j\" (UniqueName: \"kubernetes.io/projected/3254e35b-879b-4a29-84a7-41050fc425a2-kube-api-access-jhx8j\") pod \"e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6\" (UID: \"3254e35b-879b-4a29-84a7-41050fc425a2\") " pod="openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6" Jul 19 17:14:26.959005 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:26.958764 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3254e35b-879b-4a29-84a7-41050fc425a2-workload-socket\") pod \"e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6\" (UID: \"3254e35b-879b-4a29-84a7-41050fc425a2\") " pod="openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6" Jul 19 17:14:26.959005 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:26.958801 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3254e35b-879b-4a29-84a7-41050fc425a2-istio-token\") pod \"e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6\" (UID: \"3254e35b-879b-4a29-84a7-41050fc425a2\") " pod="openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6" Jul 19 17:14:26.959005 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:26.958868 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3254e35b-879b-4a29-84a7-41050fc425a2-istiod-ca-cert\") pod \"e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6\" (UID: \"3254e35b-879b-4a29-84a7-41050fc425a2\") " pod="openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6" Jul 19 17:14:26.959205 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:26.959153 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3254e35b-879b-4a29-84a7-41050fc425a2-credential-socket\") pod \"e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6\" (UID: \"3254e35b-879b-4a29-84a7-41050fc425a2\") " pod="openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6" Jul 19 17:14:26.959205 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:26.959166 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3254e35b-879b-4a29-84a7-41050fc425a2-istio-data\") pod \"e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6\" (UID: \"3254e35b-879b-4a29-84a7-41050fc425a2\") " pod="openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6" Jul 19 17:14:26.959427 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:26.959403 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3254e35b-879b-4a29-84a7-41050fc425a2-workload-certs\") pod \"e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6\" (UID: \"3254e35b-879b-4a29-84a7-41050fc425a2\") " pod="openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6" Jul 19 17:14:26.959427 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:26.959421 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3254e35b-879b-4a29-84a7-41050fc425a2-workload-socket\") pod \"e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6\" (UID: \"3254e35b-879b-4a29-84a7-41050fc425a2\") " pod="openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6" Jul 19 17:14:26.959651 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:26.959589 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3254e35b-879b-4a29-84a7-41050fc425a2-istiod-ca-cert\") pod \"e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6\" (UID: \"3254e35b-879b-4a29-84a7-41050fc425a2\") " pod="openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6" Jul 19 17:14:26.961720 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:26.961696 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3254e35b-879b-4a29-84a7-41050fc425a2-istio-podinfo\") pod \"e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6\" (UID: \"3254e35b-879b-4a29-84a7-41050fc425a2\") " pod="openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6" Jul 19 17:14:26.961850 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:26.961790 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3254e35b-879b-4a29-84a7-41050fc425a2-istio-envoy\") pod \"e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6\" (UID: \"3254e35b-879b-4a29-84a7-41050fc425a2\") " pod="openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6" Jul 19 17:14:26.974038 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:26.973999 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3254e35b-879b-4a29-84a7-41050fc425a2-istio-token\") pod \"e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6\" (UID: \"3254e35b-879b-4a29-84a7-41050fc425a2\") " pod="openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6" Jul 19 17:14:26.974178 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:26.974097 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhx8j\" (UniqueName: \"kubernetes.io/projected/3254e35b-879b-4a29-84a7-41050fc425a2-kube-api-access-jhx8j\") pod \"e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6\" (UID: \"3254e35b-879b-4a29-84a7-41050fc425a2\") " pod="openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6" Jul 19 17:14:27.078527 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:27.078502 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6" Jul 19 17:14:27.216734 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:27.216649 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6"] Jul 19 17:14:27.219016 ip-10-0-134-203 kubenswrapper[2583]: W0719 17:14:27.218991 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3254e35b_879b_4a29_84a7_41050fc425a2.slice/crio-b75dd798ad942247e6434a1fbfa7dc114b841de666bdbd6cdff6934a339665d6 WatchSource:0}: Error finding container b75dd798ad942247e6434a1fbfa7dc114b841de666bdbd6cdff6934a339665d6: Status 404 returned error can't find the container with id b75dd798ad942247e6434a1fbfa7dc114b841de666bdbd6cdff6934a339665d6 Jul 19 17:14:27.221158 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:27.221118 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:14:27.221217 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:27.221189 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:14:27.221254 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:27.221217 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:14:28.076117 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:28.076074 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6" event={"ID":"3254e35b-879b-4a29-84a7-41050fc425a2","Type":"ContainerStarted","Data":"401e2a9cc48d33baca30df5351ef18940cd0d915b2e28113afb2f6254477fd82"} Jul 19 17:14:28.076508 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:28.076127 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6" event={"ID":"3254e35b-879b-4a29-84a7-41050fc425a2","Type":"ContainerStarted","Data":"b75dd798ad942247e6434a1fbfa7dc114b841de666bdbd6cdff6934a339665d6"} Jul 19 17:14:28.078667 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:28.078645 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6" Jul 19 17:14:28.083517 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:28.083490 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6" Jul 19 17:14:28.096704 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:28.096645 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6" podStartSLOduration=2.096626464 podStartE2EDuration="2.096626464s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 17:14:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 17:14:28.094441242 +0000 UTC m=+2163.081016725" watchObservedRunningTime="2026-07-19 17:14:28.096626464 +0000 UTC m=+2163.083201947" Jul 19 17:14:29.080881 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:29.080846 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6" Jul 19 17:14:29.081821 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:29.081802 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6" Jul 19 17:14:44.140599 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:44.139805 2583 patch_prober.go:28] interesting pod/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": read tcp 10.134.0.2:58060->10.134.0.33:15021: read: connection reset by peer" start-of-body= Jul 19 17:14:44.140599 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:44.139887 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6" podUID="3254e35b-879b-4a29-84a7-41050fc425a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": read tcp 10.134.0.2:58060->10.134.0.33:15021: read: connection reset by peer" Jul 19 17:14:45.160408 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:45.160375 2583 generic.go:356] "Generic (PLEG): container finished" podID="3254e35b-879b-4a29-84a7-41050fc425a2" containerID="401e2a9cc48d33baca30df5351ef18940cd0d915b2e28113afb2f6254477fd82" exitCode=0 Jul 19 17:14:45.160801 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:45.160445 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6" event={"ID":"3254e35b-879b-4a29-84a7-41050fc425a2","Type":"ContainerDied","Data":"401e2a9cc48d33baca30df5351ef18940cd0d915b2e28113afb2f6254477fd82"} Jul 19 17:14:45.160912 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:45.160896 2583 scope.go:117] "RemoveContainer" containerID="401e2a9cc48d33baca30df5351ef18940cd0d915b2e28113afb2f6254477fd82" Jul 19 17:14:45.162092 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:45.162038 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:14:45.162230 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:45.162144 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:14:45.162230 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:45.162194 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:14:46.167401 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:46.167362 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6" event={"ID":"3254e35b-879b-4a29-84a7-41050fc425a2","Type":"ContainerStarted","Data":"137f5b7ea1757501f71a977b7629e715d9e13f0224667f3e06933d56085272c4"} Jul 19 17:14:47.079473 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:47.079438 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6" Jul 19 17:14:47.079667 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:47.079645 2583 patch_prober.go:28] interesting pod/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body= Jul 19 17:14:47.079718 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:47.079691 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6" podUID="3254e35b-879b-4a29-84a7-41050fc425a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" Jul 19 17:14:48.079420 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:48.079382 2583 patch_prober.go:28] interesting pod/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body= Jul 19 17:14:48.079836 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:48.079451 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6" podUID="3254e35b-879b-4a29-84a7-41050fc425a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" Jul 19 17:14:49.079012 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:49.078971 2583 patch_prober.go:28] interesting pod/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body= Jul 19 17:14:49.079220 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:49.079039 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6" podUID="3254e35b-879b-4a29-84a7-41050fc425a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" Jul 19 17:14:50.083182 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:50.083145 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6" Jul 19 17:14:50.083565 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:50.083403 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6" Jul 19 17:14:50.084144 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:14:50.084129 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6" Jul 19 17:15:03.412629 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:03.412591 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6"] Jul 19 17:15:03.413212 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:03.412865 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6" podUID="3254e35b-879b-4a29-84a7-41050fc425a2" containerName="istio-proxy" containerID="cri-o://137f5b7ea1757501f71a977b7629e715d9e13f0224667f3e06933d56085272c4" gracePeriod=30 Jul 19 17:15:05.085974 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:05.085932 2583 patch_prober.go:28] interesting pod/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 19 17:15:05.086391 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:05.086009 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6" podUID="3254e35b-879b-4a29-84a7-41050fc425a2" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 19 17:15:08.665998 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:08.665969 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6" Jul 19 17:15:08.754714 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:08.754619 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3254e35b-879b-4a29-84a7-41050fc425a2-istio-data\") pod \"3254e35b-879b-4a29-84a7-41050fc425a2\" (UID: \"3254e35b-879b-4a29-84a7-41050fc425a2\") " Jul 19 17:15:08.754714 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:08.754665 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3254e35b-879b-4a29-84a7-41050fc425a2-istio-envoy\") pod \"3254e35b-879b-4a29-84a7-41050fc425a2\" (UID: \"3254e35b-879b-4a29-84a7-41050fc425a2\") " Jul 19 17:15:08.754714 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:08.754698 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3254e35b-879b-4a29-84a7-41050fc425a2-istiod-ca-cert\") pod \"3254e35b-879b-4a29-84a7-41050fc425a2\" (UID: \"3254e35b-879b-4a29-84a7-41050fc425a2\") " Jul 19 17:15:08.754995 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:08.754733 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jhx8j\" (UniqueName: \"kubernetes.io/projected/3254e35b-879b-4a29-84a7-41050fc425a2-kube-api-access-jhx8j\") pod \"3254e35b-879b-4a29-84a7-41050fc425a2\" (UID: \"3254e35b-879b-4a29-84a7-41050fc425a2\") " Jul 19 17:15:08.754995 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:08.754763 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3254e35b-879b-4a29-84a7-41050fc425a2-istio-podinfo\") pod \"3254e35b-879b-4a29-84a7-41050fc425a2\" (UID: \"3254e35b-879b-4a29-84a7-41050fc425a2\") " Jul 19 17:15:08.754995 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:08.754927 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3254e35b-879b-4a29-84a7-41050fc425a2-istio-token\") pod \"3254e35b-879b-4a29-84a7-41050fc425a2\" (UID: \"3254e35b-879b-4a29-84a7-41050fc425a2\") " Jul 19 17:15:08.754995 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:08.754988 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3254e35b-879b-4a29-84a7-41050fc425a2-workload-certs\") pod \"3254e35b-879b-4a29-84a7-41050fc425a2\" (UID: \"3254e35b-879b-4a29-84a7-41050fc425a2\") " Jul 19 17:15:08.755257 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:08.755042 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3254e35b-879b-4a29-84a7-41050fc425a2-credential-socket\") pod \"3254e35b-879b-4a29-84a7-41050fc425a2\" (UID: \"3254e35b-879b-4a29-84a7-41050fc425a2\") " Jul 19 17:15:08.755257 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:08.755083 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3254e35b-879b-4a29-84a7-41050fc425a2-workload-socket\") pod \"3254e35b-879b-4a29-84a7-41050fc425a2\" (UID: \"3254e35b-879b-4a29-84a7-41050fc425a2\") " Jul 19 17:15:08.755257 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:08.755214 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3254e35b-879b-4a29-84a7-41050fc425a2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3254e35b-879b-4a29-84a7-41050fc425a2" (UID: "3254e35b-879b-4a29-84a7-41050fc425a2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 19 17:15:08.755487 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:08.755445 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3254e35b-879b-4a29-84a7-41050fc425a2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3254e35b-879b-4a29-84a7-41050fc425a2" (UID: "3254e35b-879b-4a29-84a7-41050fc425a2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:15:08.755563 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:08.755552 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3254e35b-879b-4a29-84a7-41050fc425a2-workload-certs\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:15:08.755616 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:08.755569 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3254e35b-879b-4a29-84a7-41050fc425a2-istiod-ca-cert\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:15:08.756169 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:08.755776 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3254e35b-879b-4a29-84a7-41050fc425a2-istio-data" (OuterVolumeSpecName: "istio-data") pod "3254e35b-879b-4a29-84a7-41050fc425a2" (UID: "3254e35b-879b-4a29-84a7-41050fc425a2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:15:08.756169 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:08.755844 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3254e35b-879b-4a29-84a7-41050fc425a2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3254e35b-879b-4a29-84a7-41050fc425a2" (UID: "3254e35b-879b-4a29-84a7-41050fc425a2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:15:08.756169 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:08.755949 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3254e35b-879b-4a29-84a7-41050fc425a2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3254e35b-879b-4a29-84a7-41050fc425a2" (UID: "3254e35b-879b-4a29-84a7-41050fc425a2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:15:08.757128 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:08.757104 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3254e35b-879b-4a29-84a7-41050fc425a2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3254e35b-879b-4a29-84a7-41050fc425a2" (UID: "3254e35b-879b-4a29-84a7-41050fc425a2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:15:08.757230 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:08.757177 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3254e35b-879b-4a29-84a7-41050fc425a2-kube-api-access-jhx8j" (OuterVolumeSpecName: "kube-api-access-jhx8j") pod "3254e35b-879b-4a29-84a7-41050fc425a2" (UID: "3254e35b-879b-4a29-84a7-41050fc425a2"). InnerVolumeSpecName "kube-api-access-jhx8j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 19 17:15:08.757382 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:08.757365 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3254e35b-879b-4a29-84a7-41050fc425a2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3254e35b-879b-4a29-84a7-41050fc425a2" (UID: "3254e35b-879b-4a29-84a7-41050fc425a2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 19 17:15:08.757584 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:08.757559 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3254e35b-879b-4a29-84a7-41050fc425a2-istio-token" (OuterVolumeSpecName: "istio-token") pod "3254e35b-879b-4a29-84a7-41050fc425a2" (UID: "3254e35b-879b-4a29-84a7-41050fc425a2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 19 17:15:08.856387 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:08.856348 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3254e35b-879b-4a29-84a7-41050fc425a2-credential-socket\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:15:08.856387 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:08.856380 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3254e35b-879b-4a29-84a7-41050fc425a2-workload-socket\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:15:08.856387 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:08.856391 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3254e35b-879b-4a29-84a7-41050fc425a2-istio-data\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:15:08.856387 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:08.856400 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3254e35b-879b-4a29-84a7-41050fc425a2-istio-envoy\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:15:08.856661 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:08.856408 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jhx8j\" (UniqueName: \"kubernetes.io/projected/3254e35b-879b-4a29-84a7-41050fc425a2-kube-api-access-jhx8j\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:15:08.856661 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:08.856416 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3254e35b-879b-4a29-84a7-41050fc425a2-istio-podinfo\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:15:08.856661 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:08.856424 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3254e35b-879b-4a29-84a7-41050fc425a2-istio-token\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:15:09.273041 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:09.273003 2583 generic.go:356] "Generic (PLEG): container finished" podID="3254e35b-879b-4a29-84a7-41050fc425a2" containerID="137f5b7ea1757501f71a977b7629e715d9e13f0224667f3e06933d56085272c4" exitCode=0 Jul 19 17:15:09.273241 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:09.273047 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6" Jul 19 17:15:09.273241 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:09.273090 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6" event={"ID":"3254e35b-879b-4a29-84a7-41050fc425a2","Type":"ContainerDied","Data":"137f5b7ea1757501f71a977b7629e715d9e13f0224667f3e06933d56085272c4"} Jul 19 17:15:09.273241 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:09.273155 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6" event={"ID":"3254e35b-879b-4a29-84a7-41050fc425a2","Type":"ContainerDied","Data":"b75dd798ad942247e6434a1fbfa7dc114b841de666bdbd6cdff6934a339665d6"} Jul 19 17:15:09.273241 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:09.273180 2583 scope.go:117] "RemoveContainer" containerID="137f5b7ea1757501f71a977b7629e715d9e13f0224667f3e06933d56085272c4" Jul 19 17:15:09.282772 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:09.282753 2583 scope.go:117] "RemoveContainer" containerID="401e2a9cc48d33baca30df5351ef18940cd0d915b2e28113afb2f6254477fd82" Jul 19 17:15:09.290690 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:09.290669 2583 scope.go:117] "RemoveContainer" containerID="137f5b7ea1757501f71a977b7629e715d9e13f0224667f3e06933d56085272c4" Jul 19 17:15:09.290977 ip-10-0-134-203 kubenswrapper[2583]: E0719 17:15:09.290958 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"137f5b7ea1757501f71a977b7629e715d9e13f0224667f3e06933d56085272c4\": container with ID starting with 137f5b7ea1757501f71a977b7629e715d9e13f0224667f3e06933d56085272c4 not found: ID does not exist" containerID="137f5b7ea1757501f71a977b7629e715d9e13f0224667f3e06933d56085272c4" Jul 19 17:15:09.291027 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:09.290986 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"137f5b7ea1757501f71a977b7629e715d9e13f0224667f3e06933d56085272c4"} err="failed to get container status \"137f5b7ea1757501f71a977b7629e715d9e13f0224667f3e06933d56085272c4\": rpc error: code = NotFound desc = could not find container \"137f5b7ea1757501f71a977b7629e715d9e13f0224667f3e06933d56085272c4\": container with ID starting with 137f5b7ea1757501f71a977b7629e715d9e13f0224667f3e06933d56085272c4 not found: ID does not exist" Jul 19 17:15:09.291027 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:09.291005 2583 scope.go:117] "RemoveContainer" containerID="401e2a9cc48d33baca30df5351ef18940cd0d915b2e28113afb2f6254477fd82" Jul 19 17:15:09.291342 ip-10-0-134-203 kubenswrapper[2583]: E0719 17:15:09.291304 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"401e2a9cc48d33baca30df5351ef18940cd0d915b2e28113afb2f6254477fd82\": container with ID starting with 401e2a9cc48d33baca30df5351ef18940cd0d915b2e28113afb2f6254477fd82 not found: ID does not exist" containerID="401e2a9cc48d33baca30df5351ef18940cd0d915b2e28113afb2f6254477fd82" Jul 19 17:15:09.291451 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:09.291335 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"401e2a9cc48d33baca30df5351ef18940cd0d915b2e28113afb2f6254477fd82"} err="failed to get container status \"401e2a9cc48d33baca30df5351ef18940cd0d915b2e28113afb2f6254477fd82\": rpc error: code = NotFound desc = could not find container \"401e2a9cc48d33baca30df5351ef18940cd0d915b2e28113afb2f6254477fd82\": container with ID starting with 401e2a9cc48d33baca30df5351ef18940cd0d915b2e28113afb2f6254477fd82 not found: ID does not exist" Jul 19 17:15:09.293909 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:09.293883 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6"] Jul 19 17:15:09.297293 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:09.297271 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-cf60ac3a-openshift-default-5c84ffbf47-tmhs6"] Jul 19 17:15:09.602333 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:09.602299 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3254e35b-879b-4a29-84a7-41050fc425a2" path="/var/lib/kubelet/pods/3254e35b-879b-4a29-84a7-41050fc425a2/volumes" Jul 19 17:15:23.006467 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:23.006429 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx"] Jul 19 17:15:23.007191 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:23.006793 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3254e35b-879b-4a29-84a7-41050fc425a2" containerName="istio-proxy" Jul 19 17:15:23.007191 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:23.006808 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="3254e35b-879b-4a29-84a7-41050fc425a2" containerName="istio-proxy" Jul 19 17:15:23.007191 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:23.006899 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="3254e35b-879b-4a29-84a7-41050fc425a2" containerName="istio-proxy" Jul 19 17:15:23.007191 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:23.006917 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="8557b4d5-630f-4808-8a46-ee3e9337c3eb" containerName="istio-proxy" Jul 19 17:15:23.007191 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:23.006929 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="3254e35b-879b-4a29-84a7-41050fc425a2" containerName="istio-proxy" Jul 19 17:15:23.011528 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:23.011507 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx" Jul 19 17:15:23.013626 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:23.013609 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-68c77091-openshift-default-dockercfg-k4vvw\"" Jul 19 17:15:23.021597 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:23.021566 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx"] Jul 19 17:15:23.081945 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:23.081906 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8bf85d65-901e-44c7-b012-bb8c9ef5194b-istiod-ca-cert\") pod \"e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx\" (UID: \"8bf85d65-901e-44c7-b012-bb8c9ef5194b\") " pod="openshift-ingress/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx" Jul 19 17:15:23.082198 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:23.082009 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8bf85d65-901e-44c7-b012-bb8c9ef5194b-istio-envoy\") pod \"e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx\" (UID: \"8bf85d65-901e-44c7-b012-bb8c9ef5194b\") " pod="openshift-ingress/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx" Jul 19 17:15:23.082198 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:23.082047 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8bf85d65-901e-44c7-b012-bb8c9ef5194b-workload-certs\") pod \"e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx\" (UID: \"8bf85d65-901e-44c7-b012-bb8c9ef5194b\") " pod="openshift-ingress/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx" Jul 19 17:15:23.082198 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:23.082097 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8bf85d65-901e-44c7-b012-bb8c9ef5194b-istio-data\") pod \"e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx\" (UID: \"8bf85d65-901e-44c7-b012-bb8c9ef5194b\") " pod="openshift-ingress/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx" Jul 19 17:15:23.082198 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:23.082126 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8bf85d65-901e-44c7-b012-bb8c9ef5194b-istio-token\") pod \"e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx\" (UID: \"8bf85d65-901e-44c7-b012-bb8c9ef5194b\") " pod="openshift-ingress/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx" Jul 19 17:15:23.082198 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:23.082166 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmxbn\" (UniqueName: \"kubernetes.io/projected/8bf85d65-901e-44c7-b012-bb8c9ef5194b-kube-api-access-vmxbn\") pod \"e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx\" (UID: \"8bf85d65-901e-44c7-b012-bb8c9ef5194b\") " pod="openshift-ingress/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx" Jul 19 17:15:23.082427 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:23.082239 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8bf85d65-901e-44c7-b012-bb8c9ef5194b-istio-podinfo\") pod \"e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx\" (UID: \"8bf85d65-901e-44c7-b012-bb8c9ef5194b\") " pod="openshift-ingress/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx" Jul 19 17:15:23.082427 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:23.082276 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bf85d65-901e-44c7-b012-bb8c9ef5194b-workload-socket\") pod \"e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx\" (UID: \"8bf85d65-901e-44c7-b012-bb8c9ef5194b\") " pod="openshift-ingress/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx" Jul 19 17:15:23.082427 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:23.082299 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bf85d65-901e-44c7-b012-bb8c9ef5194b-credential-socket\") pod \"e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx\" (UID: \"8bf85d65-901e-44c7-b012-bb8c9ef5194b\") " pod="openshift-ingress/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx" Jul 19 17:15:23.183544 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:23.183493 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8bf85d65-901e-44c7-b012-bb8c9ef5194b-istio-token\") pod \"e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx\" (UID: \"8bf85d65-901e-44c7-b012-bb8c9ef5194b\") " pod="openshift-ingress/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx" Jul 19 17:15:23.183755 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:23.183558 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmxbn\" (UniqueName: \"kubernetes.io/projected/8bf85d65-901e-44c7-b012-bb8c9ef5194b-kube-api-access-vmxbn\") pod \"e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx\" (UID: \"8bf85d65-901e-44c7-b012-bb8c9ef5194b\") " pod="openshift-ingress/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx" Jul 19 17:15:23.183755 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:23.183610 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8bf85d65-901e-44c7-b012-bb8c9ef5194b-istio-podinfo\") pod \"e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx\" (UID: \"8bf85d65-901e-44c7-b012-bb8c9ef5194b\") " pod="openshift-ingress/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx" Jul 19 17:15:23.183755 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:23.183643 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bf85d65-901e-44c7-b012-bb8c9ef5194b-workload-socket\") pod \"e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx\" (UID: \"8bf85d65-901e-44c7-b012-bb8c9ef5194b\") " pod="openshift-ingress/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx" Jul 19 17:15:23.183755 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:23.183665 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bf85d65-901e-44c7-b012-bb8c9ef5194b-credential-socket\") pod \"e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx\" (UID: \"8bf85d65-901e-44c7-b012-bb8c9ef5194b\") " pod="openshift-ingress/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx" Jul 19 17:15:23.183755 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:23.183706 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8bf85d65-901e-44c7-b012-bb8c9ef5194b-istiod-ca-cert\") pod \"e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx\" (UID: \"8bf85d65-901e-44c7-b012-bb8c9ef5194b\") " pod="openshift-ingress/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx" Jul 19 17:15:23.184022 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:23.183773 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8bf85d65-901e-44c7-b012-bb8c9ef5194b-istio-envoy\") pod \"e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx\" (UID: \"8bf85d65-901e-44c7-b012-bb8c9ef5194b\") " pod="openshift-ingress/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx" Jul 19 17:15:23.184022 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:23.183812 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8bf85d65-901e-44c7-b012-bb8c9ef5194b-workload-certs\") pod \"e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx\" (UID: \"8bf85d65-901e-44c7-b012-bb8c9ef5194b\") " pod="openshift-ingress/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx" Jul 19 17:15:23.184212 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:23.184183 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bf85d65-901e-44c7-b012-bb8c9ef5194b-workload-socket\") pod \"e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx\" (UID: \"8bf85d65-901e-44c7-b012-bb8c9ef5194b\") " pod="openshift-ingress/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx" Jul 19 17:15:23.184212 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:23.184203 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8bf85d65-901e-44c7-b012-bb8c9ef5194b-workload-certs\") pod \"e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx\" (UID: \"8bf85d65-901e-44c7-b012-bb8c9ef5194b\") " pod="openshift-ingress/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx" Jul 19 17:15:23.184319 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:23.184255 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8bf85d65-901e-44c7-b012-bb8c9ef5194b-istio-data\") pod \"e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx\" (UID: \"8bf85d65-901e-44c7-b012-bb8c9ef5194b\") " pod="openshift-ingress/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx" Jul 19 17:15:23.184354 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:23.184264 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bf85d65-901e-44c7-b012-bb8c9ef5194b-credential-socket\") pod \"e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx\" (UID: \"8bf85d65-901e-44c7-b012-bb8c9ef5194b\") " pod="openshift-ingress/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx" Jul 19 17:15:23.184354 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:23.184334 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8bf85d65-901e-44c7-b012-bb8c9ef5194b-istio-data\") pod \"e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx\" (UID: \"8bf85d65-901e-44c7-b012-bb8c9ef5194b\") " pod="openshift-ingress/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx" Jul 19 17:15:23.184653 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:23.184637 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8bf85d65-901e-44c7-b012-bb8c9ef5194b-istiod-ca-cert\") pod \"e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx\" (UID: \"8bf85d65-901e-44c7-b012-bb8c9ef5194b\") " pod="openshift-ingress/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx" Jul 19 17:15:23.186094 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:23.186069 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8bf85d65-901e-44c7-b012-bb8c9ef5194b-istio-envoy\") pod \"e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx\" (UID: \"8bf85d65-901e-44c7-b012-bb8c9ef5194b\") " pod="openshift-ingress/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx" Jul 19 17:15:23.186223 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:23.186207 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8bf85d65-901e-44c7-b012-bb8c9ef5194b-istio-podinfo\") pod \"e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx\" (UID: \"8bf85d65-901e-44c7-b012-bb8c9ef5194b\") " pod="openshift-ingress/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx" Jul 19 17:15:23.190773 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:23.190752 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8bf85d65-901e-44c7-b012-bb8c9ef5194b-istio-token\") pod \"e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx\" (UID: \"8bf85d65-901e-44c7-b012-bb8c9ef5194b\") " pod="openshift-ingress/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx" Jul 19 17:15:23.190899 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:23.190841 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmxbn\" (UniqueName: \"kubernetes.io/projected/8bf85d65-901e-44c7-b012-bb8c9ef5194b-kube-api-access-vmxbn\") pod \"e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx\" (UID: \"8bf85d65-901e-44c7-b012-bb8c9ef5194b\") " pod="openshift-ingress/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx" Jul 19 17:15:23.323302 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:23.323215 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx" Jul 19 17:15:23.459758 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:23.459731 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx"] Jul 19 17:15:23.461933 ip-10-0-134-203 kubenswrapper[2583]: W0719 17:15:23.461901 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bf85d65_901e_44c7_b012_bb8c9ef5194b.slice/crio-7485ea85e084a123b3b7a19022cd08ccf5a8714d76fdf5a5ba6976b7e7718907 WatchSource:0}: Error finding container 7485ea85e084a123b3b7a19022cd08ccf5a8714d76fdf5a5ba6976b7e7718907: Status 404 returned error can't find the container with id 7485ea85e084a123b3b7a19022cd08ccf5a8714d76fdf5a5ba6976b7e7718907 Jul 19 17:15:23.463999 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:23.463967 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:15:23.464106 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:23.464051 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:15:23.464174 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:23.464111 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:15:24.344849 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:24.344810 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx" event={"ID":"8bf85d65-901e-44c7-b012-bb8c9ef5194b","Type":"ContainerStarted","Data":"e9e00511b2c9910133854bee85414ad07c70b9842d2cc82379ac97400c83fe72"} Jul 19 17:15:24.345259 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:24.344871 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx" event={"ID":"8bf85d65-901e-44c7-b012-bb8c9ef5194b","Type":"ContainerStarted","Data":"7485ea85e084a123b3b7a19022cd08ccf5a8714d76fdf5a5ba6976b7e7718907"} Jul 19 17:15:24.369794 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:24.364909 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx" podStartSLOduration=2.364887888 podStartE2EDuration="2.364887888s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 17:15:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 17:15:24.362711316 +0000 UTC m=+2219.349286804" watchObservedRunningTime="2026-07-19 17:15:24.364887888 +0000 UTC m=+2219.351463372" Jul 19 17:15:25.323757 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:25.323715 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx" Jul 19 17:15:25.328531 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:25.328500 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx" Jul 19 17:15:25.349602 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:25.349575 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx" Jul 19 17:15:25.350525 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:25.350503 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx" Jul 19 17:15:40.391043 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:40.390855 2583 patch_prober.go:28] interesting pod/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": read tcp 10.134.0.2:58348->10.134.0.34:15021: read: connection reset by peer" start-of-body= Jul 19 17:15:40.391043 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:40.390937 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx" podUID="8bf85d65-901e-44c7-b012-bb8c9ef5194b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": read tcp 10.134.0.2:58348->10.134.0.34:15021: read: connection reset by peer" Jul 19 17:15:41.428610 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:41.428573 2583 generic.go:356] "Generic (PLEG): container finished" podID="8bf85d65-901e-44c7-b012-bb8c9ef5194b" containerID="e9e00511b2c9910133854bee85414ad07c70b9842d2cc82379ac97400c83fe72" exitCode=0 Jul 19 17:15:41.429001 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:41.428644 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx" event={"ID":"8bf85d65-901e-44c7-b012-bb8c9ef5194b","Type":"ContainerDied","Data":"e9e00511b2c9910133854bee85414ad07c70b9842d2cc82379ac97400c83fe72"} Jul 19 17:15:41.429082 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:41.429050 2583 scope.go:117] "RemoveContainer" containerID="e9e00511b2c9910133854bee85414ad07c70b9842d2cc82379ac97400c83fe72" Jul 19 17:15:41.430132 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:41.430101 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:15:41.430220 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:41.430204 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:15:41.430260 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:41.430249 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:15:42.438323 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:42.438268 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx" event={"ID":"8bf85d65-901e-44c7-b012-bb8c9ef5194b","Type":"ContainerStarted","Data":"85e285110193c75d217ecd90654c0e06fccc81d83303335ba85c844fc07eb672"} Jul 19 17:15:43.323707 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:43.323675 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx" Jul 19 17:15:43.323921 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:43.323899 2583 patch_prober.go:28] interesting pod/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 19 17:15:43.324009 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:43.323948 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx" podUID="8bf85d65-901e-44c7-b012-bb8c9ef5194b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 19 17:15:44.323735 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:44.323697 2583 patch_prober.go:28] interesting pod/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 19 17:15:44.324146 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:44.323764 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx" podUID="8bf85d65-901e-44c7-b012-bb8c9ef5194b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 19 17:15:45.323779 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:45.323741 2583 patch_prober.go:28] interesting pod/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 19 17:15:45.324188 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:45.323801 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx" podUID="8bf85d65-901e-44c7-b012-bb8c9ef5194b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 19 17:15:46.330859 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:46.330828 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx" Jul 19 17:15:46.331343 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:46.331081 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx" Jul 19 17:15:46.331843 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:15:46.331826 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx" Jul 19 17:17:19.299115 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:17:19.299001 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx"] Jul 19 17:17:19.299654 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:17:19.299334 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx" podUID="8bf85d65-901e-44c7-b012-bb8c9ef5194b" containerName="istio-proxy" containerID="cri-o://85e285110193c75d217ecd90654c0e06fccc81d83303335ba85c844fc07eb672" gracePeriod=30 Jul 19 17:17:24.545771 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:17:24.545746 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx" Jul 19 17:17:24.627601 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:17:24.627570 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8bf85d65-901e-44c7-b012-bb8c9ef5194b-istiod-ca-cert\") pod \"8bf85d65-901e-44c7-b012-bb8c9ef5194b\" (UID: \"8bf85d65-901e-44c7-b012-bb8c9ef5194b\") " Jul 19 17:17:24.627774 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:17:24.627611 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bf85d65-901e-44c7-b012-bb8c9ef5194b-workload-socket\") pod \"8bf85d65-901e-44c7-b012-bb8c9ef5194b\" (UID: \"8bf85d65-901e-44c7-b012-bb8c9ef5194b\") " Jul 19 17:17:24.627774 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:17:24.627644 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8bf85d65-901e-44c7-b012-bb8c9ef5194b-istio-token\") pod \"8bf85d65-901e-44c7-b012-bb8c9ef5194b\" (UID: \"8bf85d65-901e-44c7-b012-bb8c9ef5194b\") " Jul 19 17:17:24.627774 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:17:24.627668 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bf85d65-901e-44c7-b012-bb8c9ef5194b-credential-socket\") pod \"8bf85d65-901e-44c7-b012-bb8c9ef5194b\" (UID: \"8bf85d65-901e-44c7-b012-bb8c9ef5194b\") " Jul 19 17:17:24.627917 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:17:24.627877 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8bf85d65-901e-44c7-b012-bb8c9ef5194b-istio-podinfo\") pod \"8bf85d65-901e-44c7-b012-bb8c9ef5194b\" (UID: \"8bf85d65-901e-44c7-b012-bb8c9ef5194b\") " Jul 19 17:17:24.627978 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:17:24.627934 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8bf85d65-901e-44c7-b012-bb8c9ef5194b-istio-envoy\") pod \"8bf85d65-901e-44c7-b012-bb8c9ef5194b\" (UID: \"8bf85d65-901e-44c7-b012-bb8c9ef5194b\") " Jul 19 17:17:24.627978 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:17:24.627879 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bf85d65-901e-44c7-b012-bb8c9ef5194b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8bf85d65-901e-44c7-b012-bb8c9ef5194b" (UID: "8bf85d65-901e-44c7-b012-bb8c9ef5194b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:17:24.627978 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:17:24.627964 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8bf85d65-901e-44c7-b012-bb8c9ef5194b-istio-data\") pod \"8bf85d65-901e-44c7-b012-bb8c9ef5194b\" (UID: \"8bf85d65-901e-44c7-b012-bb8c9ef5194b\") " Jul 19 17:17:24.627978 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:17:24.627949 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bf85d65-901e-44c7-b012-bb8c9ef5194b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8bf85d65-901e-44c7-b012-bb8c9ef5194b" (UID: "8bf85d65-901e-44c7-b012-bb8c9ef5194b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:17:24.628223 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:17:24.628031 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8bf85d65-901e-44c7-b012-bb8c9ef5194b-workload-certs\") pod \"8bf85d65-901e-44c7-b012-bb8c9ef5194b\" (UID: \"8bf85d65-901e-44c7-b012-bb8c9ef5194b\") " Jul 19 17:17:24.628223 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:17:24.628038 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bf85d65-901e-44c7-b012-bb8c9ef5194b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8bf85d65-901e-44c7-b012-bb8c9ef5194b" (UID: "8bf85d65-901e-44c7-b012-bb8c9ef5194b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 19 17:17:24.628223 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:17:24.628089 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vmxbn\" (UniqueName: \"kubernetes.io/projected/8bf85d65-901e-44c7-b012-bb8c9ef5194b-kube-api-access-vmxbn\") pod \"8bf85d65-901e-44c7-b012-bb8c9ef5194b\" (UID: \"8bf85d65-901e-44c7-b012-bb8c9ef5194b\") " Jul 19 17:17:24.628371 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:17:24.628331 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bf85d65-901e-44c7-b012-bb8c9ef5194b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8bf85d65-901e-44c7-b012-bb8c9ef5194b" (UID: "8bf85d65-901e-44c7-b012-bb8c9ef5194b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:17:24.628467 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:17:24.628450 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8bf85d65-901e-44c7-b012-bb8c9ef5194b-workload-certs\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:17:24.628526 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:17:24.628470 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8bf85d65-901e-44c7-b012-bb8c9ef5194b-istiod-ca-cert\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:17:24.628526 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:17:24.628482 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bf85d65-901e-44c7-b012-bb8c9ef5194b-workload-socket\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:17:24.628526 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:17:24.628494 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bf85d65-901e-44c7-b012-bb8c9ef5194b-credential-socket\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:17:24.628721 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:17:24.628693 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bf85d65-901e-44c7-b012-bb8c9ef5194b-istio-data" (OuterVolumeSpecName: "istio-data") pod "8bf85d65-901e-44c7-b012-bb8c9ef5194b" (UID: "8bf85d65-901e-44c7-b012-bb8c9ef5194b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:17:24.629967 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:17:24.629944 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bf85d65-901e-44c7-b012-bb8c9ef5194b-istio-token" (OuterVolumeSpecName: "istio-token") pod "8bf85d65-901e-44c7-b012-bb8c9ef5194b" (UID: "8bf85d65-901e-44c7-b012-bb8c9ef5194b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 19 17:17:24.630107 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:17:24.630040 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8bf85d65-901e-44c7-b012-bb8c9ef5194b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8bf85d65-901e-44c7-b012-bb8c9ef5194b" (UID: "8bf85d65-901e-44c7-b012-bb8c9ef5194b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 19 17:17:24.630305 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:17:24.630289 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bf85d65-901e-44c7-b012-bb8c9ef5194b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8bf85d65-901e-44c7-b012-bb8c9ef5194b" (UID: "8bf85d65-901e-44c7-b012-bb8c9ef5194b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:17:24.630429 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:17:24.630408 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bf85d65-901e-44c7-b012-bb8c9ef5194b-kube-api-access-vmxbn" (OuterVolumeSpecName: "kube-api-access-vmxbn") pod "8bf85d65-901e-44c7-b012-bb8c9ef5194b" (UID: "8bf85d65-901e-44c7-b012-bb8c9ef5194b"). InnerVolumeSpecName "kube-api-access-vmxbn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 19 17:17:24.729883 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:17:24.729849 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8bf85d65-901e-44c7-b012-bb8c9ef5194b-istio-token\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:17:24.729883 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:17:24.729879 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8bf85d65-901e-44c7-b012-bb8c9ef5194b-istio-podinfo\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:17:24.729883 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:17:24.729887 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8bf85d65-901e-44c7-b012-bb8c9ef5194b-istio-envoy\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:17:24.730144 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:17:24.729895 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8bf85d65-901e-44c7-b012-bb8c9ef5194b-istio-data\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:17:24.730144 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:17:24.729903 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vmxbn\" (UniqueName: \"kubernetes.io/projected/8bf85d65-901e-44c7-b012-bb8c9ef5194b-kube-api-access-vmxbn\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:17:24.908477 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:17:24.908386 2583 generic.go:356] "Generic (PLEG): container finished" podID="8bf85d65-901e-44c7-b012-bb8c9ef5194b" containerID="85e285110193c75d217ecd90654c0e06fccc81d83303335ba85c844fc07eb672" exitCode=0 Jul 19 17:17:24.908477 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:17:24.908433 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx" event={"ID":"8bf85d65-901e-44c7-b012-bb8c9ef5194b","Type":"ContainerDied","Data":"85e285110193c75d217ecd90654c0e06fccc81d83303335ba85c844fc07eb672"} Jul 19 17:17:24.908477 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:17:24.908452 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx" Jul 19 17:17:24.908477 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:17:24.908476 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx" event={"ID":"8bf85d65-901e-44c7-b012-bb8c9ef5194b","Type":"ContainerDied","Data":"7485ea85e084a123b3b7a19022cd08ccf5a8714d76fdf5a5ba6976b7e7718907"} Jul 19 17:17:24.908778 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:17:24.908497 2583 scope.go:117] "RemoveContainer" containerID="85e285110193c75d217ecd90654c0e06fccc81d83303335ba85c844fc07eb672" Jul 19 17:17:24.917896 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:17:24.917879 2583 scope.go:117] "RemoveContainer" containerID="e9e00511b2c9910133854bee85414ad07c70b9842d2cc82379ac97400c83fe72" Jul 19 17:17:24.925702 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:17:24.925681 2583 scope.go:117] "RemoveContainer" containerID="85e285110193c75d217ecd90654c0e06fccc81d83303335ba85c844fc07eb672" Jul 19 17:17:24.926008 ip-10-0-134-203 kubenswrapper[2583]: E0719 17:17:24.925989 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85e285110193c75d217ecd90654c0e06fccc81d83303335ba85c844fc07eb672\": container with ID starting with 85e285110193c75d217ecd90654c0e06fccc81d83303335ba85c844fc07eb672 not found: ID does not exist" containerID="85e285110193c75d217ecd90654c0e06fccc81d83303335ba85c844fc07eb672" Jul 19 17:17:24.926074 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:17:24.926019 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85e285110193c75d217ecd90654c0e06fccc81d83303335ba85c844fc07eb672"} err="failed to get container status \"85e285110193c75d217ecd90654c0e06fccc81d83303335ba85c844fc07eb672\": rpc error: code = NotFound desc = could not find container \"85e285110193c75d217ecd90654c0e06fccc81d83303335ba85c844fc07eb672\": container with ID starting with 85e285110193c75d217ecd90654c0e06fccc81d83303335ba85c844fc07eb672 not found: ID does not exist" Jul 19 17:17:24.926074 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:17:24.926037 2583 scope.go:117] "RemoveContainer" containerID="e9e00511b2c9910133854bee85414ad07c70b9842d2cc82379ac97400c83fe72" Jul 19 17:17:24.926279 ip-10-0-134-203 kubenswrapper[2583]: E0719 17:17:24.926261 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9e00511b2c9910133854bee85414ad07c70b9842d2cc82379ac97400c83fe72\": container with ID starting with e9e00511b2c9910133854bee85414ad07c70b9842d2cc82379ac97400c83fe72 not found: ID does not exist" containerID="e9e00511b2c9910133854bee85414ad07c70b9842d2cc82379ac97400c83fe72" Jul 19 17:17:24.926335 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:17:24.926287 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9e00511b2c9910133854bee85414ad07c70b9842d2cc82379ac97400c83fe72"} err="failed to get container status \"e9e00511b2c9910133854bee85414ad07c70b9842d2cc82379ac97400c83fe72\": rpc error: code = NotFound desc = could not find container \"e9e00511b2c9910133854bee85414ad07c70b9842d2cc82379ac97400c83fe72\": container with ID starting with e9e00511b2c9910133854bee85414ad07c70b9842d2cc82379ac97400c83fe72 not found: ID does not exist" Jul 19 17:17:24.929408 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:17:24.929386 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx"] Jul 19 17:17:24.933170 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:17:24.933149 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-68c77091-openshift-default-7b5f6fc975-vqmhx"] Jul 19 17:17:25.601473 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:17:25.601441 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8bf85d65-901e-44c7-b012-bb8c9ef5194b" path="/var/lib/kubelet/pods/8bf85d65-901e-44c7-b012-bb8c9ef5194b/volumes" Jul 19 17:18:14.751711 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:14.751678 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj"] Jul 19 17:18:14.752326 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:14.752206 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3254e35b-879b-4a29-84a7-41050fc425a2" containerName="istio-proxy" Jul 19 17:18:14.752326 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:14.752228 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="3254e35b-879b-4a29-84a7-41050fc425a2" containerName="istio-proxy" Jul 19 17:18:14.752326 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:14.752248 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bf85d65-901e-44c7-b012-bb8c9ef5194b" containerName="istio-proxy" Jul 19 17:18:14.752326 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:14.752259 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bf85d65-901e-44c7-b012-bb8c9ef5194b" containerName="istio-proxy" Jul 19 17:18:14.752538 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:14.752366 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bf85d65-901e-44c7-b012-bb8c9ef5194b" containerName="istio-proxy" Jul 19 17:18:14.755369 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:14.755347 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj" Jul 19 17:18:14.757745 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:14.757727 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-7dceef-openshift-default-dockercfg-rqwfd\"" Jul 19 17:18:14.766711 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:14.766684 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj"] Jul 19 17:18:14.893150 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:14.893115 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/026872c1-0aeb-498d-8d6f-58c9bec7f386-istio-token\") pod \"e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj\" (UID: \"026872c1-0aeb-498d-8d6f-58c9bec7f386\") " pod="openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj" Jul 19 17:18:14.893352 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:14.893160 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/026872c1-0aeb-498d-8d6f-58c9bec7f386-istio-podinfo\") pod \"e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj\" (UID: \"026872c1-0aeb-498d-8d6f-58c9bec7f386\") " pod="openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj" Jul 19 17:18:14.893352 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:14.893190 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/026872c1-0aeb-498d-8d6f-58c9bec7f386-workload-certs\") pod \"e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj\" (UID: \"026872c1-0aeb-498d-8d6f-58c9bec7f386\") " pod="openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj" Jul 19 17:18:14.893352 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:14.893259 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/026872c1-0aeb-498d-8d6f-58c9bec7f386-istiod-ca-cert\") pod \"e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj\" (UID: \"026872c1-0aeb-498d-8d6f-58c9bec7f386\") " pod="openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj" Jul 19 17:18:14.893352 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:14.893312 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/026872c1-0aeb-498d-8d6f-58c9bec7f386-istio-data\") pod \"e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj\" (UID: \"026872c1-0aeb-498d-8d6f-58c9bec7f386\") " pod="openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj" Jul 19 17:18:14.893631 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:14.893369 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/026872c1-0aeb-498d-8d6f-58c9bec7f386-workload-socket\") pod \"e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj\" (UID: \"026872c1-0aeb-498d-8d6f-58c9bec7f386\") " pod="openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj" Jul 19 17:18:14.893631 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:14.893401 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/026872c1-0aeb-498d-8d6f-58c9bec7f386-istio-envoy\") pod \"e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj\" (UID: \"026872c1-0aeb-498d-8d6f-58c9bec7f386\") " pod="openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj" Jul 19 17:18:14.893631 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:14.893431 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/026872c1-0aeb-498d-8d6f-58c9bec7f386-credential-socket\") pod \"e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj\" (UID: \"026872c1-0aeb-498d-8d6f-58c9bec7f386\") " pod="openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj" Jul 19 17:18:14.893631 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:14.893514 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2fw6\" (UniqueName: \"kubernetes.io/projected/026872c1-0aeb-498d-8d6f-58c9bec7f386-kube-api-access-r2fw6\") pod \"e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj\" (UID: \"026872c1-0aeb-498d-8d6f-58c9bec7f386\") " pod="openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj" Jul 19 17:18:14.994912 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:14.994875 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/026872c1-0aeb-498d-8d6f-58c9bec7f386-workload-socket\") pod \"e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj\" (UID: \"026872c1-0aeb-498d-8d6f-58c9bec7f386\") " pod="openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj" Jul 19 17:18:14.994912 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:14.994914 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/026872c1-0aeb-498d-8d6f-58c9bec7f386-istio-envoy\") pod \"e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj\" (UID: \"026872c1-0aeb-498d-8d6f-58c9bec7f386\") " pod="openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj" Jul 19 17:18:14.995148 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:14.994934 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/026872c1-0aeb-498d-8d6f-58c9bec7f386-credential-socket\") pod \"e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj\" (UID: \"026872c1-0aeb-498d-8d6f-58c9bec7f386\") " pod="openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj" Jul 19 17:18:14.995148 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:14.994981 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2fw6\" (UniqueName: \"kubernetes.io/projected/026872c1-0aeb-498d-8d6f-58c9bec7f386-kube-api-access-r2fw6\") pod \"e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj\" (UID: \"026872c1-0aeb-498d-8d6f-58c9bec7f386\") " pod="openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj" Jul 19 17:18:14.995148 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:14.995016 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/026872c1-0aeb-498d-8d6f-58c9bec7f386-istio-token\") pod \"e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj\" (UID: \"026872c1-0aeb-498d-8d6f-58c9bec7f386\") " pod="openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj" Jul 19 17:18:14.995148 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:14.995048 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/026872c1-0aeb-498d-8d6f-58c9bec7f386-istio-podinfo\") pod \"e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj\" (UID: \"026872c1-0aeb-498d-8d6f-58c9bec7f386\") " pod="openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj" Jul 19 17:18:14.995367 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:14.995209 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/026872c1-0aeb-498d-8d6f-58c9bec7f386-workload-certs\") pod \"e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj\" (UID: \"026872c1-0aeb-498d-8d6f-58c9bec7f386\") " pod="openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj" Jul 19 17:18:14.995367 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:14.995264 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/026872c1-0aeb-498d-8d6f-58c9bec7f386-istiod-ca-cert\") pod \"e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj\" (UID: \"026872c1-0aeb-498d-8d6f-58c9bec7f386\") " pod="openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj" Jul 19 17:18:14.995367 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:14.995306 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/026872c1-0aeb-498d-8d6f-58c9bec7f386-istio-data\") pod \"e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj\" (UID: \"026872c1-0aeb-498d-8d6f-58c9bec7f386\") " pod="openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj" Jul 19 17:18:14.995519 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:14.995383 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/026872c1-0aeb-498d-8d6f-58c9bec7f386-credential-socket\") pod \"e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj\" (UID: \"026872c1-0aeb-498d-8d6f-58c9bec7f386\") " pod="openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj" Jul 19 17:18:14.995519 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:14.995412 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/026872c1-0aeb-498d-8d6f-58c9bec7f386-workload-socket\") pod \"e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj\" (UID: \"026872c1-0aeb-498d-8d6f-58c9bec7f386\") " pod="openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj" Jul 19 17:18:14.995627 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:14.995533 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/026872c1-0aeb-498d-8d6f-58c9bec7f386-workload-certs\") pod \"e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj\" (UID: \"026872c1-0aeb-498d-8d6f-58c9bec7f386\") " pod="openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj" Jul 19 17:18:14.995738 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:14.995714 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/026872c1-0aeb-498d-8d6f-58c9bec7f386-istio-data\") pod \"e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj\" (UID: \"026872c1-0aeb-498d-8d6f-58c9bec7f386\") " pod="openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj" Jul 19 17:18:14.996168 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:14.996146 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/026872c1-0aeb-498d-8d6f-58c9bec7f386-istiod-ca-cert\") pod \"e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj\" (UID: \"026872c1-0aeb-498d-8d6f-58c9bec7f386\") " pod="openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj" Jul 19 17:18:14.997215 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:14.997195 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/026872c1-0aeb-498d-8d6f-58c9bec7f386-istio-envoy\") pod \"e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj\" (UID: \"026872c1-0aeb-498d-8d6f-58c9bec7f386\") " pod="openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj" Jul 19 17:18:14.997514 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:14.997495 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/026872c1-0aeb-498d-8d6f-58c9bec7f386-istio-podinfo\") pod \"e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj\" (UID: \"026872c1-0aeb-498d-8d6f-58c9bec7f386\") " pod="openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj" Jul 19 17:18:15.002706 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:15.002660 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/026872c1-0aeb-498d-8d6f-58c9bec7f386-istio-token\") pod \"e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj\" (UID: \"026872c1-0aeb-498d-8d6f-58c9bec7f386\") " pod="openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj" Jul 19 17:18:15.003052 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:15.003032 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2fw6\" (UniqueName: \"kubernetes.io/projected/026872c1-0aeb-498d-8d6f-58c9bec7f386-kube-api-access-r2fw6\") pod \"e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj\" (UID: \"026872c1-0aeb-498d-8d6f-58c9bec7f386\") " pod="openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj" Jul 19 17:18:15.065120 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:15.065088 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj" Jul 19 17:18:15.201955 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:15.201928 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj"] Jul 19 17:18:15.204124 ip-10-0-134-203 kubenswrapper[2583]: W0719 17:18:15.204094 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod026872c1_0aeb_498d_8d6f_58c9bec7f386.slice/crio-e2bc00fb2a4d7d7cf0db9d88972e3dacea1a80715e21d3ca1462903853d5ba38 WatchSource:0}: Error finding container e2bc00fb2a4d7d7cf0db9d88972e3dacea1a80715e21d3ca1462903853d5ba38: Status 404 returned error can't find the container with id e2bc00fb2a4d7d7cf0db9d88972e3dacea1a80715e21d3ca1462903853d5ba38 Jul 19 17:18:15.206337 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:15.206301 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:18:15.206451 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:15.206385 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:18:15.206451 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:15.206429 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:18:16.145302 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:16.145263 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj" event={"ID":"026872c1-0aeb-498d-8d6f-58c9bec7f386","Type":"ContainerStarted","Data":"5a34c788424bb33bc2a51f056026f151654d280ba6339d1c566ef43477a71a34"} Jul 19 17:18:16.145653 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:16.145324 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj" event={"ID":"026872c1-0aeb-498d-8d6f-58c9bec7f386","Type":"ContainerStarted","Data":"e2bc00fb2a4d7d7cf0db9d88972e3dacea1a80715e21d3ca1462903853d5ba38"} Jul 19 17:18:16.164745 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:16.164689 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj" podStartSLOduration=2.164675704 podStartE2EDuration="2.164675704s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 17:18:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 17:18:16.162301289 +0000 UTC m=+2391.148876773" watchObservedRunningTime="2026-07-19 17:18:16.164675704 +0000 UTC m=+2391.151251248" Jul 19 17:18:17.065887 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:17.065850 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj" Jul 19 17:18:17.070716 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:17.070689 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj" Jul 19 17:18:17.151197 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:17.151166 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj" Jul 19 17:18:17.152166 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:17.152149 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj" Jul 19 17:18:23.489446 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:23.489410 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr"] Jul 19 17:18:23.489916 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:23.489808 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bf85d65-901e-44c7-b012-bb8c9ef5194b" containerName="istio-proxy" Jul 19 17:18:23.489916 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:23.489821 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bf85d65-901e-44c7-b012-bb8c9ef5194b" containerName="istio-proxy" Jul 19 17:18:23.489916 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:23.489901 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bf85d65-901e-44c7-b012-bb8c9ef5194b" containerName="istio-proxy" Jul 19 17:18:23.493424 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:23.493394 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr" Jul 19 17:18:23.495429 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:23.495405 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-ec4179-openshift-default-dockercfg-4bcbs\"" Jul 19 17:18:23.502986 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:23.502960 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr"] Jul 19 17:18:23.578202 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:23.578167 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8p2c4\" (UniqueName: \"kubernetes.io/projected/1179f6e6-4830-4c84-81a4-c64376a0a267-kube-api-access-8p2c4\") pod \"e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr\" (UID: \"1179f6e6-4830-4c84-81a4-c64376a0a267\") " pod="openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr" Jul 19 17:18:23.578410 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:23.578220 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1179f6e6-4830-4c84-81a4-c64376a0a267-istio-podinfo\") pod \"e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr\" (UID: \"1179f6e6-4830-4c84-81a4-c64376a0a267\") " pod="openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr" Jul 19 17:18:23.578410 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:23.578243 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1179f6e6-4830-4c84-81a4-c64376a0a267-istio-envoy\") pod \"e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr\" (UID: \"1179f6e6-4830-4c84-81a4-c64376a0a267\") " pod="openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr" Jul 19 17:18:23.578410 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:23.578328 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1179f6e6-4830-4c84-81a4-c64376a0a267-credential-socket\") pod \"e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr\" (UID: \"1179f6e6-4830-4c84-81a4-c64376a0a267\") " pod="openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr" Jul 19 17:18:23.578571 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:23.578408 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1179f6e6-4830-4c84-81a4-c64376a0a267-istiod-ca-cert\") pod \"e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr\" (UID: \"1179f6e6-4830-4c84-81a4-c64376a0a267\") " pod="openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr" Jul 19 17:18:23.578571 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:23.578445 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1179f6e6-4830-4c84-81a4-c64376a0a267-istio-token\") pod \"e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr\" (UID: \"1179f6e6-4830-4c84-81a4-c64376a0a267\") " pod="openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr" Jul 19 17:18:23.578571 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:23.578479 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1179f6e6-4830-4c84-81a4-c64376a0a267-istio-data\") pod \"e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr\" (UID: \"1179f6e6-4830-4c84-81a4-c64376a0a267\") " pod="openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr" Jul 19 17:18:23.578571 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:23.578555 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1179f6e6-4830-4c84-81a4-c64376a0a267-workload-socket\") pod \"e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr\" (UID: \"1179f6e6-4830-4c84-81a4-c64376a0a267\") " pod="openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr" Jul 19 17:18:23.578724 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:23.578608 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1179f6e6-4830-4c84-81a4-c64376a0a267-workload-certs\") pod \"e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr\" (UID: \"1179f6e6-4830-4c84-81a4-c64376a0a267\") " pod="openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr" Jul 19 17:18:23.679600 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:23.679551 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8p2c4\" (UniqueName: \"kubernetes.io/projected/1179f6e6-4830-4c84-81a4-c64376a0a267-kube-api-access-8p2c4\") pod \"e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr\" (UID: \"1179f6e6-4830-4c84-81a4-c64376a0a267\") " pod="openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr" Jul 19 17:18:23.679830 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:23.679619 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1179f6e6-4830-4c84-81a4-c64376a0a267-istio-podinfo\") pod \"e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr\" (UID: \"1179f6e6-4830-4c84-81a4-c64376a0a267\") " pod="openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr" Jul 19 17:18:23.679830 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:23.679642 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1179f6e6-4830-4c84-81a4-c64376a0a267-istio-envoy\") pod \"e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr\" (UID: \"1179f6e6-4830-4c84-81a4-c64376a0a267\") " pod="openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr" Jul 19 17:18:23.679830 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:23.679675 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1179f6e6-4830-4c84-81a4-c64376a0a267-credential-socket\") pod \"e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr\" (UID: \"1179f6e6-4830-4c84-81a4-c64376a0a267\") " pod="openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr" Jul 19 17:18:23.679830 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:23.679725 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1179f6e6-4830-4c84-81a4-c64376a0a267-istiod-ca-cert\") pod \"e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr\" (UID: \"1179f6e6-4830-4c84-81a4-c64376a0a267\") " pod="openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr" Jul 19 17:18:23.679830 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:23.679763 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1179f6e6-4830-4c84-81a4-c64376a0a267-istio-token\") pod \"e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr\" (UID: \"1179f6e6-4830-4c84-81a4-c64376a0a267\") " pod="openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr" Jul 19 17:18:23.679830 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:23.679792 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1179f6e6-4830-4c84-81a4-c64376a0a267-istio-data\") pod \"e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr\" (UID: \"1179f6e6-4830-4c84-81a4-c64376a0a267\") " pod="openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr" Jul 19 17:18:23.680197 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:23.679843 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1179f6e6-4830-4c84-81a4-c64376a0a267-workload-socket\") pod \"e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr\" (UID: \"1179f6e6-4830-4c84-81a4-c64376a0a267\") " pod="openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr" Jul 19 17:18:23.680197 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:23.679895 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1179f6e6-4830-4c84-81a4-c64376a0a267-workload-certs\") pod \"e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr\" (UID: \"1179f6e6-4830-4c84-81a4-c64376a0a267\") " pod="openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr" Jul 19 17:18:23.680297 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:23.680270 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1179f6e6-4830-4c84-81a4-c64376a0a267-credential-socket\") pod \"e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr\" (UID: \"1179f6e6-4830-4c84-81a4-c64376a0a267\") " pod="openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr" Jul 19 17:18:23.680360 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:23.680286 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1179f6e6-4830-4c84-81a4-c64376a0a267-istio-data\") pod \"e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr\" (UID: \"1179f6e6-4830-4c84-81a4-c64376a0a267\") " pod="openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr" Jul 19 17:18:23.680417 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:23.680387 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1179f6e6-4830-4c84-81a4-c64376a0a267-workload-certs\") pod \"e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr\" (UID: \"1179f6e6-4830-4c84-81a4-c64376a0a267\") " pod="openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr" Jul 19 17:18:23.680519 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:23.680498 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1179f6e6-4830-4c84-81a4-c64376a0a267-workload-socket\") pod \"e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr\" (UID: \"1179f6e6-4830-4c84-81a4-c64376a0a267\") " pod="openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr" Jul 19 17:18:23.680622 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:23.680605 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1179f6e6-4830-4c84-81a4-c64376a0a267-istiod-ca-cert\") pod \"e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr\" (UID: \"1179f6e6-4830-4c84-81a4-c64376a0a267\") " pod="openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr" Jul 19 17:18:23.682028 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:23.682007 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1179f6e6-4830-4c84-81a4-c64376a0a267-istio-envoy\") pod \"e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr\" (UID: \"1179f6e6-4830-4c84-81a4-c64376a0a267\") " pod="openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr" Jul 19 17:18:23.682200 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:23.682185 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1179f6e6-4830-4c84-81a4-c64376a0a267-istio-podinfo\") pod \"e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr\" (UID: \"1179f6e6-4830-4c84-81a4-c64376a0a267\") " pod="openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr" Jul 19 17:18:23.686669 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:23.686648 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1179f6e6-4830-4c84-81a4-c64376a0a267-istio-token\") pod \"e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr\" (UID: \"1179f6e6-4830-4c84-81a4-c64376a0a267\") " pod="openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr" Jul 19 17:18:23.686898 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:23.686881 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8p2c4\" (UniqueName: \"kubernetes.io/projected/1179f6e6-4830-4c84-81a4-c64376a0a267-kube-api-access-8p2c4\") pod \"e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr\" (UID: \"1179f6e6-4830-4c84-81a4-c64376a0a267\") " pod="openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr" Jul 19 17:18:23.806244 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:23.806092 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr" Jul 19 17:18:23.983240 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:23.983209 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr"] Jul 19 17:18:23.984776 ip-10-0-134-203 kubenswrapper[2583]: W0719 17:18:23.984750 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1179f6e6_4830_4c84_81a4_c64376a0a267.slice/crio-a141bcb9f561f5ad0c9f40f51101698416cbc6712190a5c6125e7660cc1308a2 WatchSource:0}: Error finding container a141bcb9f561f5ad0c9f40f51101698416cbc6712190a5c6125e7660cc1308a2: Status 404 returned error can't find the container with id a141bcb9f561f5ad0c9f40f51101698416cbc6712190a5c6125e7660cc1308a2 Jul 19 17:18:23.987023 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:23.986985 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:18:23.987132 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:23.987084 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:18:23.987205 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:23.987129 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:18:24.185182 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:24.185147 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr" event={"ID":"1179f6e6-4830-4c84-81a4-c64376a0a267","Type":"ContainerStarted","Data":"89f333abd5f9a8c0c1272ec0f4e41ff8e3d6beb46cdadd710b16cdf628a137be"} Jul 19 17:18:24.185375 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:24.185211 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr" event={"ID":"1179f6e6-4830-4c84-81a4-c64376a0a267","Type":"ContainerStarted","Data":"a141bcb9f561f5ad0c9f40f51101698416cbc6712190a5c6125e7660cc1308a2"} Jul 19 17:18:24.202553 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:24.202501 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr" podStartSLOduration=1.202484662 podStartE2EDuration="1.202484662s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 17:18:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 17:18:24.201352956 +0000 UTC m=+2399.187928442" watchObservedRunningTime="2026-07-19 17:18:24.202484662 +0000 UTC m=+2399.189060146" Jul 19 17:18:24.806944 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:24.806907 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr" Jul 19 17:18:25.811800 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:25.811769 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr" Jul 19 17:18:26.195553 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:26.195524 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr" Jul 19 17:18:26.196464 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:26.196441 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr" Jul 19 17:18:32.152786 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:32.152751 2583 patch_prober.go:28] interesting pod/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jul 19 17:18:32.153229 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:32.152817 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj" podUID="026872c1-0aeb-498d-8d6f-58c9bec7f386" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jul 19 17:18:32.224987 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:32.224951 2583 generic.go:356] "Generic (PLEG): container finished" podID="026872c1-0aeb-498d-8d6f-58c9bec7f386" containerID="5a34c788424bb33bc2a51f056026f151654d280ba6339d1c566ef43477a71a34" exitCode=0 Jul 19 17:18:32.225166 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:32.225023 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj" event={"ID":"026872c1-0aeb-498d-8d6f-58c9bec7f386","Type":"ContainerDied","Data":"5a34c788424bb33bc2a51f056026f151654d280ba6339d1c566ef43477a71a34"} Jul 19 17:18:32.225487 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:32.225473 2583 scope.go:117] "RemoveContainer" containerID="5a34c788424bb33bc2a51f056026f151654d280ba6339d1c566ef43477a71a34" Jul 19 17:18:32.226658 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:32.226629 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:18:32.226734 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:32.226718 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:18:32.226771 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:32.226762 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:18:33.234563 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:33.234512 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj" event={"ID":"026872c1-0aeb-498d-8d6f-58c9bec7f386","Type":"ContainerStarted","Data":"fdba85bc0476ba86b3f4cb31c0db3355dd77d0915a09ea4fe93e425524e6828b"} Jul 19 17:18:34.065783 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:34.065687 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj" Jul 19 17:18:34.066157 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:34.066130 2583 patch_prober.go:28] interesting pod/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jul 19 17:18:34.066341 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:34.066316 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj" podUID="026872c1-0aeb-498d-8d6f-58c9bec7f386" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jul 19 17:18:35.065994 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:35.065956 2583 patch_prober.go:28] interesting pod/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jul 19 17:18:35.066681 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:35.066647 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj" podUID="026872c1-0aeb-498d-8d6f-58c9bec7f386" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jul 19 17:18:36.066410 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:36.066375 2583 patch_prober.go:28] interesting pod/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jul 19 17:18:36.067074 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:36.067012 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj" podUID="026872c1-0aeb-498d-8d6f-58c9bec7f386" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jul 19 17:18:37.066457 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:37.066360 2583 patch_prober.go:28] interesting pod/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jul 19 17:18:37.066457 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:37.066420 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj" podUID="026872c1-0aeb-498d-8d6f-58c9bec7f386" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jul 19 17:18:38.070189 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:38.070160 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj" Jul 19 17:18:38.070637 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:38.070418 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj" Jul 19 17:18:38.071093 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:38.071074 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj" Jul 19 17:18:41.197435 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:41.197396 2583 patch_prober.go:28] interesting pod/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Jul 19 17:18:41.197914 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:41.197465 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr" podUID="1179f6e6-4830-4c84-81a4-c64376a0a267" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Jul 19 17:18:41.280641 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:41.280609 2583 generic.go:356] "Generic (PLEG): container finished" podID="1179f6e6-4830-4c84-81a4-c64376a0a267" containerID="89f333abd5f9a8c0c1272ec0f4e41ff8e3d6beb46cdadd710b16cdf628a137be" exitCode=0 Jul 19 17:18:41.280813 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:41.280683 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr" event={"ID":"1179f6e6-4830-4c84-81a4-c64376a0a267","Type":"ContainerDied","Data":"89f333abd5f9a8c0c1272ec0f4e41ff8e3d6beb46cdadd710b16cdf628a137be"} Jul 19 17:18:41.281186 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:41.281171 2583 scope.go:117] "RemoveContainer" containerID="89f333abd5f9a8c0c1272ec0f4e41ff8e3d6beb46cdadd710b16cdf628a137be" Jul 19 17:18:41.282239 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:41.282208 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:18:41.282315 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:41.282296 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:18:41.282356 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:41.282339 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:18:42.286877 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:42.286844 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr" event={"ID":"1179f6e6-4830-4c84-81a4-c64376a0a267","Type":"ContainerStarted","Data":"1a3d099b9baa3321666f1bb7ca74489e95ee213b0810e276c9912cf41ec562ce"} Jul 19 17:18:42.807260 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:42.807219 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr" Jul 19 17:18:42.807466 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:42.807421 2583 patch_prober.go:28] interesting pod/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Jul 19 17:18:42.807548 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:42.807480 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr" podUID="1179f6e6-4830-4c84-81a4-c64376a0a267" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Jul 19 17:18:43.806988 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:43.806950 2583 patch_prober.go:28] interesting pod/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Jul 19 17:18:43.807417 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:43.807023 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr" podUID="1179f6e6-4830-4c84-81a4-c64376a0a267" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Jul 19 17:18:44.807393 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:44.807360 2583 patch_prober.go:28] interesting pod/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Jul 19 17:18:44.807824 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:44.807414 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr" podUID="1179f6e6-4830-4c84-81a4-c64376a0a267" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Jul 19 17:18:45.806948 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:45.806912 2583 patch_prober.go:28] interesting pod/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Jul 19 17:18:45.807184 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:45.806978 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr" podUID="1179f6e6-4830-4c84-81a4-c64376a0a267" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Jul 19 17:18:46.810831 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:46.810804 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr" Jul 19 17:18:46.811284 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:46.811092 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr" Jul 19 17:18:46.811668 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:18:46.811653 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr" Jul 19 17:19:57.542488 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:19:57.542449 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj"] Jul 19 17:19:57.543465 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:19:57.542748 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj" podUID="026872c1-0aeb-498d-8d6f-58c9bec7f386" containerName="istio-proxy" containerID="cri-o://fdba85bc0476ba86b3f4cb31c0db3355dd77d0915a09ea4fe93e425524e6828b" gracePeriod=30 Jul 19 17:20:02.672782 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:02.672740 2583 generic.go:356] "Generic (PLEG): container finished" podID="026872c1-0aeb-498d-8d6f-58c9bec7f386" containerID="fdba85bc0476ba86b3f4cb31c0db3355dd77d0915a09ea4fe93e425524e6828b" exitCode=0 Jul 19 17:20:02.673239 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:02.672812 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj" event={"ID":"026872c1-0aeb-498d-8d6f-58c9bec7f386","Type":"ContainerDied","Data":"fdba85bc0476ba86b3f4cb31c0db3355dd77d0915a09ea4fe93e425524e6828b"} Jul 19 17:20:02.673239 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:02.672898 2583 scope.go:117] "RemoveContainer" containerID="5a34c788424bb33bc2a51f056026f151654d280ba6339d1c566ef43477a71a34" Jul 19 17:20:02.789044 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:02.789016 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj" Jul 19 17:20:02.852455 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:02.852426 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/026872c1-0aeb-498d-8d6f-58c9bec7f386-workload-socket\") pod \"026872c1-0aeb-498d-8d6f-58c9bec7f386\" (UID: \"026872c1-0aeb-498d-8d6f-58c9bec7f386\") " Jul 19 17:20:02.852745 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:02.852474 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/026872c1-0aeb-498d-8d6f-58c9bec7f386-istio-envoy\") pod \"026872c1-0aeb-498d-8d6f-58c9bec7f386\" (UID: \"026872c1-0aeb-498d-8d6f-58c9bec7f386\") " Jul 19 17:20:02.852745 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:02.852507 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/026872c1-0aeb-498d-8d6f-58c9bec7f386-credential-socket\") pod \"026872c1-0aeb-498d-8d6f-58c9bec7f386\" (UID: \"026872c1-0aeb-498d-8d6f-58c9bec7f386\") " Jul 19 17:20:02.852745 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:02.852566 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/026872c1-0aeb-498d-8d6f-58c9bec7f386-istio-podinfo\") pod \"026872c1-0aeb-498d-8d6f-58c9bec7f386\" (UID: \"026872c1-0aeb-498d-8d6f-58c9bec7f386\") " Jul 19 17:20:02.852745 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:02.852612 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/026872c1-0aeb-498d-8d6f-58c9bec7f386-istio-data\") pod \"026872c1-0aeb-498d-8d6f-58c9bec7f386\" (UID: \"026872c1-0aeb-498d-8d6f-58c9bec7f386\") " Jul 19 17:20:02.852745 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:02.852656 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/026872c1-0aeb-498d-8d6f-58c9bec7f386-istio-token\") pod \"026872c1-0aeb-498d-8d6f-58c9bec7f386\" (UID: \"026872c1-0aeb-498d-8d6f-58c9bec7f386\") " Jul 19 17:20:02.852745 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:02.852743 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/026872c1-0aeb-498d-8d6f-58c9bec7f386-workload-certs\") pod \"026872c1-0aeb-498d-8d6f-58c9bec7f386\" (UID: \"026872c1-0aeb-498d-8d6f-58c9bec7f386\") " Jul 19 17:20:02.853070 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:02.852743 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/026872c1-0aeb-498d-8d6f-58c9bec7f386-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "026872c1-0aeb-498d-8d6f-58c9bec7f386" (UID: "026872c1-0aeb-498d-8d6f-58c9bec7f386"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:20:02.853070 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:02.852781 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r2fw6\" (UniqueName: \"kubernetes.io/projected/026872c1-0aeb-498d-8d6f-58c9bec7f386-kube-api-access-r2fw6\") pod \"026872c1-0aeb-498d-8d6f-58c9bec7f386\" (UID: \"026872c1-0aeb-498d-8d6f-58c9bec7f386\") " Jul 19 17:20:02.853070 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:02.852815 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/026872c1-0aeb-498d-8d6f-58c9bec7f386-istiod-ca-cert\") pod \"026872c1-0aeb-498d-8d6f-58c9bec7f386\" (UID: \"026872c1-0aeb-498d-8d6f-58c9bec7f386\") " Jul 19 17:20:02.853070 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:02.852853 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/026872c1-0aeb-498d-8d6f-58c9bec7f386-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "026872c1-0aeb-498d-8d6f-58c9bec7f386" (UID: "026872c1-0aeb-498d-8d6f-58c9bec7f386"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:20:02.853292 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:02.853183 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/026872c1-0aeb-498d-8d6f-58c9bec7f386-workload-socket\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:20:02.853292 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:02.853202 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/026872c1-0aeb-498d-8d6f-58c9bec7f386-credential-socket\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:20:02.853292 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:02.853219 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/026872c1-0aeb-498d-8d6f-58c9bec7f386-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "026872c1-0aeb-498d-8d6f-58c9bec7f386" (UID: "026872c1-0aeb-498d-8d6f-58c9bec7f386"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:20:02.853411 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:02.853326 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/026872c1-0aeb-498d-8d6f-58c9bec7f386-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "026872c1-0aeb-498d-8d6f-58c9bec7f386" (UID: "026872c1-0aeb-498d-8d6f-58c9bec7f386"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 19 17:20:02.853648 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:02.853621 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/026872c1-0aeb-498d-8d6f-58c9bec7f386-istio-data" (OuterVolumeSpecName: "istio-data") pod "026872c1-0aeb-498d-8d6f-58c9bec7f386" (UID: "026872c1-0aeb-498d-8d6f-58c9bec7f386"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:20:02.854751 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:02.854726 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/026872c1-0aeb-498d-8d6f-58c9bec7f386-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "026872c1-0aeb-498d-8d6f-58c9bec7f386" (UID: "026872c1-0aeb-498d-8d6f-58c9bec7f386"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:20:02.854968 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:02.854951 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/026872c1-0aeb-498d-8d6f-58c9bec7f386-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "026872c1-0aeb-498d-8d6f-58c9bec7f386" (UID: "026872c1-0aeb-498d-8d6f-58c9bec7f386"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 19 17:20:02.855309 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:02.855287 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/026872c1-0aeb-498d-8d6f-58c9bec7f386-istio-token" (OuterVolumeSpecName: "istio-token") pod "026872c1-0aeb-498d-8d6f-58c9bec7f386" (UID: "026872c1-0aeb-498d-8d6f-58c9bec7f386"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 19 17:20:02.855398 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:02.855317 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/026872c1-0aeb-498d-8d6f-58c9bec7f386-kube-api-access-r2fw6" (OuterVolumeSpecName: "kube-api-access-r2fw6") pod "026872c1-0aeb-498d-8d6f-58c9bec7f386" (UID: "026872c1-0aeb-498d-8d6f-58c9bec7f386"). InnerVolumeSpecName "kube-api-access-r2fw6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 19 17:20:02.954315 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:02.954275 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/026872c1-0aeb-498d-8d6f-58c9bec7f386-workload-certs\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:20:02.954315 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:02.954308 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r2fw6\" (UniqueName: \"kubernetes.io/projected/026872c1-0aeb-498d-8d6f-58c9bec7f386-kube-api-access-r2fw6\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:20:02.954315 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:02.954322 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/026872c1-0aeb-498d-8d6f-58c9bec7f386-istiod-ca-cert\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:20:02.954559 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:02.954333 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/026872c1-0aeb-498d-8d6f-58c9bec7f386-istio-envoy\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:20:02.954559 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:02.954345 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/026872c1-0aeb-498d-8d6f-58c9bec7f386-istio-podinfo\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:20:02.954559 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:02.954355 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/026872c1-0aeb-498d-8d6f-58c9bec7f386-istio-data\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:20:02.954559 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:02.954365 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/026872c1-0aeb-498d-8d6f-58c9bec7f386-istio-token\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:20:03.679788 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:03.679758 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj" Jul 19 17:20:03.680234 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:03.679744 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj" event={"ID":"026872c1-0aeb-498d-8d6f-58c9bec7f386","Type":"ContainerDied","Data":"e2bc00fb2a4d7d7cf0db9d88972e3dacea1a80715e21d3ca1462903853d5ba38"} Jul 19 17:20:03.680234 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:03.679892 2583 scope.go:117] "RemoveContainer" containerID="fdba85bc0476ba86b3f4cb31c0db3355dd77d0915a09ea4fe93e425524e6828b" Jul 19 17:20:03.698460 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:03.698431 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj"] Jul 19 17:20:03.703587 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:03.703559 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-7dceef-openshift-default-6986d4bb5-ft7dj"] Jul 19 17:20:05.603542 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:05.603508 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="026872c1-0aeb-498d-8d6f-58c9bec7f386" path="/var/lib/kubelet/pods/026872c1-0aeb-498d-8d6f-58c9bec7f386/volumes" Jul 19 17:20:19.197549 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:19.197463 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr"] Jul 19 17:20:19.198121 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:19.197729 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr" podUID="1179f6e6-4830-4c84-81a4-c64376a0a267" containerName="istio-proxy" containerID="cri-o://1a3d099b9baa3321666f1bb7ca74489e95ee213b0810e276c9912cf41ec562ce" gracePeriod=30 Jul 19 17:20:20.063364 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:20.063333 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq"] Jul 19 17:20:20.063721 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:20.063708 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="026872c1-0aeb-498d-8d6f-58c9bec7f386" containerName="istio-proxy" Jul 19 17:20:20.063801 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:20.063723 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="026872c1-0aeb-498d-8d6f-58c9bec7f386" containerName="istio-proxy" Jul 19 17:20:20.063801 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:20.063744 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="026872c1-0aeb-498d-8d6f-58c9bec7f386" containerName="istio-proxy" Jul 19 17:20:20.063801 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:20.063750 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="026872c1-0aeb-498d-8d6f-58c9bec7f386" containerName="istio-proxy" Jul 19 17:20:20.063958 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:20.063818 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="026872c1-0aeb-498d-8d6f-58c9bec7f386" containerName="istio-proxy" Jul 19 17:20:20.068570 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:20.068548 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq" Jul 19 17:20:20.070654 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:20.070632 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-5c6a59-openshift-default-dockercfg-s4r87\"" Jul 19 17:20:20.078276 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:20.078254 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq"] Jul 19 17:20:20.221511 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:20.221474 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2ff6b2c-753c-4b40-bcef-cc2c6161a56e-workload-certs\") pod \"e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq\" (UID: \"a2ff6b2c-753c-4b40-bcef-cc2c6161a56e\") " pod="openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq" Jul 19 17:20:20.221885 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:20.221514 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2ff6b2c-753c-4b40-bcef-cc2c6161a56e-credential-socket\") pod \"e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq\" (UID: \"a2ff6b2c-753c-4b40-bcef-cc2c6161a56e\") " pod="openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq" Jul 19 17:20:20.221885 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:20.221556 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2ff6b2c-753c-4b40-bcef-cc2c6161a56e-istiod-ca-cert\") pod \"e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq\" (UID: \"a2ff6b2c-753c-4b40-bcef-cc2c6161a56e\") " pod="openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq" Jul 19 17:20:20.221885 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:20.221589 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h97ch\" (UniqueName: \"kubernetes.io/projected/a2ff6b2c-753c-4b40-bcef-cc2c6161a56e-kube-api-access-h97ch\") pod \"e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq\" (UID: \"a2ff6b2c-753c-4b40-bcef-cc2c6161a56e\") " pod="openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq" Jul 19 17:20:20.221885 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:20.221626 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2ff6b2c-753c-4b40-bcef-cc2c6161a56e-istio-podinfo\") pod \"e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq\" (UID: \"a2ff6b2c-753c-4b40-bcef-cc2c6161a56e\") " pod="openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq" Jul 19 17:20:20.221885 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:20.221700 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2ff6b2c-753c-4b40-bcef-cc2c6161a56e-workload-socket\") pod \"e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq\" (UID: \"a2ff6b2c-753c-4b40-bcef-cc2c6161a56e\") " pod="openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq" Jul 19 17:20:20.221885 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:20.221752 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2ff6b2c-753c-4b40-bcef-cc2c6161a56e-istio-envoy\") pod \"e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq\" (UID: \"a2ff6b2c-753c-4b40-bcef-cc2c6161a56e\") " pod="openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq" Jul 19 17:20:20.221885 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:20.221769 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2ff6b2c-753c-4b40-bcef-cc2c6161a56e-istio-data\") pod \"e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq\" (UID: \"a2ff6b2c-753c-4b40-bcef-cc2c6161a56e\") " pod="openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq" Jul 19 17:20:20.221885 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:20.221787 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2ff6b2c-753c-4b40-bcef-cc2c6161a56e-istio-token\") pod \"e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq\" (UID: \"a2ff6b2c-753c-4b40-bcef-cc2c6161a56e\") " pod="openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq" Jul 19 17:20:20.323112 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:20.323002 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2ff6b2c-753c-4b40-bcef-cc2c6161a56e-istio-envoy\") pod \"e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq\" (UID: \"a2ff6b2c-753c-4b40-bcef-cc2c6161a56e\") " pod="openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq" Jul 19 17:20:20.323112 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:20.323038 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2ff6b2c-753c-4b40-bcef-cc2c6161a56e-istio-data\") pod \"e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq\" (UID: \"a2ff6b2c-753c-4b40-bcef-cc2c6161a56e\") " pod="openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq" Jul 19 17:20:20.323112 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:20.323079 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2ff6b2c-753c-4b40-bcef-cc2c6161a56e-istio-token\") pod \"e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq\" (UID: \"a2ff6b2c-753c-4b40-bcef-cc2c6161a56e\") " pod="openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq" Jul 19 17:20:20.323405 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:20.323127 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2ff6b2c-753c-4b40-bcef-cc2c6161a56e-workload-certs\") pod \"e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq\" (UID: \"a2ff6b2c-753c-4b40-bcef-cc2c6161a56e\") " pod="openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq" Jul 19 17:20:20.323405 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:20.323158 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2ff6b2c-753c-4b40-bcef-cc2c6161a56e-credential-socket\") pod \"e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq\" (UID: \"a2ff6b2c-753c-4b40-bcef-cc2c6161a56e\") " pod="openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq" Jul 19 17:20:20.323405 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:20.323193 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2ff6b2c-753c-4b40-bcef-cc2c6161a56e-istiod-ca-cert\") pod \"e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq\" (UID: \"a2ff6b2c-753c-4b40-bcef-cc2c6161a56e\") " pod="openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq" Jul 19 17:20:20.323405 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:20.323241 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h97ch\" (UniqueName: \"kubernetes.io/projected/a2ff6b2c-753c-4b40-bcef-cc2c6161a56e-kube-api-access-h97ch\") pod \"e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq\" (UID: \"a2ff6b2c-753c-4b40-bcef-cc2c6161a56e\") " pod="openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq" Jul 19 17:20:20.323405 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:20.323293 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2ff6b2c-753c-4b40-bcef-cc2c6161a56e-istio-podinfo\") pod \"e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq\" (UID: \"a2ff6b2c-753c-4b40-bcef-cc2c6161a56e\") " pod="openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq" Jul 19 17:20:20.323405 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:20.323363 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2ff6b2c-753c-4b40-bcef-cc2c6161a56e-workload-socket\") pod \"e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq\" (UID: \"a2ff6b2c-753c-4b40-bcef-cc2c6161a56e\") " pod="openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq" Jul 19 17:20:20.323715 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:20.323525 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2ff6b2c-753c-4b40-bcef-cc2c6161a56e-istio-data\") pod \"e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq\" (UID: \"a2ff6b2c-753c-4b40-bcef-cc2c6161a56e\") " pod="openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq" Jul 19 17:20:20.323715 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:20.323677 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2ff6b2c-753c-4b40-bcef-cc2c6161a56e-credential-socket\") pod \"e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq\" (UID: \"a2ff6b2c-753c-4b40-bcef-cc2c6161a56e\") " pod="openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq" Jul 19 17:20:20.323715 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:20.323685 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2ff6b2c-753c-4b40-bcef-cc2c6161a56e-workload-socket\") pod \"e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq\" (UID: \"a2ff6b2c-753c-4b40-bcef-cc2c6161a56e\") " pod="openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq" Jul 19 17:20:20.323873 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:20.323733 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2ff6b2c-753c-4b40-bcef-cc2c6161a56e-workload-certs\") pod \"e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq\" (UID: \"a2ff6b2c-753c-4b40-bcef-cc2c6161a56e\") " pod="openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq" Jul 19 17:20:20.324129 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:20.324109 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2ff6b2c-753c-4b40-bcef-cc2c6161a56e-istiod-ca-cert\") pod \"e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq\" (UID: \"a2ff6b2c-753c-4b40-bcef-cc2c6161a56e\") " pod="openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq" Jul 19 17:20:20.325262 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:20.325246 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2ff6b2c-753c-4b40-bcef-cc2c6161a56e-istio-envoy\") pod \"e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq\" (UID: \"a2ff6b2c-753c-4b40-bcef-cc2c6161a56e\") " pod="openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq" Jul 19 17:20:20.325980 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:20.325959 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2ff6b2c-753c-4b40-bcef-cc2c6161a56e-istio-podinfo\") pod \"e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq\" (UID: \"a2ff6b2c-753c-4b40-bcef-cc2c6161a56e\") " pod="openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq" Jul 19 17:20:20.330398 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:20.330379 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2ff6b2c-753c-4b40-bcef-cc2c6161a56e-istio-token\") pod \"e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq\" (UID: \"a2ff6b2c-753c-4b40-bcef-cc2c6161a56e\") " pod="openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq" Jul 19 17:20:20.330838 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:20.330814 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h97ch\" (UniqueName: \"kubernetes.io/projected/a2ff6b2c-753c-4b40-bcef-cc2c6161a56e-kube-api-access-h97ch\") pod \"e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq\" (UID: \"a2ff6b2c-753c-4b40-bcef-cc2c6161a56e\") " pod="openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq" Jul 19 17:20:20.380450 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:20.380401 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq" Jul 19 17:20:20.518321 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:20.518292 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq"] Jul 19 17:20:20.520219 ip-10-0-134-203 kubenswrapper[2583]: W0719 17:20:20.520192 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2ff6b2c_753c_4b40_bcef_cc2c6161a56e.slice/crio-5d67e9e6fdfa0fe1e9495838601209716e372e4db3ff6cbbe321b2be0701b56d WatchSource:0}: Error finding container 5d67e9e6fdfa0fe1e9495838601209716e372e4db3ff6cbbe321b2be0701b56d: Status 404 returned error can't find the container with id 5d67e9e6fdfa0fe1e9495838601209716e372e4db3ff6cbbe321b2be0701b56d Jul 19 17:20:20.522690 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:20.522658 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:20:20.522758 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:20.522726 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:20:20.522804 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:20.522755 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:20:20.761776 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:20.761732 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq" event={"ID":"a2ff6b2c-753c-4b40-bcef-cc2c6161a56e","Type":"ContainerStarted","Data":"219d7489b55634478067f0b7271248071ddd4b019d74f6bdfd1e4d9ed79b903d"} Jul 19 17:20:20.761972 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:20.761804 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq" event={"ID":"a2ff6b2c-753c-4b40-bcef-cc2c6161a56e","Type":"ContainerStarted","Data":"5d67e9e6fdfa0fe1e9495838601209716e372e4db3ff6cbbe321b2be0701b56d"} Jul 19 17:20:20.779723 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:20.779671 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq" podStartSLOduration=0.779651419 podStartE2EDuration="779.651419ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 17:20:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 17:20:20.77885415 +0000 UTC m=+2515.765429639" watchObservedRunningTime="2026-07-19 17:20:20.779651419 +0000 UTC m=+2515.766226902" Jul 19 17:20:21.380885 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:21.380845 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq" Jul 19 17:20:21.385958 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:21.385926 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq" Jul 19 17:20:21.766625 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:21.766538 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq" Jul 19 17:20:21.767407 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:21.767382 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq" Jul 19 17:20:24.456405 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:24.456378 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr" Jul 19 17:20:24.569017 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:24.568937 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1179f6e6-4830-4c84-81a4-c64376a0a267-istio-data\") pod \"1179f6e6-4830-4c84-81a4-c64376a0a267\" (UID: \"1179f6e6-4830-4c84-81a4-c64376a0a267\") " Jul 19 17:20:24.569017 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:24.569011 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1179f6e6-4830-4c84-81a4-c64376a0a267-credential-socket\") pod \"1179f6e6-4830-4c84-81a4-c64376a0a267\" (UID: \"1179f6e6-4830-4c84-81a4-c64376a0a267\") " Jul 19 17:20:24.569258 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:24.569053 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1179f6e6-4830-4c84-81a4-c64376a0a267-istio-envoy\") pod \"1179f6e6-4830-4c84-81a4-c64376a0a267\" (UID: \"1179f6e6-4830-4c84-81a4-c64376a0a267\") " Jul 19 17:20:24.569258 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:24.569095 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1179f6e6-4830-4c84-81a4-c64376a0a267-workload-socket\") pod \"1179f6e6-4830-4c84-81a4-c64376a0a267\" (UID: \"1179f6e6-4830-4c84-81a4-c64376a0a267\") " Jul 19 17:20:24.569258 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:24.569149 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1179f6e6-4830-4c84-81a4-c64376a0a267-istio-token\") pod \"1179f6e6-4830-4c84-81a4-c64376a0a267\" (UID: \"1179f6e6-4830-4c84-81a4-c64376a0a267\") " Jul 19 17:20:24.569258 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:24.569179 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1179f6e6-4830-4c84-81a4-c64376a0a267-istio-podinfo\") pod \"1179f6e6-4830-4c84-81a4-c64376a0a267\" (UID: \"1179f6e6-4830-4c84-81a4-c64376a0a267\") " Jul 19 17:20:24.569258 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:24.569223 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1179f6e6-4830-4c84-81a4-c64376a0a267-workload-certs\") pod \"1179f6e6-4830-4c84-81a4-c64376a0a267\" (UID: \"1179f6e6-4830-4c84-81a4-c64376a0a267\") " Jul 19 17:20:24.569532 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:24.569269 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1179f6e6-4830-4c84-81a4-c64376a0a267-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1179f6e6-4830-4c84-81a4-c64376a0a267" (UID: "1179f6e6-4830-4c84-81a4-c64376a0a267"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:20:24.569532 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:24.569378 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1179f6e6-4830-4c84-81a4-c64376a0a267-istiod-ca-cert\") pod \"1179f6e6-4830-4c84-81a4-c64376a0a267\" (UID: \"1179f6e6-4830-4c84-81a4-c64376a0a267\") " Jul 19 17:20:24.569532 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:24.569421 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8p2c4\" (UniqueName: \"kubernetes.io/projected/1179f6e6-4830-4c84-81a4-c64376a0a267-kube-api-access-8p2c4\") pod \"1179f6e6-4830-4c84-81a4-c64376a0a267\" (UID: \"1179f6e6-4830-4c84-81a4-c64376a0a267\") " Jul 19 17:20:24.569532 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:24.569439 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1179f6e6-4830-4c84-81a4-c64376a0a267-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1179f6e6-4830-4c84-81a4-c64376a0a267" (UID: "1179f6e6-4830-4c84-81a4-c64376a0a267"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:20:24.569532 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:24.569492 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1179f6e6-4830-4c84-81a4-c64376a0a267-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1179f6e6-4830-4c84-81a4-c64376a0a267" (UID: "1179f6e6-4830-4c84-81a4-c64376a0a267"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:20:24.569807 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:24.569764 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1179f6e6-4830-4c84-81a4-c64376a0a267-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1179f6e6-4830-4c84-81a4-c64376a0a267" (UID: "1179f6e6-4830-4c84-81a4-c64376a0a267"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 19 17:20:24.569807 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:24.569787 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1179f6e6-4830-4c84-81a4-c64376a0a267-credential-socket\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:20:24.569927 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:24.569793 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1179f6e6-4830-4c84-81a4-c64376a0a267-istio-data" (OuterVolumeSpecName: "istio-data") pod "1179f6e6-4830-4c84-81a4-c64376a0a267" (UID: "1179f6e6-4830-4c84-81a4-c64376a0a267"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:20:24.569927 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:24.569829 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1179f6e6-4830-4c84-81a4-c64376a0a267-workload-socket\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:20:24.569927 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:24.569852 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1179f6e6-4830-4c84-81a4-c64376a0a267-workload-certs\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:20:24.571447 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:24.571416 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1179f6e6-4830-4c84-81a4-c64376a0a267-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1179f6e6-4830-4c84-81a4-c64376a0a267" (UID: "1179f6e6-4830-4c84-81a4-c64376a0a267"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:20:24.571447 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:24.571422 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1179f6e6-4830-4c84-81a4-c64376a0a267-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1179f6e6-4830-4c84-81a4-c64376a0a267" (UID: "1179f6e6-4830-4c84-81a4-c64376a0a267"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 19 17:20:24.571638 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:24.571478 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1179f6e6-4830-4c84-81a4-c64376a0a267-istio-token" (OuterVolumeSpecName: "istio-token") pod "1179f6e6-4830-4c84-81a4-c64376a0a267" (UID: "1179f6e6-4830-4c84-81a4-c64376a0a267"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 19 17:20:24.571638 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:24.571586 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1179f6e6-4830-4c84-81a4-c64376a0a267-kube-api-access-8p2c4" (OuterVolumeSpecName: "kube-api-access-8p2c4") pod "1179f6e6-4830-4c84-81a4-c64376a0a267" (UID: "1179f6e6-4830-4c84-81a4-c64376a0a267"). InnerVolumeSpecName "kube-api-access-8p2c4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 19 17:20:24.670932 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:24.670895 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1179f6e6-4830-4c84-81a4-c64376a0a267-istio-envoy\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:20:24.670932 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:24.670925 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1179f6e6-4830-4c84-81a4-c64376a0a267-istio-token\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:20:24.670932 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:24.670936 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1179f6e6-4830-4c84-81a4-c64376a0a267-istio-podinfo\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:20:24.671207 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:24.670945 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1179f6e6-4830-4c84-81a4-c64376a0a267-istiod-ca-cert\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:20:24.671207 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:24.670954 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8p2c4\" (UniqueName: \"kubernetes.io/projected/1179f6e6-4830-4c84-81a4-c64376a0a267-kube-api-access-8p2c4\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:20:24.671207 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:24.670964 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1179f6e6-4830-4c84-81a4-c64376a0a267-istio-data\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:20:24.783275 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:24.783245 2583 generic.go:356] "Generic (PLEG): container finished" podID="1179f6e6-4830-4c84-81a4-c64376a0a267" containerID="1a3d099b9baa3321666f1bb7ca74489e95ee213b0810e276c9912cf41ec562ce" exitCode=0 Jul 19 17:20:24.783446 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:24.783294 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr" Jul 19 17:20:24.783446 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:24.783295 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr" event={"ID":"1179f6e6-4830-4c84-81a4-c64376a0a267","Type":"ContainerDied","Data":"1a3d099b9baa3321666f1bb7ca74489e95ee213b0810e276c9912cf41ec562ce"} Jul 19 17:20:24.783446 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:24.783422 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr" event={"ID":"1179f6e6-4830-4c84-81a4-c64376a0a267","Type":"ContainerDied","Data":"a141bcb9f561f5ad0c9f40f51101698416cbc6712190a5c6125e7660cc1308a2"} Jul 19 17:20:24.783563 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:24.783455 2583 scope.go:117] "RemoveContainer" containerID="1a3d099b9baa3321666f1bb7ca74489e95ee213b0810e276c9912cf41ec562ce" Jul 19 17:20:24.794198 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:24.794175 2583 scope.go:117] "RemoveContainer" containerID="89f333abd5f9a8c0c1272ec0f4e41ff8e3d6beb46cdadd710b16cdf628a137be" Jul 19 17:20:24.802135 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:24.802112 2583 scope.go:117] "RemoveContainer" containerID="1a3d099b9baa3321666f1bb7ca74489e95ee213b0810e276c9912cf41ec562ce" Jul 19 17:20:24.802419 ip-10-0-134-203 kubenswrapper[2583]: E0719 17:20:24.802401 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a3d099b9baa3321666f1bb7ca74489e95ee213b0810e276c9912cf41ec562ce\": container with ID starting with 1a3d099b9baa3321666f1bb7ca74489e95ee213b0810e276c9912cf41ec562ce not found: ID does not exist" containerID="1a3d099b9baa3321666f1bb7ca74489e95ee213b0810e276c9912cf41ec562ce" Jul 19 17:20:24.802488 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:24.802428 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a3d099b9baa3321666f1bb7ca74489e95ee213b0810e276c9912cf41ec562ce"} err="failed to get container status \"1a3d099b9baa3321666f1bb7ca74489e95ee213b0810e276c9912cf41ec562ce\": rpc error: code = NotFound desc = could not find container \"1a3d099b9baa3321666f1bb7ca74489e95ee213b0810e276c9912cf41ec562ce\": container with ID starting with 1a3d099b9baa3321666f1bb7ca74489e95ee213b0810e276c9912cf41ec562ce not found: ID does not exist" Jul 19 17:20:24.802488 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:24.802445 2583 scope.go:117] "RemoveContainer" containerID="89f333abd5f9a8c0c1272ec0f4e41ff8e3d6beb46cdadd710b16cdf628a137be" Jul 19 17:20:24.802701 ip-10-0-134-203 kubenswrapper[2583]: E0719 17:20:24.802684 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89f333abd5f9a8c0c1272ec0f4e41ff8e3d6beb46cdadd710b16cdf628a137be\": container with ID starting with 89f333abd5f9a8c0c1272ec0f4e41ff8e3d6beb46cdadd710b16cdf628a137be not found: ID does not exist" containerID="89f333abd5f9a8c0c1272ec0f4e41ff8e3d6beb46cdadd710b16cdf628a137be" Jul 19 17:20:24.802749 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:24.802707 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89f333abd5f9a8c0c1272ec0f4e41ff8e3d6beb46cdadd710b16cdf628a137be"} err="failed to get container status \"89f333abd5f9a8c0c1272ec0f4e41ff8e3d6beb46cdadd710b16cdf628a137be\": rpc error: code = NotFound desc = could not find container \"89f333abd5f9a8c0c1272ec0f4e41ff8e3d6beb46cdadd710b16cdf628a137be\": container with ID starting with 89f333abd5f9a8c0c1272ec0f4e41ff8e3d6beb46cdadd710b16cdf628a137be not found: ID does not exist" Jul 19 17:20:24.806761 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:24.806733 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr"] Jul 19 17:20:24.812349 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:24.812322 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-ec4179-openshift-default-75969998b5-gfpmr"] Jul 19 17:20:25.602359 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:25.602317 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1179f6e6-4830-4c84-81a4-c64376a0a267" path="/var/lib/kubelet/pods/1179f6e6-4830-4c84-81a4-c64376a0a267/volumes" Jul 19 17:20:27.971712 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:27.971680 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv"] Jul 19 17:20:27.972116 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:27.972031 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1179f6e6-4830-4c84-81a4-c64376a0a267" containerName="istio-proxy" Jul 19 17:20:27.972116 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:27.972042 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="1179f6e6-4830-4c84-81a4-c64376a0a267" containerName="istio-proxy" Jul 19 17:20:27.972198 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:27.972144 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="026872c1-0aeb-498d-8d6f-58c9bec7f386" containerName="istio-proxy" Jul 19 17:20:27.972198 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:27.972152 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="1179f6e6-4830-4c84-81a4-c64376a0a267" containerName="istio-proxy" Jul 19 17:20:27.977213 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:27.977190 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv" Jul 19 17:20:27.979423 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:27.979395 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-35dde0-openshift-default-dockercfg-lw4fk\"" Jul 19 17:20:27.986427 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:27.986398 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv"] Jul 19 17:20:27.998386 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:27.998361 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b27ad183-4912-4501-b9a5-30b9fa9bd271-istio-envoy\") pod \"e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv\" (UID: \"b27ad183-4912-4501-b9a5-30b9fa9bd271\") " pod="openshift-ingress/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv" Jul 19 17:20:27.998386 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:27.998392 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tllgq\" (UniqueName: \"kubernetes.io/projected/b27ad183-4912-4501-b9a5-30b9fa9bd271-kube-api-access-tllgq\") pod \"e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv\" (UID: \"b27ad183-4912-4501-b9a5-30b9fa9bd271\") " pod="openshift-ingress/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv" Jul 19 17:20:27.998572 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:27.998426 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b27ad183-4912-4501-b9a5-30b9fa9bd271-istio-data\") pod \"e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv\" (UID: \"b27ad183-4912-4501-b9a5-30b9fa9bd271\") " pod="openshift-ingress/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv" Jul 19 17:20:27.998572 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:27.998502 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b27ad183-4912-4501-b9a5-30b9fa9bd271-istio-podinfo\") pod \"e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv\" (UID: \"b27ad183-4912-4501-b9a5-30b9fa9bd271\") " pod="openshift-ingress/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv" Jul 19 17:20:27.998572 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:27.998558 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b27ad183-4912-4501-b9a5-30b9fa9bd271-istio-token\") pod \"e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv\" (UID: \"b27ad183-4912-4501-b9a5-30b9fa9bd271\") " pod="openshift-ingress/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv" Jul 19 17:20:27.998748 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:27.998610 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b27ad183-4912-4501-b9a5-30b9fa9bd271-workload-socket\") pod \"e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv\" (UID: \"b27ad183-4912-4501-b9a5-30b9fa9bd271\") " pod="openshift-ingress/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv" Jul 19 17:20:27.998748 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:27.998686 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b27ad183-4912-4501-b9a5-30b9fa9bd271-istiod-ca-cert\") pod \"e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv\" (UID: \"b27ad183-4912-4501-b9a5-30b9fa9bd271\") " pod="openshift-ingress/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv" Jul 19 17:20:27.998748 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:27.998702 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b27ad183-4912-4501-b9a5-30b9fa9bd271-workload-certs\") pod \"e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv\" (UID: \"b27ad183-4912-4501-b9a5-30b9fa9bd271\") " pod="openshift-ingress/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv" Jul 19 17:20:27.998748 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:27.998716 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b27ad183-4912-4501-b9a5-30b9fa9bd271-credential-socket\") pod \"e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv\" (UID: \"b27ad183-4912-4501-b9a5-30b9fa9bd271\") " pod="openshift-ingress/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv" Jul 19 17:20:28.099374 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:28.099336 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b27ad183-4912-4501-b9a5-30b9fa9bd271-workload-socket\") pod \"e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv\" (UID: \"b27ad183-4912-4501-b9a5-30b9fa9bd271\") " pod="openshift-ingress/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv" Jul 19 17:20:28.099556 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:28.099411 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b27ad183-4912-4501-b9a5-30b9fa9bd271-istiod-ca-cert\") pod \"e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv\" (UID: \"b27ad183-4912-4501-b9a5-30b9fa9bd271\") " pod="openshift-ingress/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv" Jul 19 17:20:28.099556 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:28.099435 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b27ad183-4912-4501-b9a5-30b9fa9bd271-workload-certs\") pod \"e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv\" (UID: \"b27ad183-4912-4501-b9a5-30b9fa9bd271\") " pod="openshift-ingress/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv" Jul 19 17:20:28.099556 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:28.099455 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b27ad183-4912-4501-b9a5-30b9fa9bd271-credential-socket\") pod \"e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv\" (UID: \"b27ad183-4912-4501-b9a5-30b9fa9bd271\") " pod="openshift-ingress/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv" Jul 19 17:20:28.099556 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:28.099479 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b27ad183-4912-4501-b9a5-30b9fa9bd271-istio-envoy\") pod \"e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv\" (UID: \"b27ad183-4912-4501-b9a5-30b9fa9bd271\") " pod="openshift-ingress/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv" Jul 19 17:20:28.099556 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:28.099499 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tllgq\" (UniqueName: \"kubernetes.io/projected/b27ad183-4912-4501-b9a5-30b9fa9bd271-kube-api-access-tllgq\") pod \"e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv\" (UID: \"b27ad183-4912-4501-b9a5-30b9fa9bd271\") " pod="openshift-ingress/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv" Jul 19 17:20:28.099556 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:28.099547 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b27ad183-4912-4501-b9a5-30b9fa9bd271-istio-data\") pod \"e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv\" (UID: \"b27ad183-4912-4501-b9a5-30b9fa9bd271\") " pod="openshift-ingress/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv" Jul 19 17:20:28.099888 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:28.099600 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b27ad183-4912-4501-b9a5-30b9fa9bd271-istio-podinfo\") pod \"e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv\" (UID: \"b27ad183-4912-4501-b9a5-30b9fa9bd271\") " pod="openshift-ingress/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv" Jul 19 17:20:28.099888 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:28.099648 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b27ad183-4912-4501-b9a5-30b9fa9bd271-istio-token\") pod \"e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv\" (UID: \"b27ad183-4912-4501-b9a5-30b9fa9bd271\") " pod="openshift-ingress/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv" Jul 19 17:20:28.099888 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:28.099805 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b27ad183-4912-4501-b9a5-30b9fa9bd271-workload-socket\") pod \"e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv\" (UID: \"b27ad183-4912-4501-b9a5-30b9fa9bd271\") " pod="openshift-ingress/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv" Jul 19 17:20:28.099888 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:28.099835 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b27ad183-4912-4501-b9a5-30b9fa9bd271-workload-certs\") pod \"e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv\" (UID: \"b27ad183-4912-4501-b9a5-30b9fa9bd271\") " pod="openshift-ingress/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv" Jul 19 17:20:28.100116 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:28.099908 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b27ad183-4912-4501-b9a5-30b9fa9bd271-credential-socket\") pod \"e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv\" (UID: \"b27ad183-4912-4501-b9a5-30b9fa9bd271\") " pod="openshift-ingress/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv" Jul 19 17:20:28.100171 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:28.100113 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b27ad183-4912-4501-b9a5-30b9fa9bd271-istio-data\") pod \"e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv\" (UID: \"b27ad183-4912-4501-b9a5-30b9fa9bd271\") " pod="openshift-ingress/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv" Jul 19 17:20:28.100232 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:28.100197 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b27ad183-4912-4501-b9a5-30b9fa9bd271-istiod-ca-cert\") pod \"e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv\" (UID: \"b27ad183-4912-4501-b9a5-30b9fa9bd271\") " pod="openshift-ingress/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv" Jul 19 17:20:28.101796 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:28.101766 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b27ad183-4912-4501-b9a5-30b9fa9bd271-istio-envoy\") pod \"e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv\" (UID: \"b27ad183-4912-4501-b9a5-30b9fa9bd271\") " pod="openshift-ingress/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv" Jul 19 17:20:28.102039 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:28.102022 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b27ad183-4912-4501-b9a5-30b9fa9bd271-istio-podinfo\") pod \"e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv\" (UID: \"b27ad183-4912-4501-b9a5-30b9fa9bd271\") " pod="openshift-ingress/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv" Jul 19 17:20:28.107049 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:28.107028 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b27ad183-4912-4501-b9a5-30b9fa9bd271-istio-token\") pod \"e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv\" (UID: \"b27ad183-4912-4501-b9a5-30b9fa9bd271\") " pod="openshift-ingress/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv" Jul 19 17:20:28.107393 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:28.107373 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tllgq\" (UniqueName: \"kubernetes.io/projected/b27ad183-4912-4501-b9a5-30b9fa9bd271-kube-api-access-tllgq\") pod \"e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv\" (UID: \"b27ad183-4912-4501-b9a5-30b9fa9bd271\") " pod="openshift-ingress/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv" Jul 19 17:20:28.288654 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:28.288547 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv" Jul 19 17:20:28.427136 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:28.427101 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv"] Jul 19 17:20:28.429025 ip-10-0-134-203 kubenswrapper[2583]: W0719 17:20:28.428999 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb27ad183_4912_4501_b9a5_30b9fa9bd271.slice/crio-c512011b0682f01cbf0dd0e8a829d046b7963c922b1422b3a4652f8cdc9008ae WatchSource:0}: Error finding container c512011b0682f01cbf0dd0e8a829d046b7963c922b1422b3a4652f8cdc9008ae: Status 404 returned error can't find the container with id c512011b0682f01cbf0dd0e8a829d046b7963c922b1422b3a4652f8cdc9008ae Jul 19 17:20:28.431381 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:28.431348 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:20:28.431456 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:28.431413 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:20:28.431456 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:28.431445 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:20:28.804338 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:28.804293 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv" event={"ID":"b27ad183-4912-4501-b9a5-30b9fa9bd271","Type":"ContainerStarted","Data":"4ca0f363a3f314928af63b2449f590c5feddfc24f977e745d0c053f64a20cf4a"} Jul 19 17:20:28.804502 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:28.804362 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv" event={"ID":"b27ad183-4912-4501-b9a5-30b9fa9bd271","Type":"ContainerStarted","Data":"c512011b0682f01cbf0dd0e8a829d046b7963c922b1422b3a4652f8cdc9008ae"} Jul 19 17:20:28.825583 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:28.825533 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv" podStartSLOduration=1.825515986 podStartE2EDuration="1.825515986s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 17:20:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 17:20:28.823277892 +0000 UTC m=+2523.809853401" watchObservedRunningTime="2026-07-19 17:20:28.825515986 +0000 UTC m=+2523.812091469" Jul 19 17:20:29.288767 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:29.288722 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv" Jul 19 17:20:29.293702 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:29.293676 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv" Jul 19 17:20:29.809152 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:29.809114 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv" Jul 19 17:20:29.810118 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:29.810097 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv" Jul 19 17:20:36.768228 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:36.768188 2583 patch_prober.go:28] interesting pod/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 19 17:20:36.768635 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:36.768246 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq" podUID="a2ff6b2c-753c-4b40-bcef-cc2c6161a56e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 19 17:20:36.848874 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:36.848828 2583 generic.go:356] "Generic (PLEG): container finished" podID="a2ff6b2c-753c-4b40-bcef-cc2c6161a56e" containerID="219d7489b55634478067f0b7271248071ddd4b019d74f6bdfd1e4d9ed79b903d" exitCode=0 Jul 19 17:20:36.849093 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:36.848979 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq" event={"ID":"a2ff6b2c-753c-4b40-bcef-cc2c6161a56e","Type":"ContainerDied","Data":"219d7489b55634478067f0b7271248071ddd4b019d74f6bdfd1e4d9ed79b903d"} Jul 19 17:20:36.849535 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:36.849507 2583 scope.go:117] "RemoveContainer" containerID="219d7489b55634478067f0b7271248071ddd4b019d74f6bdfd1e4d9ed79b903d" Jul 19 17:20:36.850881 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:36.850834 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:20:36.850972 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:36.850938 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:20:36.851040 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:36.850987 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:20:37.859558 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:37.859519 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq" event={"ID":"a2ff6b2c-753c-4b40-bcef-cc2c6161a56e","Type":"ContainerStarted","Data":"f3d1976c11cccc2dda34cfb107ac65965d2926ffda09d3e2ad66d83272a9b81f"} Jul 19 17:20:38.381522 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:38.381486 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq" Jul 19 17:20:38.381888 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:38.381859 2583 patch_prober.go:28] interesting pod/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 19 17:20:38.381973 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:38.381935 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq" podUID="a2ff6b2c-753c-4b40-bcef-cc2c6161a56e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 19 17:20:39.381753 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:39.381710 2583 patch_prober.go:28] interesting pod/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 19 17:20:39.382151 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:39.381774 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq" podUID="a2ff6b2c-753c-4b40-bcef-cc2c6161a56e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 19 17:20:40.381280 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:40.381238 2583 patch_prober.go:28] interesting pod/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 19 17:20:40.381466 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:40.381347 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq" podUID="a2ff6b2c-753c-4b40-bcef-cc2c6161a56e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 19 17:20:41.380910 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:41.380876 2583 patch_prober.go:28] interesting pod/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 19 17:20:41.381316 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:41.380934 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq" podUID="a2ff6b2c-753c-4b40-bcef-cc2c6161a56e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 19 17:20:42.384805 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:42.384772 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq" Jul 19 17:20:42.385270 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:42.385165 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq" Jul 19 17:20:42.385919 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:42.385897 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq" Jul 19 17:20:42.887450 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:42.887420 2583 generic.go:356] "Generic (PLEG): container finished" podID="b27ad183-4912-4501-b9a5-30b9fa9bd271" containerID="4ca0f363a3f314928af63b2449f590c5feddfc24f977e745d0c053f64a20cf4a" exitCode=0 Jul 19 17:20:42.887615 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:42.887493 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv" event={"ID":"b27ad183-4912-4501-b9a5-30b9fa9bd271","Type":"ContainerDied","Data":"4ca0f363a3f314928af63b2449f590c5feddfc24f977e745d0c053f64a20cf4a"} Jul 19 17:20:42.887982 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:42.887969 2583 scope.go:117] "RemoveContainer" containerID="4ca0f363a3f314928af63b2449f590c5feddfc24f977e745d0c053f64a20cf4a" Jul 19 17:20:42.888907 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:42.888875 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:20:42.889017 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:42.888948 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:20:42.889017 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:42.888980 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:20:43.894131 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:43.894090 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv" event={"ID":"b27ad183-4912-4501-b9a5-30b9fa9bd271","Type":"ContainerStarted","Data":"98ed6e23b1876123919050582750fd410bebfd7254766dacd18a0119af684854"} Jul 19 17:20:44.289105 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:44.288995 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv" Jul 19 17:20:44.290270 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:44.290236 2583 patch_prober.go:28] interesting pod/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 19 17:20:44.290410 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:44.290286 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv" podUID="b27ad183-4912-4501-b9a5-30b9fa9bd271" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 19 17:20:44.809744 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:44.809706 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv" Jul 19 17:20:45.289646 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:45.289609 2583 patch_prober.go:28] interesting pod/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 19 17:20:45.290035 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:45.289668 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv" podUID="b27ad183-4912-4501-b9a5-30b9fa9bd271" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 19 17:20:46.289266 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:46.289231 2583 patch_prober.go:28] interesting pod/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 19 17:20:46.289451 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:46.289285 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv" podUID="b27ad183-4912-4501-b9a5-30b9fa9bd271" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 19 17:20:47.288917 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:47.288883 2583 patch_prober.go:28] interesting pod/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 19 17:20:47.289318 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:47.288936 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv" podUID="b27ad183-4912-4501-b9a5-30b9fa9bd271" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 19 17:20:48.293393 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:48.293360 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv" Jul 19 17:20:48.294283 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:20:48.294260 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv" Jul 19 17:21:09.876513 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:09.876477 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq"] Jul 19 17:21:09.877101 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:09.876707 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq" podUID="a2ff6b2c-753c-4b40-bcef-cc2c6161a56e" containerName="istio-proxy" containerID="cri-o://f3d1976c11cccc2dda34cfb107ac65965d2926ffda09d3e2ad66d83272a9b81f" gracePeriod=30 Jul 19 17:21:12.387758 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:12.387722 2583 patch_prober.go:28] interesting pod/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 19 17:21:12.388239 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:12.387797 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq" podUID="a2ff6b2c-753c-4b40-bcef-cc2c6161a56e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 19 17:21:15.041113 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:15.041050 2583 generic.go:356] "Generic (PLEG): container finished" podID="a2ff6b2c-753c-4b40-bcef-cc2c6161a56e" containerID="f3d1976c11cccc2dda34cfb107ac65965d2926ffda09d3e2ad66d83272a9b81f" exitCode=0 Jul 19 17:21:15.041593 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:15.041085 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq" event={"ID":"a2ff6b2c-753c-4b40-bcef-cc2c6161a56e","Type":"ContainerDied","Data":"f3d1976c11cccc2dda34cfb107ac65965d2926ffda09d3e2ad66d83272a9b81f"} Jul 19 17:21:15.041593 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:15.041178 2583 scope.go:117] "RemoveContainer" containerID="219d7489b55634478067f0b7271248071ddd4b019d74f6bdfd1e4d9ed79b903d" Jul 19 17:21:15.130065 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:15.130040 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq" Jul 19 17:21:15.171553 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:15.171475 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2ff6b2c-753c-4b40-bcef-cc2c6161a56e-workload-socket\") pod \"a2ff6b2c-753c-4b40-bcef-cc2c6161a56e\" (UID: \"a2ff6b2c-753c-4b40-bcef-cc2c6161a56e\") " Jul 19 17:21:15.171687 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:15.171576 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2ff6b2c-753c-4b40-bcef-cc2c6161a56e-workload-certs\") pod \"a2ff6b2c-753c-4b40-bcef-cc2c6161a56e\" (UID: \"a2ff6b2c-753c-4b40-bcef-cc2c6161a56e\") " Jul 19 17:21:15.171687 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:15.171601 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h97ch\" (UniqueName: \"kubernetes.io/projected/a2ff6b2c-753c-4b40-bcef-cc2c6161a56e-kube-api-access-h97ch\") pod \"a2ff6b2c-753c-4b40-bcef-cc2c6161a56e\" (UID: \"a2ff6b2c-753c-4b40-bcef-cc2c6161a56e\") " Jul 19 17:21:15.171687 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:15.171664 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2ff6b2c-753c-4b40-bcef-cc2c6161a56e-istio-podinfo\") pod \"a2ff6b2c-753c-4b40-bcef-cc2c6161a56e\" (UID: \"a2ff6b2c-753c-4b40-bcef-cc2c6161a56e\") " Jul 19 17:21:15.171839 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:15.171697 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2ff6b2c-753c-4b40-bcef-cc2c6161a56e-credential-socket\") pod \"a2ff6b2c-753c-4b40-bcef-cc2c6161a56e\" (UID: \"a2ff6b2c-753c-4b40-bcef-cc2c6161a56e\") " Jul 19 17:21:15.171839 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:15.171731 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2ff6b2c-753c-4b40-bcef-cc2c6161a56e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a2ff6b2c-753c-4b40-bcef-cc2c6161a56e" (UID: "a2ff6b2c-753c-4b40-bcef-cc2c6161a56e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:21:15.171839 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:15.171741 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2ff6b2c-753c-4b40-bcef-cc2c6161a56e-istio-envoy\") pod \"a2ff6b2c-753c-4b40-bcef-cc2c6161a56e\" (UID: \"a2ff6b2c-753c-4b40-bcef-cc2c6161a56e\") " Jul 19 17:21:15.171839 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:15.171793 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2ff6b2c-753c-4b40-bcef-cc2c6161a56e-istio-data\") pod \"a2ff6b2c-753c-4b40-bcef-cc2c6161a56e\" (UID: \"a2ff6b2c-753c-4b40-bcef-cc2c6161a56e\") " Jul 19 17:21:15.172047 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:15.171821 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2ff6b2c-753c-4b40-bcef-cc2c6161a56e-istio-token\") pod \"a2ff6b2c-753c-4b40-bcef-cc2c6161a56e\" (UID: \"a2ff6b2c-753c-4b40-bcef-cc2c6161a56e\") " Jul 19 17:21:15.172047 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:15.171875 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2ff6b2c-753c-4b40-bcef-cc2c6161a56e-istiod-ca-cert\") pod \"a2ff6b2c-753c-4b40-bcef-cc2c6161a56e\" (UID: \"a2ff6b2c-753c-4b40-bcef-cc2c6161a56e\") " Jul 19 17:21:15.172047 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:15.171896 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2ff6b2c-753c-4b40-bcef-cc2c6161a56e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a2ff6b2c-753c-4b40-bcef-cc2c6161a56e" (UID: "a2ff6b2c-753c-4b40-bcef-cc2c6161a56e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:21:15.172047 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:15.172026 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2ff6b2c-753c-4b40-bcef-cc2c6161a56e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a2ff6b2c-753c-4b40-bcef-cc2c6161a56e" (UID: "a2ff6b2c-753c-4b40-bcef-cc2c6161a56e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:21:15.172332 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:15.172309 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2ff6b2c-753c-4b40-bcef-cc2c6161a56e-workload-socket\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:21:15.172423 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:15.172338 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2ff6b2c-753c-4b40-bcef-cc2c6161a56e-workload-certs\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:21:15.172423 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:15.172352 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2ff6b2c-753c-4b40-bcef-cc2c6161a56e-credential-socket\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:21:15.172423 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:15.172350 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2ff6b2c-753c-4b40-bcef-cc2c6161a56e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a2ff6b2c-753c-4b40-bcef-cc2c6161a56e" (UID: "a2ff6b2c-753c-4b40-bcef-cc2c6161a56e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 19 17:21:15.172929 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:15.172901 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2ff6b2c-753c-4b40-bcef-cc2c6161a56e-istio-data" (OuterVolumeSpecName: "istio-data") pod "a2ff6b2c-753c-4b40-bcef-cc2c6161a56e" (UID: "a2ff6b2c-753c-4b40-bcef-cc2c6161a56e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:21:15.174023 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:15.173997 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2ff6b2c-753c-4b40-bcef-cc2c6161a56e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a2ff6b2c-753c-4b40-bcef-cc2c6161a56e" (UID: "a2ff6b2c-753c-4b40-bcef-cc2c6161a56e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:21:15.174195 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:15.174176 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2ff6b2c-753c-4b40-bcef-cc2c6161a56e-istio-token" (OuterVolumeSpecName: "istio-token") pod "a2ff6b2c-753c-4b40-bcef-cc2c6161a56e" (UID: "a2ff6b2c-753c-4b40-bcef-cc2c6161a56e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 19 17:21:15.174338 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:15.174321 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2ff6b2c-753c-4b40-bcef-cc2c6161a56e-kube-api-access-h97ch" (OuterVolumeSpecName: "kube-api-access-h97ch") pod "a2ff6b2c-753c-4b40-bcef-cc2c6161a56e" (UID: "a2ff6b2c-753c-4b40-bcef-cc2c6161a56e"). InnerVolumeSpecName "kube-api-access-h97ch". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 19 17:21:15.174475 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:15.174450 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a2ff6b2c-753c-4b40-bcef-cc2c6161a56e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a2ff6b2c-753c-4b40-bcef-cc2c6161a56e" (UID: "a2ff6b2c-753c-4b40-bcef-cc2c6161a56e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 19 17:21:15.273538 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:15.273504 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2ff6b2c-753c-4b40-bcef-cc2c6161a56e-istio-token\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:21:15.273538 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:15.273532 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2ff6b2c-753c-4b40-bcef-cc2c6161a56e-istiod-ca-cert\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:21:15.273538 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:15.273543 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h97ch\" (UniqueName: \"kubernetes.io/projected/a2ff6b2c-753c-4b40-bcef-cc2c6161a56e-kube-api-access-h97ch\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:21:15.273763 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:15.273552 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2ff6b2c-753c-4b40-bcef-cc2c6161a56e-istio-podinfo\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:21:15.273763 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:15.273560 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2ff6b2c-753c-4b40-bcef-cc2c6161a56e-istio-envoy\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:21:15.273763 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:15.273568 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2ff6b2c-753c-4b40-bcef-cc2c6161a56e-istio-data\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:21:16.047552 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:16.047463 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq" Jul 19 17:21:16.047552 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:16.047492 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq" event={"ID":"a2ff6b2c-753c-4b40-bcef-cc2c6161a56e","Type":"ContainerDied","Data":"5d67e9e6fdfa0fe1e9495838601209716e372e4db3ff6cbbe321b2be0701b56d"} Jul 19 17:21:16.047552 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:16.047544 2583 scope.go:117] "RemoveContainer" containerID="f3d1976c11cccc2dda34cfb107ac65965d2926ffda09d3e2ad66d83272a9b81f" Jul 19 17:21:16.065081 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:16.065029 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq"] Jul 19 17:21:16.070657 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:16.070630 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-5c6a59-openshift-default-7956899b65-jfpcq"] Jul 19 17:21:17.602423 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:17.602389 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2ff6b2c-753c-4b40-bcef-cc2c6161a56e" path="/var/lib/kubelet/pods/a2ff6b2c-753c-4b40-bcef-cc2c6161a56e/volumes" Jul 19 17:21:36.616015 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:36.615972 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv"] Jul 19 17:21:36.616482 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:36.616277 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv" podUID="b27ad183-4912-4501-b9a5-30b9fa9bd271" containerName="istio-proxy" containerID="cri-o://98ed6e23b1876123919050582750fd410bebfd7254766dacd18a0119af684854" gracePeriod=30 Jul 19 17:21:37.513464 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:37.513427 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw"] Jul 19 17:21:37.513876 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:37.513859 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2ff6b2c-753c-4b40-bcef-cc2c6161a56e" containerName="istio-proxy" Jul 19 17:21:37.513979 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:37.513880 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2ff6b2c-753c-4b40-bcef-cc2c6161a56e" containerName="istio-proxy" Jul 19 17:21:37.513979 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:37.513912 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1179f6e6-4830-4c84-81a4-c64376a0a267" containerName="istio-proxy" Jul 19 17:21:37.513979 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:37.513922 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="1179f6e6-4830-4c84-81a4-c64376a0a267" containerName="istio-proxy" Jul 19 17:21:37.514159 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:37.514007 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2ff6b2c-753c-4b40-bcef-cc2c6161a56e" containerName="istio-proxy" Jul 19 17:21:37.514159 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:37.514022 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="1179f6e6-4830-4c84-81a4-c64376a0a267" containerName="istio-proxy" Jul 19 17:21:37.514159 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:37.514033 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2ff6b2c-753c-4b40-bcef-cc2c6161a56e" containerName="istio-proxy" Jul 19 17:21:37.517152 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:37.517132 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw" Jul 19 17:21:37.519469 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:37.519448 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-8d2863-openshift-default-dockercfg-bq27d\"" Jul 19 17:21:37.528689 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:37.528669 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw"] Jul 19 17:21:37.692690 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:37.692652 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb-istiod-ca-cert\") pod \"e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw\" (UID: \"b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb\") " pod="openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw" Jul 19 17:21:37.692690 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:37.692689 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb-workload-certs\") pod \"e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw\" (UID: \"b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb\") " pod="openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw" Jul 19 17:21:37.693168 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:37.692710 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb-credential-socket\") pod \"e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw\" (UID: \"b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb\") " pod="openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw" Jul 19 17:21:37.693168 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:37.692765 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb-workload-socket\") pod \"e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw\" (UID: \"b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb\") " pod="openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw" Jul 19 17:21:37.693168 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:37.692815 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb-istio-podinfo\") pod \"e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw\" (UID: \"b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb\") " pod="openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw" Jul 19 17:21:37.693168 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:37.692849 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb-istio-token\") pod \"e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw\" (UID: \"b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb\") " pod="openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw" Jul 19 17:21:37.693168 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:37.692904 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb-istio-data\") pod \"e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw\" (UID: \"b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb\") " pod="openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw" Jul 19 17:21:37.693168 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:37.692949 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb-istio-envoy\") pod \"e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw\" (UID: \"b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb\") " pod="openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw" Jul 19 17:21:37.693168 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:37.692974 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fb779\" (UniqueName: \"kubernetes.io/projected/b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb-kube-api-access-fb779\") pod \"e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw\" (UID: \"b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb\") " pod="openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw" Jul 19 17:21:37.794354 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:37.794252 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb-istio-podinfo\") pod \"e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw\" (UID: \"b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb\") " pod="openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw" Jul 19 17:21:37.794538 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:37.794362 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb-istio-token\") pod \"e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw\" (UID: \"b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb\") " pod="openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw" Jul 19 17:21:37.794538 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:37.794406 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb-istio-data\") pod \"e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw\" (UID: \"b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb\") " pod="openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw" Jul 19 17:21:37.794538 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:37.794464 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb-istio-envoy\") pod \"e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw\" (UID: \"b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb\") " pod="openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw" Jul 19 17:21:37.794538 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:37.794489 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fb779\" (UniqueName: \"kubernetes.io/projected/b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb-kube-api-access-fb779\") pod \"e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw\" (UID: \"b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb\") " pod="openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw" Jul 19 17:21:37.794754 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:37.794555 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb-istiod-ca-cert\") pod \"e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw\" (UID: \"b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb\") " pod="openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw" Jul 19 17:21:37.794754 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:37.794582 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb-workload-certs\") pod \"e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw\" (UID: \"b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb\") " pod="openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw" Jul 19 17:21:37.794754 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:37.794613 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb-credential-socket\") pod \"e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw\" (UID: \"b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb\") " pod="openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw" Jul 19 17:21:37.794754 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:37.794651 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb-workload-socket\") pod \"e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw\" (UID: \"b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb\") " pod="openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw" Jul 19 17:21:37.795024 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:37.794873 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb-istio-data\") pod \"e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw\" (UID: \"b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb\") " pod="openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw" Jul 19 17:21:37.795024 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:37.794948 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb-workload-certs\") pod \"e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw\" (UID: \"b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb\") " pod="openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw" Jul 19 17:21:37.795179 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:37.795158 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb-workload-socket\") pod \"e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw\" (UID: \"b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb\") " pod="openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw" Jul 19 17:21:37.795451 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:37.795430 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb-credential-socket\") pod \"e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw\" (UID: \"b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb\") " pod="openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw" Jul 19 17:21:37.795526 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:37.795459 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb-istiod-ca-cert\") pod \"e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw\" (UID: \"b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb\") " pod="openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw" Jul 19 17:21:37.796827 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:37.796800 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb-istio-envoy\") pod \"e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw\" (UID: \"b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb\") " pod="openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw" Jul 19 17:21:37.796918 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:37.796881 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb-istio-podinfo\") pod \"e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw\" (UID: \"b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb\") " pod="openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw" Jul 19 17:21:37.802080 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:37.802031 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb-istio-token\") pod \"e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw\" (UID: \"b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb\") " pod="openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw" Jul 19 17:21:37.802179 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:37.802143 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fb779\" (UniqueName: \"kubernetes.io/projected/b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb-kube-api-access-fb779\") pod \"e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw\" (UID: \"b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb\") " pod="openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw" Jul 19 17:21:37.827925 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:37.827890 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw" Jul 19 17:21:37.966297 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:37.966269 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw"] Jul 19 17:21:37.968365 ip-10-0-134-203 kubenswrapper[2583]: W0719 17:21:37.968333 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb025ac83_86ce_4c4a_a70e_5f6f5eeed3bb.slice/crio-f70eed990461daf5a953b24c4c098cbfc7c4f7f4b4ee8b7b8b568748ba026041 WatchSource:0}: Error finding container f70eed990461daf5a953b24c4c098cbfc7c4f7f4b4ee8b7b8b568748ba026041: Status 404 returned error can't find the container with id f70eed990461daf5a953b24c4c098cbfc7c4f7f4b4ee8b7b8b568748ba026041 Jul 19 17:21:37.970526 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:37.970492 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:21:37.970621 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:37.970559 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:21:37.970621 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:37.970587 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:21:38.155419 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:38.155387 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw" event={"ID":"b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb","Type":"ContainerStarted","Data":"2490de64d8dff989b7fcdc795413fae8a6906d4df53c2b0577e836b21d83a856"} Jul 19 17:21:38.155603 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:38.155449 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw" event={"ID":"b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb","Type":"ContainerStarted","Data":"f70eed990461daf5a953b24c4c098cbfc7c4f7f4b4ee8b7b8b568748ba026041"} Jul 19 17:21:38.173459 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:38.173408 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw" podStartSLOduration=1.173392741 podStartE2EDuration="1.173392741s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 17:21:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 17:21:38.172461717 +0000 UTC m=+2593.159037203" watchObservedRunningTime="2026-07-19 17:21:38.173392741 +0000 UTC m=+2593.159968225" Jul 19 17:21:38.828352 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:38.828310 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw" Jul 19 17:21:39.833356 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:39.833324 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw" Jul 19 17:21:40.164341 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:40.164250 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw" Jul 19 17:21:40.165323 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:40.165304 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw" Jul 19 17:21:41.878915 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:41.878891 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv" Jul 19 17:21:41.932367 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:41.932286 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b27ad183-4912-4501-b9a5-30b9fa9bd271-credential-socket\") pod \"b27ad183-4912-4501-b9a5-30b9fa9bd271\" (UID: \"b27ad183-4912-4501-b9a5-30b9fa9bd271\") " Jul 19 17:21:41.932367 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:41.932329 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b27ad183-4912-4501-b9a5-30b9fa9bd271-istio-token\") pod \"b27ad183-4912-4501-b9a5-30b9fa9bd271\" (UID: \"b27ad183-4912-4501-b9a5-30b9fa9bd271\") " Jul 19 17:21:41.932367 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:41.932346 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b27ad183-4912-4501-b9a5-30b9fa9bd271-istiod-ca-cert\") pod \"b27ad183-4912-4501-b9a5-30b9fa9bd271\" (UID: \"b27ad183-4912-4501-b9a5-30b9fa9bd271\") " Jul 19 17:21:41.932620 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:41.932373 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b27ad183-4912-4501-b9a5-30b9fa9bd271-workload-socket\") pod \"b27ad183-4912-4501-b9a5-30b9fa9bd271\" (UID: \"b27ad183-4912-4501-b9a5-30b9fa9bd271\") " Jul 19 17:21:41.932620 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:41.932389 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b27ad183-4912-4501-b9a5-30b9fa9bd271-workload-certs\") pod \"b27ad183-4912-4501-b9a5-30b9fa9bd271\" (UID: \"b27ad183-4912-4501-b9a5-30b9fa9bd271\") " Jul 19 17:21:41.932620 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:41.932418 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b27ad183-4912-4501-b9a5-30b9fa9bd271-istio-data\") pod \"b27ad183-4912-4501-b9a5-30b9fa9bd271\" (UID: \"b27ad183-4912-4501-b9a5-30b9fa9bd271\") " Jul 19 17:21:41.932620 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:41.932473 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b27ad183-4912-4501-b9a5-30b9fa9bd271-istio-podinfo\") pod \"b27ad183-4912-4501-b9a5-30b9fa9bd271\" (UID: \"b27ad183-4912-4501-b9a5-30b9fa9bd271\") " Jul 19 17:21:41.932620 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:41.932549 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b27ad183-4912-4501-b9a5-30b9fa9bd271-istio-envoy\") pod \"b27ad183-4912-4501-b9a5-30b9fa9bd271\" (UID: \"b27ad183-4912-4501-b9a5-30b9fa9bd271\") " Jul 19 17:21:41.932887 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:41.932621 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b27ad183-4912-4501-b9a5-30b9fa9bd271-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b27ad183-4912-4501-b9a5-30b9fa9bd271" (UID: "b27ad183-4912-4501-b9a5-30b9fa9bd271"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:21:41.932887 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:41.932649 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tllgq\" (UniqueName: \"kubernetes.io/projected/b27ad183-4912-4501-b9a5-30b9fa9bd271-kube-api-access-tllgq\") pod \"b27ad183-4912-4501-b9a5-30b9fa9bd271\" (UID: \"b27ad183-4912-4501-b9a5-30b9fa9bd271\") " Jul 19 17:21:41.932887 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:41.932667 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b27ad183-4912-4501-b9a5-30b9fa9bd271-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b27ad183-4912-4501-b9a5-30b9fa9bd271" (UID: "b27ad183-4912-4501-b9a5-30b9fa9bd271"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:21:41.932887 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:41.932799 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b27ad183-4912-4501-b9a5-30b9fa9bd271-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b27ad183-4912-4501-b9a5-30b9fa9bd271" (UID: "b27ad183-4912-4501-b9a5-30b9fa9bd271"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 19 17:21:41.933117 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:41.933019 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b27ad183-4912-4501-b9a5-30b9fa9bd271-workload-socket\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:21:41.933117 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:41.933039 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b27ad183-4912-4501-b9a5-30b9fa9bd271-credential-socket\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:21:41.933117 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:41.933078 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b27ad183-4912-4501-b9a5-30b9fa9bd271-istiod-ca-cert\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:21:41.933117 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:41.933102 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b27ad183-4912-4501-b9a5-30b9fa9bd271-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b27ad183-4912-4501-b9a5-30b9fa9bd271" (UID: "b27ad183-4912-4501-b9a5-30b9fa9bd271"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:21:41.933348 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:41.933324 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b27ad183-4912-4501-b9a5-30b9fa9bd271-istio-data" (OuterVolumeSpecName: "istio-data") pod "b27ad183-4912-4501-b9a5-30b9fa9bd271" (UID: "b27ad183-4912-4501-b9a5-30b9fa9bd271"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:21:41.934718 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:41.934691 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b27ad183-4912-4501-b9a5-30b9fa9bd271-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b27ad183-4912-4501-b9a5-30b9fa9bd271" (UID: "b27ad183-4912-4501-b9a5-30b9fa9bd271"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:21:41.934718 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:41.934694 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b27ad183-4912-4501-b9a5-30b9fa9bd271-kube-api-access-tllgq" (OuterVolumeSpecName: "kube-api-access-tllgq") pod "b27ad183-4912-4501-b9a5-30b9fa9bd271" (UID: "b27ad183-4912-4501-b9a5-30b9fa9bd271"). InnerVolumeSpecName "kube-api-access-tllgq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 19 17:21:41.935092 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:41.935043 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b27ad183-4912-4501-b9a5-30b9fa9bd271-istio-token" (OuterVolumeSpecName: "istio-token") pod "b27ad183-4912-4501-b9a5-30b9fa9bd271" (UID: "b27ad183-4912-4501-b9a5-30b9fa9bd271"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 19 17:21:41.935203 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:41.935069 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b27ad183-4912-4501-b9a5-30b9fa9bd271-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b27ad183-4912-4501-b9a5-30b9fa9bd271" (UID: "b27ad183-4912-4501-b9a5-30b9fa9bd271"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 19 17:21:42.034050 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:42.034015 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b27ad183-4912-4501-b9a5-30b9fa9bd271-istio-token\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:21:42.034050 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:42.034046 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b27ad183-4912-4501-b9a5-30b9fa9bd271-workload-certs\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:21:42.034050 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:42.034077 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b27ad183-4912-4501-b9a5-30b9fa9bd271-istio-data\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:21:42.034312 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:42.034089 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b27ad183-4912-4501-b9a5-30b9fa9bd271-istio-podinfo\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:21:42.034312 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:42.034101 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b27ad183-4912-4501-b9a5-30b9fa9bd271-istio-envoy\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:21:42.034312 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:42.034114 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tllgq\" (UniqueName: \"kubernetes.io/projected/b27ad183-4912-4501-b9a5-30b9fa9bd271-kube-api-access-tllgq\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:21:42.174202 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:42.174168 2583 generic.go:356] "Generic (PLEG): container finished" podID="b27ad183-4912-4501-b9a5-30b9fa9bd271" containerID="98ed6e23b1876123919050582750fd410bebfd7254766dacd18a0119af684854" exitCode=0 Jul 19 17:21:42.174393 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:42.174215 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv" Jul 19 17:21:42.174393 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:42.174257 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv" event={"ID":"b27ad183-4912-4501-b9a5-30b9fa9bd271","Type":"ContainerDied","Data":"98ed6e23b1876123919050582750fd410bebfd7254766dacd18a0119af684854"} Jul 19 17:21:42.174393 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:42.174321 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv" event={"ID":"b27ad183-4912-4501-b9a5-30b9fa9bd271","Type":"ContainerDied","Data":"c512011b0682f01cbf0dd0e8a829d046b7963c922b1422b3a4652f8cdc9008ae"} Jul 19 17:21:42.174393 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:42.174354 2583 scope.go:117] "RemoveContainer" containerID="98ed6e23b1876123919050582750fd410bebfd7254766dacd18a0119af684854" Jul 19 17:21:42.183857 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:42.183840 2583 scope.go:117] "RemoveContainer" containerID="4ca0f363a3f314928af63b2449f590c5feddfc24f977e745d0c053f64a20cf4a" Jul 19 17:21:42.192159 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:42.192142 2583 scope.go:117] "RemoveContainer" containerID="98ed6e23b1876123919050582750fd410bebfd7254766dacd18a0119af684854" Jul 19 17:21:42.192433 ip-10-0-134-203 kubenswrapper[2583]: E0719 17:21:42.192412 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98ed6e23b1876123919050582750fd410bebfd7254766dacd18a0119af684854\": container with ID starting with 98ed6e23b1876123919050582750fd410bebfd7254766dacd18a0119af684854 not found: ID does not exist" containerID="98ed6e23b1876123919050582750fd410bebfd7254766dacd18a0119af684854" Jul 19 17:21:42.192483 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:42.192442 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98ed6e23b1876123919050582750fd410bebfd7254766dacd18a0119af684854"} err="failed to get container status \"98ed6e23b1876123919050582750fd410bebfd7254766dacd18a0119af684854\": rpc error: code = NotFound desc = could not find container \"98ed6e23b1876123919050582750fd410bebfd7254766dacd18a0119af684854\": container with ID starting with 98ed6e23b1876123919050582750fd410bebfd7254766dacd18a0119af684854 not found: ID does not exist" Jul 19 17:21:42.192483 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:42.192459 2583 scope.go:117] "RemoveContainer" containerID="4ca0f363a3f314928af63b2449f590c5feddfc24f977e745d0c053f64a20cf4a" Jul 19 17:21:42.192701 ip-10-0-134-203 kubenswrapper[2583]: E0719 17:21:42.192683 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ca0f363a3f314928af63b2449f590c5feddfc24f977e745d0c053f64a20cf4a\": container with ID starting with 4ca0f363a3f314928af63b2449f590c5feddfc24f977e745d0c053f64a20cf4a not found: ID does not exist" containerID="4ca0f363a3f314928af63b2449f590c5feddfc24f977e745d0c053f64a20cf4a" Jul 19 17:21:42.192751 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:42.192708 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ca0f363a3f314928af63b2449f590c5feddfc24f977e745d0c053f64a20cf4a"} err="failed to get container status \"4ca0f363a3f314928af63b2449f590c5feddfc24f977e745d0c053f64a20cf4a\": rpc error: code = NotFound desc = could not find container \"4ca0f363a3f314928af63b2449f590c5feddfc24f977e745d0c053f64a20cf4a\": container with ID starting with 4ca0f363a3f314928af63b2449f590c5feddfc24f977e745d0c053f64a20cf4a not found: ID does not exist" Jul 19 17:21:42.196408 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:42.196386 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv"] Jul 19 17:21:42.200162 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:42.200142 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-35dde0-openshift-default-bf6cd54f9-wsbmv"] Jul 19 17:21:43.601474 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:43.601440 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b27ad183-4912-4501-b9a5-30b9fa9bd271" path="/var/lib/kubelet/pods/b27ad183-4912-4501-b9a5-30b9fa9bd271/volumes" Jul 19 17:21:45.248721 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:45.248686 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj"] Jul 19 17:21:45.249125 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:45.249081 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2ff6b2c-753c-4b40-bcef-cc2c6161a56e" containerName="istio-proxy" Jul 19 17:21:45.249125 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:45.249096 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2ff6b2c-753c-4b40-bcef-cc2c6161a56e" containerName="istio-proxy" Jul 19 17:21:45.249125 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:45.249110 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b27ad183-4912-4501-b9a5-30b9fa9bd271" containerName="istio-proxy" Jul 19 17:21:45.249125 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:45.249118 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b27ad183-4912-4501-b9a5-30b9fa9bd271" containerName="istio-proxy" Jul 19 17:21:45.249269 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:45.249138 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b27ad183-4912-4501-b9a5-30b9fa9bd271" containerName="istio-proxy" Jul 19 17:21:45.249269 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:45.249144 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b27ad183-4912-4501-b9a5-30b9fa9bd271" containerName="istio-proxy" Jul 19 17:21:45.249269 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:45.249224 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="b27ad183-4912-4501-b9a5-30b9fa9bd271" containerName="istio-proxy" Jul 19 17:21:45.249269 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:45.249233 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="b27ad183-4912-4501-b9a5-30b9fa9bd271" containerName="istio-proxy" Jul 19 17:21:45.253356 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:45.253331 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj" Jul 19 17:21:45.255469 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:45.255436 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-0fa4bd-openshift-default-dockercfg-hhmz5\"" Jul 19 17:21:45.264912 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:45.264885 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj"] Jul 19 17:21:45.367443 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:45.367372 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e40b6b10-37a8-4240-bdea-2332df9fd1e8-istio-data\") pod \"e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj\" (UID: \"e40b6b10-37a8-4240-bdea-2332df9fd1e8\") " pod="openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj" Jul 19 17:21:45.367443 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:45.367428 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e40b6b10-37a8-4240-bdea-2332df9fd1e8-istio-token\") pod \"e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj\" (UID: \"e40b6b10-37a8-4240-bdea-2332df9fd1e8\") " pod="openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj" Jul 19 17:21:45.367443 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:45.367456 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e40b6b10-37a8-4240-bdea-2332df9fd1e8-istiod-ca-cert\") pod \"e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj\" (UID: \"e40b6b10-37a8-4240-bdea-2332df9fd1e8\") " pod="openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj" Jul 19 17:21:45.367772 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:45.367483 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2dqg\" (UniqueName: \"kubernetes.io/projected/e40b6b10-37a8-4240-bdea-2332df9fd1e8-kube-api-access-h2dqg\") pod \"e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj\" (UID: \"e40b6b10-37a8-4240-bdea-2332df9fd1e8\") " pod="openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj" Jul 19 17:21:45.367772 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:45.367519 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e40b6b10-37a8-4240-bdea-2332df9fd1e8-workload-certs\") pod \"e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj\" (UID: \"e40b6b10-37a8-4240-bdea-2332df9fd1e8\") " pod="openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj" Jul 19 17:21:45.367772 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:45.367558 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e40b6b10-37a8-4240-bdea-2332df9fd1e8-workload-socket\") pod \"e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj\" (UID: \"e40b6b10-37a8-4240-bdea-2332df9fd1e8\") " pod="openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj" Jul 19 17:21:45.367772 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:45.367600 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e40b6b10-37a8-4240-bdea-2332df9fd1e8-istio-podinfo\") pod \"e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj\" (UID: \"e40b6b10-37a8-4240-bdea-2332df9fd1e8\") " pod="openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj" Jul 19 17:21:45.367772 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:45.367617 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e40b6b10-37a8-4240-bdea-2332df9fd1e8-istio-envoy\") pod \"e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj\" (UID: \"e40b6b10-37a8-4240-bdea-2332df9fd1e8\") " pod="openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj" Jul 19 17:21:45.367772 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:45.367729 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e40b6b10-37a8-4240-bdea-2332df9fd1e8-credential-socket\") pod \"e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj\" (UID: \"e40b6b10-37a8-4240-bdea-2332df9fd1e8\") " pod="openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj" Jul 19 17:21:45.469279 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:45.469232 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e40b6b10-37a8-4240-bdea-2332df9fd1e8-credential-socket\") pod \"e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj\" (UID: \"e40b6b10-37a8-4240-bdea-2332df9fd1e8\") " pod="openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj" Jul 19 17:21:45.469455 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:45.469324 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e40b6b10-37a8-4240-bdea-2332df9fd1e8-istio-data\") pod \"e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj\" (UID: \"e40b6b10-37a8-4240-bdea-2332df9fd1e8\") " pod="openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj" Jul 19 17:21:45.469455 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:45.469354 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e40b6b10-37a8-4240-bdea-2332df9fd1e8-istio-token\") pod \"e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj\" (UID: \"e40b6b10-37a8-4240-bdea-2332df9fd1e8\") " pod="openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj" Jul 19 17:21:45.469455 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:45.469378 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e40b6b10-37a8-4240-bdea-2332df9fd1e8-istiod-ca-cert\") pod \"e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj\" (UID: \"e40b6b10-37a8-4240-bdea-2332df9fd1e8\") " pod="openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj" Jul 19 17:21:45.469581 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:45.469509 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2dqg\" (UniqueName: \"kubernetes.io/projected/e40b6b10-37a8-4240-bdea-2332df9fd1e8-kube-api-access-h2dqg\") pod \"e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj\" (UID: \"e40b6b10-37a8-4240-bdea-2332df9fd1e8\") " pod="openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj" Jul 19 17:21:45.469581 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:45.469569 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e40b6b10-37a8-4240-bdea-2332df9fd1e8-workload-certs\") pod \"e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj\" (UID: \"e40b6b10-37a8-4240-bdea-2332df9fd1e8\") " pod="openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj" Jul 19 17:21:45.469681 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:45.469601 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e40b6b10-37a8-4240-bdea-2332df9fd1e8-workload-socket\") pod \"e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj\" (UID: \"e40b6b10-37a8-4240-bdea-2332df9fd1e8\") " pod="openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj" Jul 19 17:21:45.469681 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:45.469659 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e40b6b10-37a8-4240-bdea-2332df9fd1e8-istio-podinfo\") pod \"e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj\" (UID: \"e40b6b10-37a8-4240-bdea-2332df9fd1e8\") " pod="openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj" Jul 19 17:21:45.469790 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:45.469686 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e40b6b10-37a8-4240-bdea-2332df9fd1e8-istio-envoy\") pod \"e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj\" (UID: \"e40b6b10-37a8-4240-bdea-2332df9fd1e8\") " pod="openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj" Jul 19 17:21:45.469790 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:45.469726 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e40b6b10-37a8-4240-bdea-2332df9fd1e8-istio-data\") pod \"e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj\" (UID: \"e40b6b10-37a8-4240-bdea-2332df9fd1e8\") " pod="openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj" Jul 19 17:21:45.469790 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:45.469742 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e40b6b10-37a8-4240-bdea-2332df9fd1e8-credential-socket\") pod \"e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj\" (UID: \"e40b6b10-37a8-4240-bdea-2332df9fd1e8\") " pod="openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj" Jul 19 17:21:45.469949 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:45.469926 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e40b6b10-37a8-4240-bdea-2332df9fd1e8-workload-certs\") pod \"e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj\" (UID: \"e40b6b10-37a8-4240-bdea-2332df9fd1e8\") " pod="openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj" Jul 19 17:21:45.470000 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:45.469969 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e40b6b10-37a8-4240-bdea-2332df9fd1e8-workload-socket\") pod \"e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj\" (UID: \"e40b6b10-37a8-4240-bdea-2332df9fd1e8\") " pod="openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj" Jul 19 17:21:45.470208 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:45.470183 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e40b6b10-37a8-4240-bdea-2332df9fd1e8-istiod-ca-cert\") pod \"e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj\" (UID: \"e40b6b10-37a8-4240-bdea-2332df9fd1e8\") " pod="openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj" Jul 19 17:21:45.471866 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:45.471839 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e40b6b10-37a8-4240-bdea-2332df9fd1e8-istio-envoy\") pod \"e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj\" (UID: \"e40b6b10-37a8-4240-bdea-2332df9fd1e8\") " pod="openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj" Jul 19 17:21:45.472011 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:45.471991 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e40b6b10-37a8-4240-bdea-2332df9fd1e8-istio-podinfo\") pod \"e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj\" (UID: \"e40b6b10-37a8-4240-bdea-2332df9fd1e8\") " pod="openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj" Jul 19 17:21:45.476693 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:45.476670 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2dqg\" (UniqueName: \"kubernetes.io/projected/e40b6b10-37a8-4240-bdea-2332df9fd1e8-kube-api-access-h2dqg\") pod \"e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj\" (UID: \"e40b6b10-37a8-4240-bdea-2332df9fd1e8\") " pod="openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj" Jul 19 17:21:45.477206 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:45.477186 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e40b6b10-37a8-4240-bdea-2332df9fd1e8-istio-token\") pod \"e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj\" (UID: \"e40b6b10-37a8-4240-bdea-2332df9fd1e8\") " pod="openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj" Jul 19 17:21:45.566103 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:45.566000 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj" Jul 19 17:21:45.702148 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:45.702119 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj"] Jul 19 17:21:45.703634 ip-10-0-134-203 kubenswrapper[2583]: W0719 17:21:45.703609 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode40b6b10_37a8_4240_bdea_2332df9fd1e8.slice/crio-215e4ac3dae96bc876327a0dacf5f66c8bc7af5bed0d9c7e082ff83ce047e5a3 WatchSource:0}: Error finding container 215e4ac3dae96bc876327a0dacf5f66c8bc7af5bed0d9c7e082ff83ce047e5a3: Status 404 returned error can't find the container with id 215e4ac3dae96bc876327a0dacf5f66c8bc7af5bed0d9c7e082ff83ce047e5a3 Jul 19 17:21:45.705747 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:45.705711 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:21:45.705856 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:45.705780 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:21:45.705856 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:45.705813 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:21:46.195473 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:46.195436 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj" event={"ID":"e40b6b10-37a8-4240-bdea-2332df9fd1e8","Type":"ContainerStarted","Data":"61be982dc2c8e2ddfbce68d2a53bc1ce75eda6d10db2dd09d9585610a384d26f"} Jul 19 17:21:46.195631 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:46.195492 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj" event={"ID":"e40b6b10-37a8-4240-bdea-2332df9fd1e8","Type":"ContainerStarted","Data":"215e4ac3dae96bc876327a0dacf5f66c8bc7af5bed0d9c7e082ff83ce047e5a3"} Jul 19 17:21:46.214021 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:46.213958 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj" podStartSLOduration=1.213940322 podStartE2EDuration="1.213940322s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 17:21:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 17:21:46.213099907 +0000 UTC m=+2601.199675392" watchObservedRunningTime="2026-07-19 17:21:46.213940322 +0000 UTC m=+2601.200515807" Jul 19 17:21:46.566901 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:46.566807 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj" Jul 19 17:21:46.568449 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:46.568420 2583 patch_prober.go:28] interesting pod/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 19 17:21:46.568574 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:46.568470 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj" podUID="e40b6b10-37a8-4240-bdea-2332df9fd1e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 19 17:21:47.570810 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:47.570731 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj" Jul 19 17:21:48.205200 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:48.205173 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj" Jul 19 17:21:48.206203 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:48.206180 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj" Jul 19 17:21:55.706805 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:55.706307 2583 patch_prober.go:28] interesting pod/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": read tcp 10.134.0.2:40576->10.134.0.39:15021: read: connection reset by peer" start-of-body= Jul 19 17:21:55.706805 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:55.706377 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw" podUID="b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": read tcp 10.134.0.2:40576->10.134.0.39:15021: read: connection reset by peer" Jul 19 17:21:56.250082 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:56.250025 2583 generic.go:356] "Generic (PLEG): container finished" podID="b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb" containerID="2490de64d8dff989b7fcdc795413fae8a6906d4df53c2b0577e836b21d83a856" exitCode=0 Jul 19 17:21:56.250292 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:56.250100 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw" event={"ID":"b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb","Type":"ContainerDied","Data":"2490de64d8dff989b7fcdc795413fae8a6906d4df53c2b0577e836b21d83a856"} Jul 19 17:21:56.250638 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:56.250592 2583 scope.go:117] "RemoveContainer" containerID="2490de64d8dff989b7fcdc795413fae8a6906d4df53c2b0577e836b21d83a856" Jul 19 17:21:56.252365 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:56.252322 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:21:56.252470 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:56.252420 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:21:56.252517 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:56.252470 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:21:57.266151 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:57.266108 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw" event={"ID":"b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb","Type":"ContainerStarted","Data":"4e5148f57e2f8c8c7e58bfbfcb64b3b7ceb3af46ddf531ced2e2b6ac3f06c92d"} Jul 19 17:21:57.829251 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:57.829209 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw" Jul 19 17:21:57.829470 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:57.829348 2583 patch_prober.go:28] interesting pod/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 19 17:21:57.829470 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:57.829395 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw" podUID="b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 19 17:21:58.829182 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:58.829140 2583 patch_prober.go:28] interesting pod/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 19 17:21:58.829674 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:58.829207 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw" podUID="b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 19 17:21:59.828972 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:59.828931 2583 patch_prober.go:28] interesting pod/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 19 17:21:59.829279 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:21:59.829240 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw" podUID="b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 19 17:22:00.829385 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:22:00.829339 2583 patch_prober.go:28] interesting pod/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 19 17:22:00.829864 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:22:00.829409 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw" podUID="b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 19 17:22:01.832541 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:22:01.832504 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw" Jul 19 17:22:01.832973 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:22:01.832784 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw" Jul 19 17:22:01.833604 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:22:01.833585 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw" Jul 19 17:22:03.207162 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:22:03.207122 2583 patch_prober.go:28] interesting pod/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 19 17:22:03.207572 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:22:03.207192 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj" podUID="e40b6b10-37a8-4240-bdea-2332df9fd1e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 19 17:22:03.298401 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:22:03.298363 2583 generic.go:356] "Generic (PLEG): container finished" podID="e40b6b10-37a8-4240-bdea-2332df9fd1e8" containerID="61be982dc2c8e2ddfbce68d2a53bc1ce75eda6d10db2dd09d9585610a384d26f" exitCode=0 Jul 19 17:22:03.298584 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:22:03.298436 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj" event={"ID":"e40b6b10-37a8-4240-bdea-2332df9fd1e8","Type":"ContainerDied","Data":"61be982dc2c8e2ddfbce68d2a53bc1ce75eda6d10db2dd09d9585610a384d26f"} Jul 19 17:22:03.299024 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:22:03.299007 2583 scope.go:117] "RemoveContainer" containerID="61be982dc2c8e2ddfbce68d2a53bc1ce75eda6d10db2dd09d9585610a384d26f" Jul 19 17:22:03.300093 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:22:03.300037 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:22:03.300165 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:22:03.300134 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:22:03.300213 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:22:03.300170 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 19 17:22:04.307122 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:22:04.307089 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj" event={"ID":"e40b6b10-37a8-4240-bdea-2332df9fd1e8","Type":"ContainerStarted","Data":"f4c1814256db3717bb01f34c98ab2a367432f543586196f21b8e23ef63370978"} Jul 19 17:22:04.567331 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:22:04.567232 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj" Jul 19 17:22:04.567508 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:22:04.567484 2583 patch_prober.go:28] interesting pod/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 19 17:22:04.567559 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:22:04.567537 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj" podUID="e40b6b10-37a8-4240-bdea-2332df9fd1e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 19 17:22:05.566807 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:22:05.566773 2583 patch_prober.go:28] interesting pod/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 19 17:22:05.567225 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:22:05.566832 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj" podUID="e40b6b10-37a8-4240-bdea-2332df9fd1e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 19 17:22:06.567289 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:22:06.567252 2583 patch_prober.go:28] interesting pod/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 19 17:22:06.567669 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:22:06.567310 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj" podUID="e40b6b10-37a8-4240-bdea-2332df9fd1e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 19 17:22:07.566688 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:22:07.566648 2583 patch_prober.go:28] interesting pod/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 19 17:22:07.566868 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:22:07.566717 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj" podUID="e40b6b10-37a8-4240-bdea-2332df9fd1e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 19 17:22:08.570709 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:22:08.570678 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj" Jul 19 17:22:08.571137 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:22:08.570944 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj" Jul 19 17:22:08.571610 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:22:08.571592 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj" Jul 19 17:28:04.734210 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:04.734174 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw"] Jul 19 17:28:04.734639 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:04.734402 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw" podUID="b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb" containerName="istio-proxy" containerID="cri-o://4e5148f57e2f8c8c7e58bfbfcb64b3b7ceb3af46ddf531ced2e2b6ac3f06c92d" gracePeriod=30 Jul 19 17:28:09.995930 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:09.995902 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw" Jul 19 17:28:10.029641 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:10.029556 2583 generic.go:356] "Generic (PLEG): container finished" podID="b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb" containerID="4e5148f57e2f8c8c7e58bfbfcb64b3b7ceb3af46ddf531ced2e2b6ac3f06c92d" exitCode=0 Jul 19 17:28:10.029641 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:10.029602 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw" Jul 19 17:28:10.029817 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:10.029596 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw" event={"ID":"b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb","Type":"ContainerDied","Data":"4e5148f57e2f8c8c7e58bfbfcb64b3b7ceb3af46ddf531ced2e2b6ac3f06c92d"} Jul 19 17:28:10.029817 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:10.029722 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw" event={"ID":"b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb","Type":"ContainerDied","Data":"f70eed990461daf5a953b24c4c098cbfc7c4f7f4b4ee8b7b8b568748ba026041"} Jul 19 17:28:10.029817 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:10.029744 2583 scope.go:117] "RemoveContainer" containerID="4e5148f57e2f8c8c7e58bfbfcb64b3b7ceb3af46ddf531ced2e2b6ac3f06c92d" Jul 19 17:28:10.038628 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:10.038607 2583 scope.go:117] "RemoveContainer" containerID="2490de64d8dff989b7fcdc795413fae8a6906d4df53c2b0577e836b21d83a856" Jul 19 17:28:10.048156 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:10.048139 2583 scope.go:117] "RemoveContainer" containerID="4e5148f57e2f8c8c7e58bfbfcb64b3b7ceb3af46ddf531ced2e2b6ac3f06c92d" Jul 19 17:28:10.048421 ip-10-0-134-203 kubenswrapper[2583]: E0719 17:28:10.048399 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e5148f57e2f8c8c7e58bfbfcb64b3b7ceb3af46ddf531ced2e2b6ac3f06c92d\": container with ID starting with 4e5148f57e2f8c8c7e58bfbfcb64b3b7ceb3af46ddf531ced2e2b6ac3f06c92d not found: ID does not exist" containerID="4e5148f57e2f8c8c7e58bfbfcb64b3b7ceb3af46ddf531ced2e2b6ac3f06c92d" Jul 19 17:28:10.048496 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:10.048432 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e5148f57e2f8c8c7e58bfbfcb64b3b7ceb3af46ddf531ced2e2b6ac3f06c92d"} err="failed to get container status \"4e5148f57e2f8c8c7e58bfbfcb64b3b7ceb3af46ddf531ced2e2b6ac3f06c92d\": rpc error: code = NotFound desc = could not find container \"4e5148f57e2f8c8c7e58bfbfcb64b3b7ceb3af46ddf531ced2e2b6ac3f06c92d\": container with ID starting with 4e5148f57e2f8c8c7e58bfbfcb64b3b7ceb3af46ddf531ced2e2b6ac3f06c92d not found: ID does not exist" Jul 19 17:28:10.048496 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:10.048450 2583 scope.go:117] "RemoveContainer" containerID="2490de64d8dff989b7fcdc795413fae8a6906d4df53c2b0577e836b21d83a856" Jul 19 17:28:10.048733 ip-10-0-134-203 kubenswrapper[2583]: E0719 17:28:10.048709 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2490de64d8dff989b7fcdc795413fae8a6906d4df53c2b0577e836b21d83a856\": container with ID starting with 2490de64d8dff989b7fcdc795413fae8a6906d4df53c2b0577e836b21d83a856 not found: ID does not exist" containerID="2490de64d8dff989b7fcdc795413fae8a6906d4df53c2b0577e836b21d83a856" Jul 19 17:28:10.048783 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:10.048734 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2490de64d8dff989b7fcdc795413fae8a6906d4df53c2b0577e836b21d83a856"} err="failed to get container status \"2490de64d8dff989b7fcdc795413fae8a6906d4df53c2b0577e836b21d83a856\": rpc error: code = NotFound desc = could not find container \"2490de64d8dff989b7fcdc795413fae8a6906d4df53c2b0577e836b21d83a856\": container with ID starting with 2490de64d8dff989b7fcdc795413fae8a6906d4df53c2b0577e836b21d83a856 not found: ID does not exist" Jul 19 17:28:10.083149 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:10.083112 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb-istio-podinfo\") pod \"b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb\" (UID: \"b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb\") " Jul 19 17:28:10.083291 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:10.083197 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb-workload-certs\") pod \"b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb\" (UID: \"b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb\") " Jul 19 17:28:10.083370 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:10.083349 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb-istio-token\") pod \"b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb\" (UID: \"b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb\") " Jul 19 17:28:10.083484 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:10.083463 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb" (UID: "b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:28:10.083538 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:10.083483 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb-istio-envoy\") pod \"b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb\" (UID: \"b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb\") " Jul 19 17:28:10.083538 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:10.083507 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb-workload-socket\") pod \"b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb\" (UID: \"b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb\") " Jul 19 17:28:10.083538 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:10.083531 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb-istiod-ca-cert\") pod \"b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb\" (UID: \"b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb\") " Jul 19 17:28:10.083687 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:10.083556 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fb779\" (UniqueName: \"kubernetes.io/projected/b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb-kube-api-access-fb779\") pod \"b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb\" (UID: \"b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb\") " Jul 19 17:28:10.083687 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:10.083636 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb-credential-socket\") pod \"b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb\" (UID: \"b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb\") " Jul 19 17:28:10.083687 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:10.083669 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb-istio-data\") pod \"b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb\" (UID: \"b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb\") " Jul 19 17:28:10.083843 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:10.083770 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb" (UID: "b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:28:10.083980 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:10.083958 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb" (UID: "b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:28:10.083980 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:10.083887 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb" (UID: "b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 19 17:28:10.084137 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:10.084116 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb-workload-socket\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:28:10.084192 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:10.084144 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb-istiod-ca-cert\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:28:10.084192 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:10.084160 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb-credential-socket\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:28:10.084192 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:10.084174 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb-workload-certs\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:28:10.084937 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:10.084903 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb-istio-data" (OuterVolumeSpecName: "istio-data") pod "b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb" (UID: "b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:28:10.085363 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:10.085344 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb" (UID: "b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 19 17:28:10.085727 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:10.085704 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb" (UID: "b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:28:10.085808 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:10.085740 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb-istio-token" (OuterVolumeSpecName: "istio-token") pod "b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb" (UID: "b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 19 17:28:10.086120 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:10.086091 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb-kube-api-access-fb779" (OuterVolumeSpecName: "kube-api-access-fb779") pod "b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb" (UID: "b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb"). InnerVolumeSpecName "kube-api-access-fb779". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 19 17:28:10.184829 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:10.184789 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb-istio-data\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:28:10.184829 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:10.184822 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb-istio-podinfo\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:28:10.184829 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:10.184832 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb-istio-token\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:28:10.184829 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:10.184840 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb-istio-envoy\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:28:10.185146 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:10.184849 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fb779\" (UniqueName: \"kubernetes.io/projected/b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb-kube-api-access-fb779\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:28:10.351585 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:10.351550 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw"] Jul 19 17:28:10.356512 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:10.356487 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-8d2863-openshift-default-5b99887d75-r2fcw"] Jul 19 17:28:11.602375 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:11.602341 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb" path="/var/lib/kubelet/pods/b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb/volumes" Jul 19 17:28:26.693689 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:26.693653 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj"] Jul 19 17:28:26.694229 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:26.693885 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj" podUID="e40b6b10-37a8-4240-bdea-2332df9fd1e8" containerName="istio-proxy" containerID="cri-o://f4c1814256db3717bb01f34c98ab2a367432f543586196f21b8e23ef63370978" gracePeriod=30 Jul 19 17:28:31.946087 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:31.946043 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj" Jul 19 17:28:32.095215 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:32.095187 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e40b6b10-37a8-4240-bdea-2332df9fd1e8-credential-socket\") pod \"e40b6b10-37a8-4240-bdea-2332df9fd1e8\" (UID: \"e40b6b10-37a8-4240-bdea-2332df9fd1e8\") " Jul 19 17:28:32.095381 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:32.095246 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e40b6b10-37a8-4240-bdea-2332df9fd1e8-istiod-ca-cert\") pod \"e40b6b10-37a8-4240-bdea-2332df9fd1e8\" (UID: \"e40b6b10-37a8-4240-bdea-2332df9fd1e8\") " Jul 19 17:28:32.095381 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:32.095270 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e40b6b10-37a8-4240-bdea-2332df9fd1e8-istio-envoy\") pod \"e40b6b10-37a8-4240-bdea-2332df9fd1e8\" (UID: \"e40b6b10-37a8-4240-bdea-2332df9fd1e8\") " Jul 19 17:28:32.095381 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:32.095320 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e40b6b10-37a8-4240-bdea-2332df9fd1e8-istio-podinfo\") pod \"e40b6b10-37a8-4240-bdea-2332df9fd1e8\" (UID: \"e40b6b10-37a8-4240-bdea-2332df9fd1e8\") " Jul 19 17:28:32.095381 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:32.095354 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e40b6b10-37a8-4240-bdea-2332df9fd1e8-istio-token\") pod \"e40b6b10-37a8-4240-bdea-2332df9fd1e8\" (UID: \"e40b6b10-37a8-4240-bdea-2332df9fd1e8\") " Jul 19 17:28:32.095602 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:32.095417 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h2dqg\" (UniqueName: \"kubernetes.io/projected/e40b6b10-37a8-4240-bdea-2332df9fd1e8-kube-api-access-h2dqg\") pod \"e40b6b10-37a8-4240-bdea-2332df9fd1e8\" (UID: \"e40b6b10-37a8-4240-bdea-2332df9fd1e8\") " Jul 19 17:28:32.095602 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:32.095441 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e40b6b10-37a8-4240-bdea-2332df9fd1e8-workload-certs\") pod \"e40b6b10-37a8-4240-bdea-2332df9fd1e8\" (UID: \"e40b6b10-37a8-4240-bdea-2332df9fd1e8\") " Jul 19 17:28:32.095602 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:32.095485 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e40b6b10-37a8-4240-bdea-2332df9fd1e8-istio-data\") pod \"e40b6b10-37a8-4240-bdea-2332df9fd1e8\" (UID: \"e40b6b10-37a8-4240-bdea-2332df9fd1e8\") " Jul 19 17:28:32.095602 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:32.095481 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e40b6b10-37a8-4240-bdea-2332df9fd1e8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e40b6b10-37a8-4240-bdea-2332df9fd1e8" (UID: "e40b6b10-37a8-4240-bdea-2332df9fd1e8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:28:32.095602 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:32.095522 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e40b6b10-37a8-4240-bdea-2332df9fd1e8-workload-socket\") pod \"e40b6b10-37a8-4240-bdea-2332df9fd1e8\" (UID: \"e40b6b10-37a8-4240-bdea-2332df9fd1e8\") " Jul 19 17:28:32.095856 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:32.095720 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e40b6b10-37a8-4240-bdea-2332df9fd1e8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e40b6b10-37a8-4240-bdea-2332df9fd1e8" (UID: "e40b6b10-37a8-4240-bdea-2332df9fd1e8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:28:32.095856 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:32.095725 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e40b6b10-37a8-4240-bdea-2332df9fd1e8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e40b6b10-37a8-4240-bdea-2332df9fd1e8" (UID: "e40b6b10-37a8-4240-bdea-2332df9fd1e8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 19 17:28:32.095955 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:32.095891 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e40b6b10-37a8-4240-bdea-2332df9fd1e8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e40b6b10-37a8-4240-bdea-2332df9fd1e8" (UID: "e40b6b10-37a8-4240-bdea-2332df9fd1e8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:28:32.095955 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:32.095905 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e40b6b10-37a8-4240-bdea-2332df9fd1e8-workload-certs\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:28:32.095955 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:32.095924 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e40b6b10-37a8-4240-bdea-2332df9fd1e8-credential-socket\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:28:32.095955 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:32.095937 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e40b6b10-37a8-4240-bdea-2332df9fd1e8-istiod-ca-cert\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:28:32.096561 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:32.096531 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e40b6b10-37a8-4240-bdea-2332df9fd1e8-istio-data" (OuterVolumeSpecName: "istio-data") pod "e40b6b10-37a8-4240-bdea-2332df9fd1e8" (UID: "e40b6b10-37a8-4240-bdea-2332df9fd1e8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:28:32.097709 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:32.097684 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e40b6b10-37a8-4240-bdea-2332df9fd1e8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e40b6b10-37a8-4240-bdea-2332df9fd1e8" (UID: "e40b6b10-37a8-4240-bdea-2332df9fd1e8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 19 17:28:32.097796 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:32.097751 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e40b6b10-37a8-4240-bdea-2332df9fd1e8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e40b6b10-37a8-4240-bdea-2332df9fd1e8" (UID: "e40b6b10-37a8-4240-bdea-2332df9fd1e8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 19 17:28:32.097796 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:32.097775 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e40b6b10-37a8-4240-bdea-2332df9fd1e8-istio-token" (OuterVolumeSpecName: "istio-token") pod "e40b6b10-37a8-4240-bdea-2332df9fd1e8" (UID: "e40b6b10-37a8-4240-bdea-2332df9fd1e8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 19 17:28:32.097939 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:32.097920 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e40b6b10-37a8-4240-bdea-2332df9fd1e8-kube-api-access-h2dqg" (OuterVolumeSpecName: "kube-api-access-h2dqg") pod "e40b6b10-37a8-4240-bdea-2332df9fd1e8" (UID: "e40b6b10-37a8-4240-bdea-2332df9fd1e8"). InnerVolumeSpecName "kube-api-access-h2dqg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 19 17:28:32.134867 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:32.134839 2583 generic.go:356] "Generic (PLEG): container finished" podID="e40b6b10-37a8-4240-bdea-2332df9fd1e8" containerID="f4c1814256db3717bb01f34c98ab2a367432f543586196f21b8e23ef63370978" exitCode=0 Jul 19 17:28:32.135017 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:32.134889 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj" Jul 19 17:28:32.135017 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:32.134891 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj" event={"ID":"e40b6b10-37a8-4240-bdea-2332df9fd1e8","Type":"ContainerDied","Data":"f4c1814256db3717bb01f34c98ab2a367432f543586196f21b8e23ef63370978"} Jul 19 17:28:32.135175 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:32.135022 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj" event={"ID":"e40b6b10-37a8-4240-bdea-2332df9fd1e8","Type":"ContainerDied","Data":"215e4ac3dae96bc876327a0dacf5f66c8bc7af5bed0d9c7e082ff83ce047e5a3"} Jul 19 17:28:32.135175 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:32.135080 2583 scope.go:117] "RemoveContainer" containerID="f4c1814256db3717bb01f34c98ab2a367432f543586196f21b8e23ef63370978" Jul 19 17:28:32.144274 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:32.144258 2583 scope.go:117] "RemoveContainer" containerID="61be982dc2c8e2ddfbce68d2a53bc1ce75eda6d10db2dd09d9585610a384d26f" Jul 19 17:28:32.152047 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:32.152030 2583 scope.go:117] "RemoveContainer" containerID="f4c1814256db3717bb01f34c98ab2a367432f543586196f21b8e23ef63370978" Jul 19 17:28:32.152326 ip-10-0-134-203 kubenswrapper[2583]: E0719 17:28:32.152304 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4c1814256db3717bb01f34c98ab2a367432f543586196f21b8e23ef63370978\": container with ID starting with f4c1814256db3717bb01f34c98ab2a367432f543586196f21b8e23ef63370978 not found: ID does not exist" containerID="f4c1814256db3717bb01f34c98ab2a367432f543586196f21b8e23ef63370978" Jul 19 17:28:32.152393 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:32.152333 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4c1814256db3717bb01f34c98ab2a367432f543586196f21b8e23ef63370978"} err="failed to get container status \"f4c1814256db3717bb01f34c98ab2a367432f543586196f21b8e23ef63370978\": rpc error: code = NotFound desc = could not find container \"f4c1814256db3717bb01f34c98ab2a367432f543586196f21b8e23ef63370978\": container with ID starting with f4c1814256db3717bb01f34c98ab2a367432f543586196f21b8e23ef63370978 not found: ID does not exist" Jul 19 17:28:32.152393 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:32.152350 2583 scope.go:117] "RemoveContainer" containerID="61be982dc2c8e2ddfbce68d2a53bc1ce75eda6d10db2dd09d9585610a384d26f" Jul 19 17:28:32.152579 ip-10-0-134-203 kubenswrapper[2583]: E0719 17:28:32.152564 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61be982dc2c8e2ddfbce68d2a53bc1ce75eda6d10db2dd09d9585610a384d26f\": container with ID starting with 61be982dc2c8e2ddfbce68d2a53bc1ce75eda6d10db2dd09d9585610a384d26f not found: ID does not exist" containerID="61be982dc2c8e2ddfbce68d2a53bc1ce75eda6d10db2dd09d9585610a384d26f" Jul 19 17:28:32.152616 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:32.152583 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61be982dc2c8e2ddfbce68d2a53bc1ce75eda6d10db2dd09d9585610a384d26f"} err="failed to get container status \"61be982dc2c8e2ddfbce68d2a53bc1ce75eda6d10db2dd09d9585610a384d26f\": rpc error: code = NotFound desc = could not find container \"61be982dc2c8e2ddfbce68d2a53bc1ce75eda6d10db2dd09d9585610a384d26f\": container with ID starting with 61be982dc2c8e2ddfbce68d2a53bc1ce75eda6d10db2dd09d9585610a384d26f not found: ID does not exist" Jul 19 17:28:32.156561 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:32.156540 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj"] Jul 19 17:28:32.161877 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:32.161856 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-0fa4bd-openshift-default-8f7d58fdc-55ktj"] Jul 19 17:28:32.197483 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:32.197452 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e40b6b10-37a8-4240-bdea-2332df9fd1e8-istio-envoy\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:28:32.197483 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:32.197481 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e40b6b10-37a8-4240-bdea-2332df9fd1e8-istio-podinfo\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:28:32.197483 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:32.197490 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e40b6b10-37a8-4240-bdea-2332df9fd1e8-istio-token\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:28:32.197664 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:32.197499 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h2dqg\" (UniqueName: \"kubernetes.io/projected/e40b6b10-37a8-4240-bdea-2332df9fd1e8-kube-api-access-h2dqg\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:28:32.197664 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:32.197509 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e40b6b10-37a8-4240-bdea-2332df9fd1e8-istio-data\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:28:32.197664 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:32.197517 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e40b6b10-37a8-4240-bdea-2332df9fd1e8-workload-socket\") on node \"ip-10-0-134-203.ec2.internal\" DevicePath \"\"" Jul 19 17:28:33.601850 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:33.601817 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e40b6b10-37a8-4240-bdea-2332df9fd1e8" path="/var/lib/kubelet/pods/e40b6b10-37a8-4240-bdea-2332df9fd1e8/volumes" Jul 19 17:28:34.025103 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:34.025002 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5f7db45576-hndzz_fab2e64a-f91c-4227-bfef-7f77d0538adb/copy-manifests/0.log" Jul 19 17:28:34.032566 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:34.032534 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5f7db45576-hndzz_fab2e64a-f91c-4227-bfef-7f77d0538adb/manager/0.log" Jul 19 17:28:37.585424 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:37.585389 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4kvhsx_2833b8cd-b6e8-454f-88ea-fd29daf550db/istio-proxy/0.log" Jul 19 17:28:37.712892 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:37.712860 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-fvj5h_53fb7f67-967e-46d9-baed-fbc03e25e04b/discovery/0.log" Jul 19 17:28:38.075710 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:38.075619 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-5frw9_9660736b-a005-469d-af64-0088155863a5/istio-proxy/1.log" Jul 19 17:28:38.452892 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:38.452864 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-755474c78d-xctf5_c580fe52-93d3-46c6-bf10-b90f9af30ee5/router/0.log" Jul 19 17:28:38.941275 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:38.941232 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-qd65d_0854ed41-d942-4cf3-b58f-0d396128a995/main/0.log" Jul 19 17:28:39.170490 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:39.170460 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzcg_c43c4dbe-e471-45fe-8dd5-465b5f2fc774/main/0.log" Jul 19 17:28:48.508028 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:48.507988 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-smdwh_ea859009-b3b3-4344-8d24-493c6a7547fa/global-pull-secret-syncer/0.log" Jul 19 17:28:48.610726 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:48.610687 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-vm6wh_afaa8a6b-a7e4-4e3e-b75e-edc55cd09956/konnectivity-agent/0.log" Jul 19 17:28:48.631053 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:48.631018 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-134-203.ec2.internal_e73bc9e2542284e7faf575627ba3b0a4/haproxy/0.log" Jul 19 17:28:55.214738 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:55.214707 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-bnvf8_dfeed6fd-e812-449b-ae0e-9746e3953fa3/kube-state-metrics/0.log" Jul 19 17:28:55.234827 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:55.234801 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-bnvf8_dfeed6fd-e812-449b-ae0e-9746e3953fa3/kube-rbac-proxy-main/0.log" Jul 19 17:28:55.256831 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:55.256805 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-bnvf8_dfeed6fd-e812-449b-ae0e-9746e3953fa3/kube-rbac-proxy-self/0.log" Jul 19 17:28:55.362257 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:55.362229 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4x8vz_b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8/node-exporter/0.log" Jul 19 17:28:55.385253 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:55.385228 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4x8vz_b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8/kube-rbac-proxy/0.log" Jul 19 17:28:55.411402 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:55.411381 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4x8vz_b569186e-76bf-4bc9-bd0f-b9aa4eddb2a8/init-textfile/0.log" Jul 19 17:28:55.747151 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:55.747040 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0e0a7170-1745-4ae0-8cc1-4713f1cc3d61/prometheus/0.log" Jul 19 17:28:55.773302 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:55.773277 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0e0a7170-1745-4ae0-8cc1-4713f1cc3d61/config-reloader/0.log" Jul 19 17:28:55.795205 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:55.795181 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0e0a7170-1745-4ae0-8cc1-4713f1cc3d61/thanos-sidecar/0.log" Jul 19 17:28:55.815515 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:55.815494 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0e0a7170-1745-4ae0-8cc1-4713f1cc3d61/kube-rbac-proxy-web/0.log" Jul 19 17:28:55.834648 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:55.834621 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0e0a7170-1745-4ae0-8cc1-4713f1cc3d61/kube-rbac-proxy/0.log" Jul 19 17:28:55.856985 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:55.856960 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0e0a7170-1745-4ae0-8cc1-4713f1cc3d61/kube-rbac-proxy-thanos/0.log" Jul 19 17:28:55.882071 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:55.882043 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0e0a7170-1745-4ae0-8cc1-4713f1cc3d61/init-config-reloader/0.log" Jul 19 17:28:55.907783 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:55.907754 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-p2ghv_c9c72ebb-d55e-4020-b6e0-e9a384bae506/prometheus-operator/0.log" Jul 19 17:28:55.929695 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:55.929673 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-p2ghv_c9c72ebb-d55e-4020-b6e0-e9a384bae506/kube-rbac-proxy/0.log" Jul 19 17:28:56.062854 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:56.062776 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-59bc996f9b-9jtxb_6cb1f54c-e94c-4d10-be59-e5fb67c1539b/thanos-query/0.log" Jul 19 17:28:56.084492 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:56.084462 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-59bc996f9b-9jtxb_6cb1f54c-e94c-4d10-be59-e5fb67c1539b/kube-rbac-proxy-web/0.log" Jul 19 17:28:56.107104 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:56.107052 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-59bc996f9b-9jtxb_6cb1f54c-e94c-4d10-be59-e5fb67c1539b/kube-rbac-proxy/0.log" Jul 19 17:28:56.129546 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:56.129520 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-59bc996f9b-9jtxb_6cb1f54c-e94c-4d10-be59-e5fb67c1539b/prom-label-proxy/0.log" Jul 19 17:28:56.149037 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:56.149010 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-59bc996f9b-9jtxb_6cb1f54c-e94c-4d10-be59-e5fb67c1539b/kube-rbac-proxy-rules/0.log" Jul 19 17:28:56.168020 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:56.167994 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-59bc996f9b-9jtxb_6cb1f54c-e94c-4d10-be59-e5fb67c1539b/kube-rbac-proxy-metrics/0.log" Jul 19 17:28:58.243132 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:58.243095 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-m6jsn/perf-node-gather-daemonset-7qxhs"] Jul 19 17:28:58.243577 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:58.243425 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e40b6b10-37a8-4240-bdea-2332df9fd1e8" containerName="istio-proxy" Jul 19 17:28:58.243577 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:58.243435 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="e40b6b10-37a8-4240-bdea-2332df9fd1e8" containerName="istio-proxy" Jul 19 17:28:58.243577 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:58.243446 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb" containerName="istio-proxy" Jul 19 17:28:58.243577 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:58.243452 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb" containerName="istio-proxy" Jul 19 17:28:58.243577 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:58.243540 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb" containerName="istio-proxy" Jul 19 17:28:58.243577 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:58.243548 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="e40b6b10-37a8-4240-bdea-2332df9fd1e8" containerName="istio-proxy" Jul 19 17:28:58.243577 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:58.243557 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="b025ac83-86ce-4c4a-a70e-5f6f5eeed3bb" containerName="istio-proxy" Jul 19 17:28:58.246727 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:58.246707 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-m6jsn/perf-node-gather-daemonset-7qxhs" Jul 19 17:28:58.248756 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:58.248731 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-m6jsn\"/\"default-dockercfg-xgbgc\"" Jul 19 17:28:58.248756 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:58.248730 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-m6jsn\"/\"openshift-service-ca.crt\"" Jul 19 17:28:58.249446 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:58.249429 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-m6jsn\"/\"kube-root-ca.crt\"" Jul 19 17:28:58.255693 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:58.255670 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-m6jsn/perf-node-gather-daemonset-7qxhs"] Jul 19 17:28:58.336109 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:58.336078 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/68c8cd44-462e-4be0-9487-8b38d356806f-podres\") pod \"perf-node-gather-daemonset-7qxhs\" (UID: \"68c8cd44-462e-4be0-9487-8b38d356806f\") " pod="openshift-must-gather-m6jsn/perf-node-gather-daemonset-7qxhs" Jul 19 17:28:58.336268 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:58.336124 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdxb4\" (UniqueName: \"kubernetes.io/projected/68c8cd44-462e-4be0-9487-8b38d356806f-kube-api-access-qdxb4\") pod \"perf-node-gather-daemonset-7qxhs\" (UID: \"68c8cd44-462e-4be0-9487-8b38d356806f\") " pod="openshift-must-gather-m6jsn/perf-node-gather-daemonset-7qxhs" Jul 19 17:28:58.336268 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:58.336151 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/68c8cd44-462e-4be0-9487-8b38d356806f-sys\") pod \"perf-node-gather-daemonset-7qxhs\" (UID: \"68c8cd44-462e-4be0-9487-8b38d356806f\") " pod="openshift-must-gather-m6jsn/perf-node-gather-daemonset-7qxhs" Jul 19 17:28:58.336268 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:58.336188 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/68c8cd44-462e-4be0-9487-8b38d356806f-lib-modules\") pod \"perf-node-gather-daemonset-7qxhs\" (UID: \"68c8cd44-462e-4be0-9487-8b38d356806f\") " pod="openshift-must-gather-m6jsn/perf-node-gather-daemonset-7qxhs" Jul 19 17:28:58.336268 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:58.336204 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/68c8cd44-462e-4be0-9487-8b38d356806f-proc\") pod \"perf-node-gather-daemonset-7qxhs\" (UID: \"68c8cd44-462e-4be0-9487-8b38d356806f\") " pod="openshift-must-gather-m6jsn/perf-node-gather-daemonset-7qxhs" Jul 19 17:28:58.437334 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:58.437293 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/68c8cd44-462e-4be0-9487-8b38d356806f-podres\") pod \"perf-node-gather-daemonset-7qxhs\" (UID: \"68c8cd44-462e-4be0-9487-8b38d356806f\") " pod="openshift-must-gather-m6jsn/perf-node-gather-daemonset-7qxhs" Jul 19 17:28:58.437489 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:58.437352 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdxb4\" (UniqueName: \"kubernetes.io/projected/68c8cd44-462e-4be0-9487-8b38d356806f-kube-api-access-qdxb4\") pod \"perf-node-gather-daemonset-7qxhs\" (UID: \"68c8cd44-462e-4be0-9487-8b38d356806f\") " pod="openshift-must-gather-m6jsn/perf-node-gather-daemonset-7qxhs" Jul 19 17:28:58.437489 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:58.437387 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/68c8cd44-462e-4be0-9487-8b38d356806f-sys\") pod \"perf-node-gather-daemonset-7qxhs\" (UID: \"68c8cd44-462e-4be0-9487-8b38d356806f\") " pod="openshift-must-gather-m6jsn/perf-node-gather-daemonset-7qxhs" Jul 19 17:28:58.437489 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:58.437419 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/68c8cd44-462e-4be0-9487-8b38d356806f-lib-modules\") pod \"perf-node-gather-daemonset-7qxhs\" (UID: \"68c8cd44-462e-4be0-9487-8b38d356806f\") " pod="openshift-must-gather-m6jsn/perf-node-gather-daemonset-7qxhs" Jul 19 17:28:58.437489 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:58.437476 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/68c8cd44-462e-4be0-9487-8b38d356806f-proc\") pod \"perf-node-gather-daemonset-7qxhs\" (UID: \"68c8cd44-462e-4be0-9487-8b38d356806f\") " pod="openshift-must-gather-m6jsn/perf-node-gather-daemonset-7qxhs" Jul 19 17:28:58.437688 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:58.437503 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/68c8cd44-462e-4be0-9487-8b38d356806f-sys\") pod \"perf-node-gather-daemonset-7qxhs\" (UID: \"68c8cd44-462e-4be0-9487-8b38d356806f\") " pod="openshift-must-gather-m6jsn/perf-node-gather-daemonset-7qxhs" Jul 19 17:28:58.437688 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:58.437483 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/68c8cd44-462e-4be0-9487-8b38d356806f-podres\") pod \"perf-node-gather-daemonset-7qxhs\" (UID: \"68c8cd44-462e-4be0-9487-8b38d356806f\") " pod="openshift-must-gather-m6jsn/perf-node-gather-daemonset-7qxhs" Jul 19 17:28:58.437688 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:58.437509 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/68c8cd44-462e-4be0-9487-8b38d356806f-lib-modules\") pod \"perf-node-gather-daemonset-7qxhs\" (UID: \"68c8cd44-462e-4be0-9487-8b38d356806f\") " pod="openshift-must-gather-m6jsn/perf-node-gather-daemonset-7qxhs" Jul 19 17:28:58.437688 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:58.437537 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/68c8cd44-462e-4be0-9487-8b38d356806f-proc\") pod \"perf-node-gather-daemonset-7qxhs\" (UID: \"68c8cd44-462e-4be0-9487-8b38d356806f\") " pod="openshift-must-gather-m6jsn/perf-node-gather-daemonset-7qxhs" Jul 19 17:28:58.444641 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:58.444610 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdxb4\" (UniqueName: \"kubernetes.io/projected/68c8cd44-462e-4be0-9487-8b38d356806f-kube-api-access-qdxb4\") pod \"perf-node-gather-daemonset-7qxhs\" (UID: \"68c8cd44-462e-4be0-9487-8b38d356806f\") " pod="openshift-must-gather-m6jsn/perf-node-gather-daemonset-7qxhs" Jul 19 17:28:58.557625 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:58.557536 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-m6jsn/perf-node-gather-daemonset-7qxhs" Jul 19 17:28:58.690668 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:58.690608 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-m6jsn/perf-node-gather-daemonset-7qxhs"] Jul 19 17:28:58.692967 ip-10-0-134-203 kubenswrapper[2583]: W0719 17:28:58.692942 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod68c8cd44_462e_4be0_9487_8b38d356806f.slice/crio-463f453cf19b38cb39b8f55ff72cf35a6a164670f4041245d7270b7fc119536b WatchSource:0}: Error finding container 463f453cf19b38cb39b8f55ff72cf35a6a164670f4041245d7270b7fc119536b: Status 404 returned error can't find the container with id 463f453cf19b38cb39b8f55ff72cf35a6a164670f4041245d7270b7fc119536b Jul 19 17:28:59.260218 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:59.260181 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-m6jsn/perf-node-gather-daemonset-7qxhs" event={"ID":"68c8cd44-462e-4be0-9487-8b38d356806f","Type":"ContainerStarted","Data":"3902952d993f8a10922404f6f7eb8ed7b6b76540f8f7b1de3fa62c7f467aa7dc"} Jul 19 17:28:59.260602 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:59.260235 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-m6jsn/perf-node-gather-daemonset-7qxhs" event={"ID":"68c8cd44-462e-4be0-9487-8b38d356806f","Type":"ContainerStarted","Data":"463f453cf19b38cb39b8f55ff72cf35a6a164670f4041245d7270b7fc119536b"} Jul 19 17:28:59.260602 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:59.260319 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-m6jsn/perf-node-gather-daemonset-7qxhs" Jul 19 17:28:59.275543 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:28:59.275495 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-m6jsn/perf-node-gather-daemonset-7qxhs" podStartSLOduration=1.275479264 podStartE2EDuration="1.275479264s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 17:28:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 17:28:59.273521019 +0000 UTC m=+3034.260096506" watchObservedRunningTime="2026-07-19 17:28:59.275479264 +0000 UTC m=+3034.262054774" Jul 19 17:29:00.276956 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:29:00.276927 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-znbhx_9e1a462c-d8cc-4635-9eac-2271aed8e352/dns/0.log" Jul 19 17:29:00.299044 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:29:00.299018 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-znbhx_9e1a462c-d8cc-4635-9eac-2271aed8e352/kube-rbac-proxy/0.log" Jul 19 17:29:00.343078 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:29:00.343027 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-cv2rk_c1a2fb73-9db3-436a-8b9b-f4e5e7355889/dns-node-resolver/0.log" Jul 19 17:29:00.890633 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:29:00.890602 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-j6dwh_07d22c2c-bf9e-47af-bdf6-8e535bad46b2/node-ca/0.log" Jul 19 17:29:01.820109 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:29:01.820077 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4kvhsx_2833b8cd-b6e8-454f-88ea-fd29daf550db/istio-proxy/0.log" Jul 19 17:29:01.942035 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:29:01.942004 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-fvj5h_53fb7f67-967e-46d9-baed-fbc03e25e04b/discovery/0.log" Jul 19 17:29:02.009113 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:29:02.009078 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-5frw9_9660736b-a005-469d-af64-0088155863a5/istio-proxy/0.log" Jul 19 17:29:02.036514 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:29:02.036482 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-5frw9_9660736b-a005-469d-af64-0088155863a5/istio-proxy/1.log" Jul 19 17:29:02.109754 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:29:02.109725 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-755474c78d-xctf5_c580fe52-93d3-46c6-bf10-b90f9af30ee5/router/0.log" Jul 19 17:29:02.804964 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:29:02.804934 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-txwnf_f133de90-2db7-4454-9d67-26e266dec955/serve-healthcheck-canary/0.log" Jul 19 17:29:03.273628 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:29:03.273604 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4cjfp_7387825e-ee08-4bcd-a6e3-7dcc34ed8036/kube-rbac-proxy/0.log" Jul 19 17:29:03.293758 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:29:03.293729 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4cjfp_7387825e-ee08-4bcd-a6e3-7dcc34ed8036/exporter/0.log" Jul 19 17:29:03.314487 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:29:03.314457 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4cjfp_7387825e-ee08-4bcd-a6e3-7dcc34ed8036/extractor/0.log" Jul 19 17:29:05.274011 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:29:05.273980 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-m6jsn/perf-node-gather-daemonset-7qxhs" Jul 19 17:29:05.928381 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:29:05.928350 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5f7db45576-hndzz_fab2e64a-f91c-4227-bfef-7f77d0538adb/manager/0.log" Jul 19 17:29:05.946517 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:29:05.946492 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5f7db45576-hndzz_fab2e64a-f91c-4227-bfef-7f77d0538adb/copy-manifests/0.log" Jul 19 17:29:07.235514 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:29:07.235470 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5dffb557f4-rndm6_9b779d11-4e6f-4da7-85cb-5c2aac527bd1/manager/0.log" Jul 19 17:29:08.167721 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:29:08.167640 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4kvhsx_2833b8cd-b6e8-454f-88ea-fd29daf550db/istio-proxy/0.log" Jul 19 17:29:08.287577 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:29:08.287546 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-fvj5h_53fb7f67-967e-46d9-baed-fbc03e25e04b/discovery/0.log" Jul 19 17:29:08.340046 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:29:08.340018 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-5frw9_9660736b-a005-469d-af64-0088155863a5/istio-proxy/0.log" Jul 19 17:29:08.364231 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:29:08.364202 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-5frw9_9660736b-a005-469d-af64-0088155863a5/istio-proxy/1.log" Jul 19 17:29:08.414128 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:29:08.414101 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-755474c78d-xctf5_c580fe52-93d3-46c6-bf10-b90f9af30ee5/router/0.log" Jul 19 17:29:09.478321 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:29:09.478291 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4kvhsx_2833b8cd-b6e8-454f-88ea-fd29daf550db/istio-proxy/0.log" Jul 19 17:29:09.598353 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:29:09.598326 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-fvj5h_53fb7f67-967e-46d9-baed-fbc03e25e04b/discovery/0.log" Jul 19 17:29:09.652000 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:29:09.651972 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-5frw9_9660736b-a005-469d-af64-0088155863a5/istio-proxy/0.log" Jul 19 17:29:09.679040 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:29:09.678999 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-5frw9_9660736b-a005-469d-af64-0088155863a5/istio-proxy/1.log" Jul 19 17:29:09.735886 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:29:09.735859 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-755474c78d-xctf5_c580fe52-93d3-46c6-bf10-b90f9af30ee5/router/0.log" Jul 19 17:29:13.614355 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:29:13.614330 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-spjzx_ab2a92d3-ad97-4dd4-9566-9b46f2f85de5/kube-multus-additional-cni-plugins/0.log" Jul 19 17:29:13.633798 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:29:13.633767 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-spjzx_ab2a92d3-ad97-4dd4-9566-9b46f2f85de5/egress-router-binary-copy/0.log" Jul 19 17:29:13.652201 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:29:13.652174 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-spjzx_ab2a92d3-ad97-4dd4-9566-9b46f2f85de5/cni-plugins/0.log" Jul 19 17:29:13.671461 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:29:13.671439 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-spjzx_ab2a92d3-ad97-4dd4-9566-9b46f2f85de5/bond-cni-plugin/0.log" Jul 19 17:29:13.689658 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:29:13.689632 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-spjzx_ab2a92d3-ad97-4dd4-9566-9b46f2f85de5/routeoverride-cni/0.log" Jul 19 17:29:13.708585 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:29:13.708565 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-spjzx_ab2a92d3-ad97-4dd4-9566-9b46f2f85de5/whereabouts-cni-bincopy/0.log" Jul 19 17:29:13.726942 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:29:13.726920 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-spjzx_ab2a92d3-ad97-4dd4-9566-9b46f2f85de5/whereabouts-cni/0.log" Jul 19 17:29:13.937902 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:29:13.937816 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-nqtmf_e4c13454-e438-40ed-b2fd-03a0d684fc8c/kube-multus/0.log" Jul 19 17:29:13.988268 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:29:13.988242 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-thqxw_9c487ff0-684c-41bd-aca1-74636fae5571/network-metrics-daemon/0.log" Jul 19 17:29:14.005537 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:29:14.005512 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-thqxw_9c487ff0-684c-41bd-aca1-74636fae5571/kube-rbac-proxy/0.log" Jul 19 17:29:15.013945 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:29:15.013916 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mt8d2_6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a/ovn-controller/0.log" Jul 19 17:29:15.030201 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:29:15.030176 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mt8d2_6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a/ovn-acl-logging/0.log" Jul 19 17:29:15.053156 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:29:15.053128 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mt8d2_6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a/kube-rbac-proxy-node/0.log" Jul 19 17:29:15.074786 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:29:15.074761 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mt8d2_6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a/kube-rbac-proxy-ovn-metrics/0.log" Jul 19 17:29:15.093344 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:29:15.093322 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mt8d2_6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a/northd/0.log" Jul 19 17:29:15.111095 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:29:15.111072 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mt8d2_6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a/nbdb/0.log" Jul 19 17:29:15.130273 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:29:15.130248 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mt8d2_6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a/sbdb/0.log" Jul 19 17:29:15.188636 ip-10-0-134-203 kubenswrapper[2583]: I0719 17:29:15.188591 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mt8d2_6e0fb240-17f5-46e2-aee6-3f8ef4a9b18a/ovnkube-controller/0.log"