Jul 30 00:15:27.288671 ip-10-0-133-10 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 30 00:15:27.288684 ip-10-0-133-10 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 30 00:15:27.288694 ip-10-0-133-10 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 30 00:15:27.288918 ip-10-0-133-10 systemd[1]: Failed to start Kubernetes Kubelet. Jul 30 00:15:37.538027 ip-10-0-133-10 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 30 00:15:37.538045 ip-10-0-133-10 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 2136457e9b894c968e43d2f86f47f205 -- Jul 30 00:17:13.689147 ip-10-0-133-10 systemd[1]: Starting Kubernetes Kubelet... Jul 30 00:17:14.125932 ip-10-0-133-10 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 30 00:17:14.125932 ip-10-0-133-10 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 30 00:17:14.125932 ip-10-0-133-10 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 30 00:17:14.125932 ip-10-0-133-10 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 30 00:17:14.125932 ip-10-0-133-10 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 30 00:17:14.126670 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.126001 2563 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 30 00:17:14.130176 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130158 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 30 00:17:14.130176 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130174 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 30 00:17:14.130249 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130179 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 30 00:17:14.130249 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130182 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 30 00:17:14.130249 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130187 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 30 00:17:14.130249 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130192 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 30 00:17:14.130249 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130195 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 30 00:17:14.130249 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130198 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 30 00:17:14.130249 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130201 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 30 00:17:14.130249 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130204 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 30 00:17:14.130249 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130207 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 30 00:17:14.130249 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130209 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 30 00:17:14.130249 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130213 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 30 00:17:14.130249 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130215 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 30 00:17:14.130249 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130218 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 30 00:17:14.130249 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130221 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 30 00:17:14.130249 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130223 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 30 00:17:14.130249 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130226 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 30 00:17:14.130249 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130229 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 30 00:17:14.130249 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130231 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 30 00:17:14.130249 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130234 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 30 00:17:14.130249 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130244 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 30 00:17:14.130740 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130247 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 30 00:17:14.130740 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130249 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 30 00:17:14.130740 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130252 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 30 00:17:14.130740 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130255 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 30 00:17:14.130740 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130258 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 30 00:17:14.130740 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130261 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 30 00:17:14.130740 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130263 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 30 00:17:14.130740 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130266 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 30 00:17:14.130740 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130269 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 30 00:17:14.130740 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130271 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 30 00:17:14.130740 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130274 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 30 00:17:14.130740 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130276 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 30 00:17:14.130740 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130279 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 30 00:17:14.130740 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130282 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 30 00:17:14.130740 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130284 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 30 00:17:14.130740 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130287 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 30 00:17:14.130740 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130290 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 30 00:17:14.130740 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130292 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 30 00:17:14.130740 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130294 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 30 00:17:14.130740 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130297 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 30 00:17:14.131234 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130300 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 30 00:17:14.131234 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130302 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 30 00:17:14.131234 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130304 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 30 00:17:14.131234 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130307 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 30 00:17:14.131234 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130310 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 30 00:17:14.131234 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130315 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 30 00:17:14.131234 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130317 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 30 00:17:14.131234 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130320 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 30 00:17:14.131234 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130322 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 30 00:17:14.131234 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130325 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 30 00:17:14.131234 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130327 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 30 00:17:14.131234 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130331 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 30 00:17:14.131234 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130335 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 30 00:17:14.131234 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130339 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 30 00:17:14.131234 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130342 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 30 00:17:14.131234 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130345 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 30 00:17:14.131234 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130347 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 30 00:17:14.131234 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130350 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 30 00:17:14.131234 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130353 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 30 00:17:14.131722 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130355 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 30 00:17:14.131722 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130358 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 30 00:17:14.131722 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130360 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 30 00:17:14.131722 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130363 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 30 00:17:14.131722 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130365 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 30 00:17:14.131722 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130368 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 30 00:17:14.131722 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130371 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 30 00:17:14.131722 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130373 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 30 00:17:14.131722 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130376 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 30 00:17:14.131722 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130378 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 30 00:17:14.131722 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130381 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 30 00:17:14.131722 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130384 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 30 00:17:14.131722 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130386 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 30 00:17:14.131722 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130389 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 30 00:17:14.131722 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130391 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 30 00:17:14.131722 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130394 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 30 00:17:14.131722 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130396 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 30 00:17:14.131722 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130399 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 30 00:17:14.131722 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130401 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 30 00:17:14.131722 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130404 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 30 00:17:14.132210 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130408 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 30 00:17:14.132210 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130410 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 30 00:17:14.132210 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130413 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 30 00:17:14.132210 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130415 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 30 00:17:14.132210 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130418 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 30 00:17:14.132210 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130420 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 30 00:17:14.132210 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130425 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 30 00:17:14.132210 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130428 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 30 00:17:14.132210 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130431 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 30 00:17:14.132210 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130435 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 30 00:17:14.132210 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130438 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 30 00:17:14.132210 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130441 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 30 00:17:14.132210 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130444 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 30 00:17:14.132210 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130446 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 30 00:17:14.132210 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130449 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 30 00:17:14.132210 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130451 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 30 00:17:14.132210 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130454 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 30 00:17:14.132210 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.130457 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 30 00:17:14.132210 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131220 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 30 00:17:14.132668 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131229 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 30 00:17:14.132668 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131232 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 30 00:17:14.132668 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131237 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 30 00:17:14.132668 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131241 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 30 00:17:14.132668 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131244 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 30 00:17:14.132668 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131246 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 30 00:17:14.132668 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131251 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 30 00:17:14.132668 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131254 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 30 00:17:14.132668 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131257 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 30 00:17:14.132668 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131262 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 30 00:17:14.132668 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131264 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 30 00:17:14.132668 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131269 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 30 00:17:14.132668 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131275 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 30 00:17:14.132668 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131279 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 30 00:17:14.132668 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131282 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 30 00:17:14.132668 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131287 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 30 00:17:14.132668 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131290 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 30 00:17:14.132668 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131293 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 30 00:17:14.132668 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131298 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 30 00:17:14.133232 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131302 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 30 00:17:14.133232 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131304 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 30 00:17:14.133232 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131310 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 30 00:17:14.133232 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131313 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 30 00:17:14.133232 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131316 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 30 00:17:14.133232 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131318 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 30 00:17:14.133232 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131323 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 30 00:17:14.133232 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131326 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 30 00:17:14.133232 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131329 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 30 00:17:14.133232 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131332 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 30 00:17:14.133232 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131336 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 30 00:17:14.133232 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131339 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 30 00:17:14.133232 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131342 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 30 00:17:14.133232 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131347 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 30 00:17:14.133232 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131350 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 30 00:17:14.133232 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131352 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 30 00:17:14.133232 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131357 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 30 00:17:14.133232 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131360 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 30 00:17:14.133232 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131362 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 30 00:17:14.133232 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131365 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 30 00:17:14.133763 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131368 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 30 00:17:14.133763 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131370 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 30 00:17:14.133763 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131373 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 30 00:17:14.133763 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131375 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 30 00:17:14.133763 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131378 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 30 00:17:14.133763 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131384 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 30 00:17:14.133763 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131387 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 30 00:17:14.133763 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131390 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 30 00:17:14.133763 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131395 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 30 00:17:14.133763 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131397 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 30 00:17:14.133763 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131400 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 30 00:17:14.133763 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131405 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 30 00:17:14.133763 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131408 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 30 00:17:14.133763 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131411 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 30 00:17:14.133763 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131414 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 30 00:17:14.133763 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131418 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 30 00:17:14.133763 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131421 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 30 00:17:14.133763 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131424 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 30 00:17:14.133763 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131426 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 30 00:17:14.133763 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131431 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 30 00:17:14.134264 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131434 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 30 00:17:14.134264 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131436 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 30 00:17:14.134264 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131441 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 30 00:17:14.134264 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131444 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 30 00:17:14.134264 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131447 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 30 00:17:14.134264 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131449 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 30 00:17:14.134264 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131454 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 30 00:17:14.134264 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131458 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 30 00:17:14.134264 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131461 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 30 00:17:14.134264 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131465 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 30 00:17:14.134264 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131468 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 30 00:17:14.134264 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131471 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 30 00:17:14.134264 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131473 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 30 00:17:14.134264 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131478 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 30 00:17:14.134264 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131481 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 30 00:17:14.134264 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131483 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 30 00:17:14.134264 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131486 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 30 00:17:14.134264 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131490 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 30 00:17:14.134264 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131493 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 30 00:17:14.134264 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131495 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 30 00:17:14.134755 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131501 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 30 00:17:14.134755 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131503 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 30 00:17:14.134755 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131506 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 30 00:17:14.134755 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131514 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 30 00:17:14.134755 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131519 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 30 00:17:14.134755 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131522 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 30 00:17:14.134755 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131527 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 30 00:17:14.134755 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131530 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 30 00:17:14.134755 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131533 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 30 00:17:14.134755 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131538 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 30 00:17:14.134755 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131541 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 30 00:17:14.134755 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131544 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 30 00:17:14.134755 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131548 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 30 00:17:14.134755 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131552 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 30 00:17:14.134755 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131554 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 30 00:17:14.134755 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131557 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 30 00:17:14.134755 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131563 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 30 00:17:14.134755 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131566 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 30 00:17:14.135203 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.131571 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 30 00:17:14.135203 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132057 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 30 00:17:14.135203 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132065 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 30 00:17:14.135203 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132068 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 30 00:17:14.135203 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132071 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 30 00:17:14.135203 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132074 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 30 00:17:14.135203 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132079 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 30 00:17:14.135203 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132082 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 30 00:17:14.135203 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132085 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 30 00:17:14.135203 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132088 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 30 00:17:14.135203 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132093 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 30 00:17:14.135203 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132096 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 30 00:17:14.135203 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132099 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 30 00:17:14.135203 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132104 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 30 00:17:14.135203 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132108 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 30 00:17:14.135203 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132111 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 30 00:17:14.135203 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132113 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 30 00:17:14.135203 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132120 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 30 00:17:14.135203 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132123 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 30 00:17:14.135203 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132126 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 30 00:17:14.135694 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132130 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 30 00:17:14.135694 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132133 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 30 00:17:14.135694 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132135 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 30 00:17:14.135694 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132138 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 30 00:17:14.135694 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132143 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 30 00:17:14.135694 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132145 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 30 00:17:14.135694 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132148 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 30 00:17:14.135694 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132151 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 30 00:17:14.135694 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132153 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 30 00:17:14.135694 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132158 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 30 00:17:14.135694 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132160 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 30 00:17:14.135694 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132163 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 30 00:17:14.135694 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132169 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 30 00:17:14.135694 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132174 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 30 00:17:14.135694 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132180 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 30 00:17:14.135694 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132183 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 30 00:17:14.135694 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132186 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 30 00:17:14.135694 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132189 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 30 00:17:14.135694 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132194 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 30 00:17:14.136184 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132197 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 30 00:17:14.136184 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132201 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 30 00:17:14.136184 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132206 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 30 00:17:14.136184 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132209 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 30 00:17:14.136184 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132212 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 30 00:17:14.136184 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132218 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 30 00:17:14.136184 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132221 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 30 00:17:14.136184 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132224 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 30 00:17:14.136184 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132229 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 30 00:17:14.136184 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132232 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 30 00:17:14.136184 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132236 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 30 00:17:14.136184 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132240 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 30 00:17:14.136184 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132243 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 30 00:17:14.136184 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132246 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 30 00:17:14.136184 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132249 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 30 00:17:14.136184 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132251 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 30 00:17:14.136184 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132254 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 30 00:17:14.136184 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132256 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 30 00:17:14.136184 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132259 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 30 00:17:14.136649 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132262 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 30 00:17:14.136649 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132264 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 30 00:17:14.136649 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132270 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 30 00:17:14.136649 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132273 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 30 00:17:14.136649 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132275 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 30 00:17:14.136649 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132280 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 30 00:17:14.136649 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132283 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 30 00:17:14.136649 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132286 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 30 00:17:14.136649 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132288 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 30 00:17:14.136649 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132292 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 30 00:17:14.136649 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132295 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 30 00:17:14.136649 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132298 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 30 00:17:14.136649 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132300 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 30 00:17:14.136649 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132310 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 30 00:17:14.136649 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132313 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 30 00:17:14.136649 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132318 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 30 00:17:14.136649 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132320 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 30 00:17:14.136649 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132323 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 30 00:17:14.136649 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132326 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 30 00:17:14.137139 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132331 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 30 00:17:14.137139 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132334 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 30 00:17:14.137139 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132336 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 30 00:17:14.137139 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132346 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 30 00:17:14.137139 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132350 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 30 00:17:14.137139 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132354 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 30 00:17:14.137139 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132357 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 30 00:17:14.137139 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132359 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 30 00:17:14.137139 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132362 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 30 00:17:14.137139 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132366 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 30 00:17:14.137139 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132369 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 30 00:17:14.137139 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132371 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 30 00:17:14.137139 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132374 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 30 00:17:14.137139 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132378 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 30 00:17:14.137139 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132381 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 30 00:17:14.137139 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132383 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 30 00:17:14.137139 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132386 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 30 00:17:14.137139 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132390 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 30 00:17:14.137139 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132393 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 30 00:17:14.137139 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132395 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 30 00:17:14.137623 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132398 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 30 00:17:14.137623 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132400 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 30 00:17:14.137623 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.132404 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 30 00:17:14.137623 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132501 2563 flags.go:64] FLAG: --address="0.0.0.0" Jul 30 00:17:14.137623 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132515 2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 30 00:17:14.137623 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132533 2563 flags.go:64] FLAG: --anonymous-auth="true" Jul 30 00:17:14.137623 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132543 2563 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 30 00:17:14.137623 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132547 2563 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 30 00:17:14.137623 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132551 2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 30 00:17:14.137623 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132557 2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 30 00:17:14.137623 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132562 2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 30 00:17:14.137623 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132565 2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 30 00:17:14.137623 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132570 2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 30 00:17:14.137623 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132574 2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 30 00:17:14.137623 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132579 2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 30 00:17:14.137623 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132582 2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 30 00:17:14.137623 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132586 2563 flags.go:64] FLAG: --cgroup-root="" Jul 30 00:17:14.137623 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132589 2563 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 30 00:17:14.137623 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132594 2563 flags.go:64] FLAG: --client-ca-file="" Jul 30 00:17:14.137623 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132597 2563 flags.go:64] FLAG: --cloud-provider="external" Jul 30 00:17:14.137623 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132600 2563 flags.go:64] FLAG: --cluster-dns="[]" Jul 30 00:17:14.137623 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132606 2563 flags.go:64] FLAG: --cluster-domain="" Jul 30 00:17:14.137623 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132609 2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 30 00:17:14.137623 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132612 2563 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 30 00:17:14.138258 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132615 2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 30 00:17:14.138258 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132620 2563 flags.go:64] FLAG: --container-log-max-files="5" Jul 30 00:17:14.138258 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132625 2563 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 30 00:17:14.138258 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132627 2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 30 00:17:14.138258 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132632 2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 30 00:17:14.138258 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132636 2563 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 30 00:17:14.138258 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132639 2563 flags.go:64] FLAG: --contention-profiling="false" Jul 30 00:17:14.138258 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132644 2563 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 30 00:17:14.138258 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132647 2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 30 00:17:14.138258 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132650 2563 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 30 00:17:14.138258 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132653 2563 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 30 00:17:14.138258 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132659 2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 30 00:17:14.138258 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132662 2563 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 30 00:17:14.138258 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132664 2563 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 30 00:17:14.138258 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132670 2563 flags.go:64] FLAG: --enable-load-reader="false" Jul 30 00:17:14.138258 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132673 2563 flags.go:64] FLAG: --enable-server="true" Jul 30 00:17:14.138258 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132678 2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 30 00:17:14.138258 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132683 2563 flags.go:64] FLAG: --event-burst="100" Jul 30 00:17:14.138258 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132686 2563 flags.go:64] FLAG: --event-qps="50" Jul 30 00:17:14.138258 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132689 2563 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 30 00:17:14.138258 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132693 2563 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 30 00:17:14.138258 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132696 2563 flags.go:64] FLAG: --eviction-hard="" Jul 30 00:17:14.138258 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132701 2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 30 00:17:14.138258 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132704 2563 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 30 00:17:14.138832 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132708 2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 30 00:17:14.138832 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132713 2563 flags.go:64] FLAG: --eviction-soft="" Jul 30 00:17:14.138832 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132716 2563 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 30 00:17:14.138832 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132719 2563 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 30 00:17:14.138832 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132722 2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 30 00:17:14.138832 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132727 2563 flags.go:64] FLAG: --experimental-mounter-path="" Jul 30 00:17:14.138832 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132730 2563 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 30 00:17:14.138832 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132733 2563 flags.go:64] FLAG: --fail-swap-on="true" Jul 30 00:17:14.138832 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132736 2563 flags.go:64] FLAG: --feature-gates="" Jul 30 00:17:14.138832 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132742 2563 flags.go:64] FLAG: --file-check-frequency="20s" Jul 30 00:17:14.138832 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132744 2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 30 00:17:14.138832 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132748 2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 30 00:17:14.138832 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132752 2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 30 00:17:14.138832 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132755 2563 flags.go:64] FLAG: --healthz-port="10248" Jul 30 00:17:14.138832 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132758 2563 flags.go:64] FLAG: --help="false" Jul 30 00:17:14.138832 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132763 2563 flags.go:64] FLAG: --hostname-override="ip-10-0-133-10.ec2.internal" Jul 30 00:17:14.138832 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132766 2563 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 30 00:17:14.138832 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132769 2563 flags.go:64] FLAG: --http-check-frequency="20s" Jul 30 00:17:14.138832 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132774 2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 30 00:17:14.138832 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132778 2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 30 00:17:14.138832 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132782 2563 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 30 00:17:14.138832 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132787 2563 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 30 00:17:14.138832 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132792 2563 flags.go:64] FLAG: --image-service-endpoint="" Jul 30 00:17:14.139477 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132795 2563 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 30 00:17:14.139477 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132799 2563 flags.go:64] FLAG: --kube-api-burst="100" Jul 30 00:17:14.139477 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132802 2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 30 00:17:14.139477 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132806 2563 flags.go:64] FLAG: --kube-api-qps="50" Jul 30 00:17:14.139477 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132808 2563 flags.go:64] FLAG: --kube-reserved="" Jul 30 00:17:14.139477 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132813 2563 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 30 00:17:14.139477 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132816 2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 30 00:17:14.139477 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132819 2563 flags.go:64] FLAG: --kubelet-cgroups="" Jul 30 00:17:14.139477 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132823 2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 30 00:17:14.139477 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132828 2563 flags.go:64] FLAG: --lock-file="" Jul 30 00:17:14.139477 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132831 2563 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 30 00:17:14.139477 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132840 2563 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 30 00:17:14.139477 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132843 2563 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 30 00:17:14.139477 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132850 2563 flags.go:64] FLAG: --log-json-split-stream="false" Jul 30 00:17:14.139477 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132853 2563 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 30 00:17:14.139477 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132857 2563 flags.go:64] FLAG: --log-text-split-stream="false" Jul 30 00:17:14.139477 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132873 2563 flags.go:64] FLAG: --logging-format="text" Jul 30 00:17:14.139477 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132876 2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 30 00:17:14.139477 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132879 2563 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 30 00:17:14.139477 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132884 2563 flags.go:64] FLAG: --manifest-url="" Jul 30 00:17:14.139477 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132887 2563 flags.go:64] FLAG: --manifest-url-header="" Jul 30 00:17:14.139477 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132893 2563 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 30 00:17:14.139477 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132897 2563 flags.go:64] FLAG: --max-open-files="1000000" Jul 30 00:17:14.139477 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132901 2563 flags.go:64] FLAG: --max-pods="110" Jul 30 00:17:14.139477 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132904 2563 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 30 00:17:14.140122 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132908 2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 30 00:17:14.140122 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132910 2563 flags.go:64] FLAG: --memory-manager-policy="None" Jul 30 00:17:14.140122 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132914 2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 30 00:17:14.140122 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132918 2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 30 00:17:14.140122 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132921 2563 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 30 00:17:14.140122 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132924 2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 30 00:17:14.140122 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132935 2563 flags.go:64] FLAG: --node-status-max-images="50" Jul 30 00:17:14.140122 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132942 2563 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 30 00:17:14.140122 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132947 2563 flags.go:64] FLAG: --oom-score-adj="-999" Jul 30 00:17:14.140122 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132955 2563 flags.go:64] FLAG: --pod-cidr="" Jul 30 00:17:14.140122 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132960 2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb" Jul 30 00:17:14.140122 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132968 2563 flags.go:64] FLAG: --pod-manifest-path="" Jul 30 00:17:14.140122 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132971 2563 flags.go:64] FLAG: --pod-max-pids="-1" Jul 30 00:17:14.140122 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132975 2563 flags.go:64] FLAG: --pods-per-core="0" Jul 30 00:17:14.140122 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132980 2563 flags.go:64] FLAG: --port="10250" Jul 30 00:17:14.140122 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132983 2563 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 30 00:17:14.140122 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132989 2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-063a67800ea7821c9" Jul 30 00:17:14.140122 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132993 2563 flags.go:64] FLAG: --qos-reserved="" Jul 30 00:17:14.140122 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.132996 2563 flags.go:64] FLAG: --read-only-port="10255" Jul 30 00:17:14.140122 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.133002 2563 flags.go:64] FLAG: --register-node="true" Jul 30 00:17:14.140122 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.133005 2563 flags.go:64] FLAG: --register-with-taints="" Jul 30 00:17:14.140122 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.133009 2563 flags.go:64] FLAG: --registry-burst="10" Jul 30 00:17:14.140122 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.133014 2563 flags.go:64] FLAG: --registry-qps="5" Jul 30 00:17:14.140122 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.133017 2563 flags.go:64] FLAG: --reserved-cpus="" Jul 30 00:17:14.140715 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.133020 2563 flags.go:64] FLAG: --reserved-memory="" Jul 30 00:17:14.140715 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.133031 2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 30 00:17:14.140715 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.133034 2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 30 00:17:14.140715 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.133037 2563 flags.go:64] FLAG: --rotate-certificates="false" Jul 30 00:17:14.140715 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.133042 2563 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 30 00:17:14.140715 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.133045 2563 flags.go:64] FLAG: --runonce="false" Jul 30 00:17:14.140715 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.133048 2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 30 00:17:14.140715 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.133054 2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 30 00:17:14.140715 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.133057 2563 flags.go:64] FLAG: --seccomp-default="false" Jul 30 00:17:14.140715 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.133060 2563 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 30 00:17:14.140715 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.133063 2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 30 00:17:14.140715 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.133068 2563 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 30 00:17:14.140715 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.133071 2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 30 00:17:14.140715 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.133075 2563 flags.go:64] FLAG: --storage-driver-password="root" Jul 30 00:17:14.140715 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.133079 2563 flags.go:64] FLAG: --storage-driver-secure="false" Jul 30 00:17:14.140715 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.133084 2563 flags.go:64] FLAG: --storage-driver-table="stats" Jul 30 00:17:14.140715 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.133087 2563 flags.go:64] FLAG: --storage-driver-user="root" Jul 30 00:17:14.140715 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.133092 2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 30 00:17:14.140715 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.133095 2563 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 30 00:17:14.140715 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.133118 2563 flags.go:64] FLAG: --system-cgroups="" Jul 30 00:17:14.140715 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.133122 2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 30 00:17:14.140715 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.133128 2563 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 30 00:17:14.140715 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.133132 2563 flags.go:64] FLAG: --tls-cert-file="" Jul 30 00:17:14.140715 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.133137 2563 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 30 00:17:14.140715 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.133143 2563 flags.go:64] FLAG: --tls-min-version="" Jul 30 00:17:14.141364 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.133147 2563 flags.go:64] FLAG: --tls-private-key-file="" Jul 30 00:17:14.141364 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.133150 2563 flags.go:64] FLAG: --topology-manager-policy="none" Jul 30 00:17:14.141364 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.133153 2563 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 30 00:17:14.141364 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.133159 2563 flags.go:64] FLAG: --topology-manager-scope="container" Jul 30 00:17:14.141364 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.133162 2563 flags.go:64] FLAG: --v="2" Jul 30 00:17:14.141364 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.133166 2563 flags.go:64] FLAG: --version="false" Jul 30 00:17:14.141364 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.133173 2563 flags.go:64] FLAG: --vmodule="" Jul 30 00:17:14.141364 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.133177 2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 30 00:17:14.141364 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.133182 2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 30 00:17:14.141364 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133322 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 30 00:17:14.141364 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133328 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 30 00:17:14.141364 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133331 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 30 00:17:14.141364 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133335 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 30 00:17:14.141364 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133338 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 30 00:17:14.141364 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133341 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 30 00:17:14.141364 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133344 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 30 00:17:14.141364 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133348 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 30 00:17:14.141364 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133351 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 30 00:17:14.141364 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133353 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 30 00:17:14.141364 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133356 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 30 00:17:14.141364 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133360 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 30 00:17:14.141364 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133363 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 30 00:17:14.141911 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133370 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 30 00:17:14.141911 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133375 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 30 00:17:14.141911 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133378 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 30 00:17:14.141911 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133383 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 30 00:17:14.141911 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133386 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 30 00:17:14.141911 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133389 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 30 00:17:14.141911 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133394 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 30 00:17:14.141911 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133397 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 30 00:17:14.141911 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133399 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 30 00:17:14.141911 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133406 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 30 00:17:14.141911 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133408 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 30 00:17:14.141911 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133411 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 30 00:17:14.141911 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133414 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 30 00:17:14.141911 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133419 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 30 00:17:14.141911 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133425 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 30 00:17:14.141911 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133430 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 30 00:17:14.141911 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133433 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 30 00:17:14.141911 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133436 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 30 00:17:14.141911 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133440 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 30 00:17:14.142387 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133444 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 30 00:17:14.142387 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133448 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 30 00:17:14.142387 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133453 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 30 00:17:14.142387 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133455 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 30 00:17:14.142387 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133458 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 30 00:17:14.142387 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133461 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 30 00:17:14.142387 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133465 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 30 00:17:14.142387 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133468 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 30 00:17:14.142387 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133470 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 30 00:17:14.142387 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133475 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 30 00:17:14.142387 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133477 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 30 00:17:14.142387 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133480 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 30 00:17:14.142387 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133482 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 30 00:17:14.142387 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133488 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 30 00:17:14.142387 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133490 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 30 00:17:14.142387 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133493 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 30 00:17:14.142387 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133498 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 30 00:17:14.142387 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133501 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 30 00:17:14.142387 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133504 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 30 00:17:14.142387 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133506 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 30 00:17:14.142941 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133511 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 30 00:17:14.142941 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133514 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 30 00:17:14.142941 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133517 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 30 00:17:14.142941 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133522 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 30 00:17:14.142941 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133525 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 30 00:17:14.142941 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133527 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 30 00:17:14.142941 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133531 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 30 00:17:14.142941 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133534 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 30 00:17:14.142941 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133537 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 30 00:17:14.142941 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133540 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 30 00:17:14.142941 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133543 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 30 00:17:14.142941 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133546 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 30 00:17:14.142941 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133548 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 30 00:17:14.142941 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133551 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 30 00:17:14.142941 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133553 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 30 00:17:14.142941 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133557 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 30 00:17:14.142941 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133560 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 30 00:17:14.142941 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133562 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 30 00:17:14.142941 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133565 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 30 00:17:14.143420 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133569 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 30 00:17:14.143420 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133572 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 30 00:17:14.143420 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133575 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 30 00:17:14.143420 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133577 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 30 00:17:14.143420 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133582 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 30 00:17:14.143420 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133585 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 30 00:17:14.143420 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133589 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 30 00:17:14.143420 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133593 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 30 00:17:14.143420 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133596 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 30 00:17:14.143420 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133599 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 30 00:17:14.143420 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133603 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 30 00:17:14.143420 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133606 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 30 00:17:14.143420 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133608 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 30 00:17:14.143420 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133611 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 30 00:17:14.143420 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133616 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 30 00:17:14.143420 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133619 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 30 00:17:14.143420 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133622 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 30 00:17:14.143420 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133627 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 30 00:17:14.143420 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133630 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 30 00:17:14.143420 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133634 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 30 00:17:14.143928 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133639 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 30 00:17:14.143928 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133642 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 30 00:17:14.143928 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133646 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 30 00:17:14.143928 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133651 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 30 00:17:14.143928 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133653 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 30 00:17:14.143928 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133656 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 30 00:17:14.143928 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133659 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 30 00:17:14.143928 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.133664 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 30 00:17:14.143928 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.134384 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 30 00:17:14.143928 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.141400 2563 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 30 00:17:14.143928 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.141416 2563 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 30 00:17:14.143928 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.141431 2563 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 30 00:17:14.143928 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.141440 2563 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 30 00:17:14.143928 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141500 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 30 00:17:14.144295 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141505 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 30 00:17:14.144295 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141509 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 30 00:17:14.144295 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141512 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 30 00:17:14.144295 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141515 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 30 00:17:14.144295 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141518 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 30 00:17:14.144295 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141521 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 30 00:17:14.144295 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141523 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 30 00:17:14.144295 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141526 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 30 00:17:14.144295 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141529 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 30 00:17:14.144295 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141531 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 30 00:17:14.144295 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141534 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 30 00:17:14.144295 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141536 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 30 00:17:14.144295 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141539 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 30 00:17:14.144295 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141541 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 30 00:17:14.144295 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141544 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 30 00:17:14.144295 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141547 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 30 00:17:14.144295 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141550 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 30 00:17:14.144295 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141552 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 30 00:17:14.144295 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141555 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 30 00:17:14.144295 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141558 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 30 00:17:14.144782 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141560 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 30 00:17:14.144782 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141563 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 30 00:17:14.144782 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141566 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 30 00:17:14.144782 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141569 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 30 00:17:14.144782 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141572 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 30 00:17:14.144782 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141576 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 30 00:17:14.144782 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141579 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 30 00:17:14.144782 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141582 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 30 00:17:14.144782 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141586 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 30 00:17:14.144782 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141589 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 30 00:17:14.144782 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141591 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 30 00:17:14.144782 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141594 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 30 00:17:14.144782 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141597 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 30 00:17:14.144782 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141599 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 30 00:17:14.144782 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141602 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 30 00:17:14.144782 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141605 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 30 00:17:14.144782 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141607 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 30 00:17:14.144782 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141610 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 30 00:17:14.144782 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141612 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 30 00:17:14.144782 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141615 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 30 00:17:14.145300 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141617 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 30 00:17:14.145300 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141620 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 30 00:17:14.145300 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141622 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 30 00:17:14.145300 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141625 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 30 00:17:14.145300 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141627 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 30 00:17:14.145300 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141630 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 30 00:17:14.145300 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141633 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 30 00:17:14.145300 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141635 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 30 00:17:14.145300 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141638 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 30 00:17:14.145300 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141640 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 30 00:17:14.145300 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141643 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 30 00:17:14.145300 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141645 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 30 00:17:14.145300 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141648 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 30 00:17:14.145300 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141650 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 30 00:17:14.145300 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141653 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 30 00:17:14.145300 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141657 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 30 00:17:14.145300 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141660 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 30 00:17:14.145300 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141664 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 30 00:17:14.145300 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141666 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 30 00:17:14.145762 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141669 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 30 00:17:14.145762 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141672 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 30 00:17:14.145762 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141676 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 30 00:17:14.145762 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141678 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 30 00:17:14.145762 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141681 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 30 00:17:14.145762 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141684 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 30 00:17:14.145762 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141686 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 30 00:17:14.145762 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141689 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 30 00:17:14.145762 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141692 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 30 00:17:14.145762 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141694 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 30 00:17:14.145762 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141696 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 30 00:17:14.145762 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141699 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 30 00:17:14.145762 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141701 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 30 00:17:14.145762 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141704 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 30 00:17:14.145762 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141706 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 30 00:17:14.145762 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141709 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 30 00:17:14.145762 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141712 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 30 00:17:14.145762 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141715 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 30 00:17:14.145762 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141717 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 30 00:17:14.145762 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141720 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 30 00:17:14.146276 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141722 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 30 00:17:14.146276 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141725 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 30 00:17:14.146276 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141728 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 30 00:17:14.146276 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141732 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 30 00:17:14.146276 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141735 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 30 00:17:14.146276 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141737 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 30 00:17:14.146276 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141740 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 30 00:17:14.146276 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141742 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 30 00:17:14.146276 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141745 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 30 00:17:14.146276 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141748 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 30 00:17:14.146276 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141750 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 30 00:17:14.146276 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141753 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 30 00:17:14.146276 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141755 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 30 00:17:14.146276 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141758 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 30 00:17:14.146276 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141761 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 30 00:17:14.146276 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141764 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 30 00:17:14.146276 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141767 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 30 00:17:14.146276 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141769 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 30 00:17:14.146276 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141772 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 30 00:17:14.146754 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.141777 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 30 00:17:14.146754 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141930 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 30 00:17:14.146754 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141937 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 30 00:17:14.146754 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141940 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 30 00:17:14.146754 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141943 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 30 00:17:14.146754 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141946 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 30 00:17:14.146754 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141949 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 30 00:17:14.146754 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141952 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 30 00:17:14.146754 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141956 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 30 00:17:14.146754 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141959 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 30 00:17:14.146754 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141963 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 30 00:17:14.146754 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141965 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 30 00:17:14.146754 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141968 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 30 00:17:14.146754 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141971 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 30 00:17:14.146754 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141973 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 30 00:17:14.147188 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141976 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 30 00:17:14.147188 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141978 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 30 00:17:14.147188 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141981 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 30 00:17:14.147188 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141984 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 30 00:17:14.147188 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141986 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 30 00:17:14.147188 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141988 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 30 00:17:14.147188 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141991 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 30 00:17:14.147188 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141993 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 30 00:17:14.147188 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141996 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 30 00:17:14.147188 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.141998 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 30 00:17:14.147188 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142001 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 30 00:17:14.147188 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142004 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 30 00:17:14.147188 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142007 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 30 00:17:14.147188 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142010 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 30 00:17:14.147188 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142013 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 30 00:17:14.147188 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142015 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 30 00:17:14.147188 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142018 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 30 00:17:14.147188 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142021 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 30 00:17:14.147188 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142023 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 30 00:17:14.147188 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142026 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 30 00:17:14.147685 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142029 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 30 00:17:14.147685 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142031 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 30 00:17:14.147685 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142034 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 30 00:17:14.147685 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142036 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 30 00:17:14.147685 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142042 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 30 00:17:14.147685 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142044 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 30 00:17:14.147685 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142048 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 30 00:17:14.147685 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142051 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 30 00:17:14.147685 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142054 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 30 00:17:14.147685 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142057 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 30 00:17:14.147685 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142060 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 30 00:17:14.147685 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142063 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 30 00:17:14.147685 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142065 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 30 00:17:14.147685 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142069 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 30 00:17:14.147685 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142071 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 30 00:17:14.147685 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142074 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 30 00:17:14.147685 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142076 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 30 00:17:14.147685 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142079 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 30 00:17:14.147685 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142082 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 30 00:17:14.148149 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142084 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 30 00:17:14.148149 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142086 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 30 00:17:14.148149 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142089 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 30 00:17:14.148149 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142091 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 30 00:17:14.148149 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142094 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 30 00:17:14.148149 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142097 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 30 00:17:14.148149 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142099 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 30 00:17:14.148149 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142102 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 30 00:17:14.148149 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142105 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 30 00:17:14.148149 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142108 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 30 00:17:14.148149 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142110 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 30 00:17:14.148149 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142113 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 30 00:17:14.148149 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142115 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 30 00:17:14.148149 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142118 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 30 00:17:14.148149 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142121 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 30 00:17:14.148149 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142123 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 30 00:17:14.148149 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142126 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 30 00:17:14.148149 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142128 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 30 00:17:14.148149 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142131 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 30 00:17:14.148149 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142134 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 30 00:17:14.148645 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142136 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 30 00:17:14.148645 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142138 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 30 00:17:14.148645 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142141 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 30 00:17:14.148645 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142143 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 30 00:17:14.148645 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142146 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 30 00:17:14.148645 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142149 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 30 00:17:14.148645 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142151 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 30 00:17:14.148645 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142154 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 30 00:17:14.148645 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142156 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 30 00:17:14.148645 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142159 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 30 00:17:14.148645 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142161 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 30 00:17:14.148645 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142164 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 30 00:17:14.148645 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142166 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 30 00:17:14.148645 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142168 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 30 00:17:14.148645 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142171 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 30 00:17:14.148645 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142173 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 30 00:17:14.148645 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142175 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 30 00:17:14.148645 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142178 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 30 00:17:14.148645 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142181 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 30 00:17:14.148645 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142183 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 30 00:17:14.149154 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142186 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 30 00:17:14.149154 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142189 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 30 00:17:14.149154 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142191 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 30 00:17:14.149154 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142195 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 30 00:17:14.149154 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142199 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 30 00:17:14.149154 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:14.142201 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 30 00:17:14.149154 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.142206 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 30 00:17:14.149154 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.142327 2563 server.go:962] "Client rotation is on, will bootstrap in background" Jul 30 00:17:14.149154 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.144452 2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 30 00:17:14.149154 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.145678 2563 server.go:1019] "Starting client certificate rotation" Jul 30 00:17:14.149154 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.145777 2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 30 00:17:14.149154 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.145838 2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 30 00:17:14.149469 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.149282 2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 30 00:17:14.150414 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.150387 2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 30 00:17:14.162421 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.162398 2563 log.go:25] "Validated CRI v1 runtime API" Jul 30 00:17:14.164581 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.164562 2563 log.go:25] "Validated CRI v1 image API" Jul 30 00:17:14.165420 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.165404 2563 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 30 00:17:14.168403 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.168383 2563 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 c3714d13-c106-40c4-b1ec-48233ecd0fd7:/dev/nvme0n1p3 c9fe27fd-5bef-4cae-9c57-3fd2325efdf3:/dev/nvme0n1p4] Jul 30 00:17:14.168465 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.168401 2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 30 00:17:14.174140 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.173991 2563 manager.go:216] Machine: {Timestamp:2026-07-30 00:17:14.172409554 +0000 UTC m=+0.354039027 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099737 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e8e55f9727322b8cf2f3d2d08b699 SystemUUID:ec2e8e55-f972-7322-b8cf-2f3d2d08b699 BootID:2136457e-9b89-4c96-8e43-d2f86f47f205 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:fb:be:a8:34:43 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:fb:be:a8:34:43 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:b6:28:40:0c:fd:f6 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 30 00:17:14.174240 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.174143 2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 30 00:17:14.174274 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.174246 2563 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 30 00:17:14.176088 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.176071 2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 30 00:17:14.177730 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.177707 2563 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 30 00:17:14.177889 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.177735 2563 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-10.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 30 00:17:14.179057 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.179047 2563 topology_manager.go:138] "Creating topology manager with none policy" Jul 30 00:17:14.179097 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.179061 2563 container_manager_linux.go:309] "Creating device plugin manager" Jul 30 00:17:14.179097 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.179069 2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 30 00:17:14.179839 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.179828 2563 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 30 00:17:14.180683 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.180673 2563 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 30 00:17:14.182474 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.182463 2563 state_mem.go:36] "Initialized new in-memory state store" Jul 30 00:17:14.182596 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.182586 2563 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 30 00:17:14.182631 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.182615 2563 kubelet.go:486] "Attempting to sync node with API server" Jul 30 00:17:14.182664 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.182632 2563 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 30 00:17:14.182664 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.182651 2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 30 00:17:14.182664 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.182661 2563 kubelet.go:398] "Adding apiserver pod source" Jul 30 00:17:14.182740 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.182672 2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 30 00:17:14.183059 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.183046 2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 30 00:17:14.183100 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.183068 2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 30 00:17:14.185854 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.185835 2563 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 30 00:17:14.187845 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.187826 2563 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 30 00:17:14.187944 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.187884 2563 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 30 00:17:14.189315 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.189299 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 30 00:17:14.189394 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.189318 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 30 00:17:14.189394 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.189327 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 30 00:17:14.189394 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.189343 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 30 00:17:14.189394 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.189355 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 30 00:17:14.189394 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.189364 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 30 00:17:14.189394 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.189375 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 30 00:17:14.189394 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.189384 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 30 00:17:14.189394 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.189393 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 30 00:17:14.189721 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.189402 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 30 00:17:14.189721 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.189420 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 30 00:17:14.189721 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.189429 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 30 00:17:14.189721 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.189460 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 30 00:17:14.189721 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.189468 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 30 00:17:14.192580 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.192558 2563 server.go:1268] "Started kubelet" Jul 30 00:17:14.192691 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.192652 2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 30 00:17:14.192755 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.192649 2563 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 30 00:17:14.192755 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.192726 2563 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 30 00:17:14.197300 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.197281 2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-10.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 30 00:17:14.197382 ip-10-0-133-10 systemd[1]: Started Kubernetes Kubelet. Jul 30 00:17:14.197604 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.197531 2563 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 30 00:17:14.197910 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:14.197881 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 30 00:17:14.198485 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:14.197850 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-10.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 30 00:17:14.201347 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:14.201128 2563 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 30 00:17:14.201457 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.201442 2563 server.go:310] "Adding debug handlers to kubelet server" Jul 30 00:17:14.204432 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:14.203584 2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-10.ec2.internal.18c6e921eb75a1b8 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-10.ec2.internal,UID:ip-10-0-133-10.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-10.ec2.internal,},FirstTimestamp:2026-07-30 00:17:14.1925278 +0000 UTC m=+0.374157327,LastTimestamp:2026-07-30 00:17:14.1925278 +0000 UTC m=+0.374157327,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-10.ec2.internal,}" Jul 30 00:17:14.208904 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.208885 2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 30 00:17:14.208996 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.208905 2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 30 00:17:14.209063 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:14.209007 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-10.ec2.internal\" not found" Jul 30 00:17:14.209599 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.209576 2563 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 30 00:17:14.210816 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.210792 2563 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 30 00:17:14.210816 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.210817 2563 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 30 00:17:14.210963 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.210851 2563 factory.go:55] Registering systemd factory Jul 30 00:17:14.210963 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.210896 2563 factory.go:223] Registration of the systemd container factory successfully Jul 30 00:17:14.210963 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.210933 2563 reconstruct.go:97] "Volume reconstruction finished" Jul 30 00:17:14.210963 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.210944 2563 reconciler.go:29] "Reconciler: start to sync state" Jul 30 00:17:14.211284 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:14.211253 2563 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-10.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 30 00:17:14.211421 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.211407 2563 factory.go:153] Registering CRI-O factory Jul 30 00:17:14.211496 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.211425 2563 factory.go:223] Registration of the crio container factory successfully Jul 30 00:17:14.211496 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:14.211452 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 30 00:17:14.211496 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.211471 2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 30 00:17:14.211496 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.211493 2563 factory.go:103] Registering Raw factory Jul 30 00:17:14.211673 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.211507 2563 manager.go:1196] Started watching for new ooms in manager Jul 30 00:17:14.211899 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.211889 2563 manager.go:349] Starting recovery of all containers Jul 30 00:17:14.218483 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.218439 2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 30 00:17:14.221288 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.221272 2563 manager.go:354] Recovery completed Jul 30 00:17:14.224716 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.224700 2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jptg9" Jul 30 00:17:14.226367 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.226354 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 00:17:14.228175 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.228157 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientMemory" Jul 30 00:17:14.228240 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.228191 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 00:17:14.228240 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.228204 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientPID" Jul 30 00:17:14.228747 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.228729 2563 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 30 00:17:14.228747 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.228740 2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 30 00:17:14.228855 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.228762 2563 state_mem.go:36] "Initialized new in-memory state store" Jul 30 00:17:14.228855 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.228781 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 00:17:14.229525 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.229509 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientMemory" Jul 30 00:17:14.229570 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.229541 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 00:17:14.229570 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.229551 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientPID" Jul 30 00:17:14.229975 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.229955 2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jptg9" Jul 30 00:17:14.230142 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:14.230065 2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-10.ec2.internal.18c6e921ed958f6a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-10.ec2.internal,UID:ip-10-0-133-10.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-10.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-10.ec2.internal,},FirstTimestamp:2026-07-30 00:17:14.228174698 +0000 UTC m=+0.409804178,LastTimestamp:2026-07-30 00:17:14.228174698 +0000 UTC m=+0.409804178,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-10.ec2.internal,}" Jul 30 00:17:14.231722 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.231709 2563 policy_none.go:49] "None policy: Start" Jul 30 00:17:14.231780 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.231726 2563 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 30 00:17:14.231780 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.231738 2563 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 30 00:17:14.232792 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.232780 2563 policy_none.go:47] "Start" Jul 30 00:17:14.258404 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.258390 2563 manager.go:337] "Starting Device Plugin manager" Jul 30 00:17:14.258500 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:14.258419 2563 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 30 00:17:14.258500 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.258429 2563 server.go:87] "Starting device plugin registration server" Jul 30 00:17:14.258680 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.258670 2563 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 30 00:17:14.258724 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.258682 2563 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 30 00:17:14.258823 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.258808 2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 30 00:17:14.258917 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.258896 2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 30 00:17:14.258917 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.258908 2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 30 00:17:14.259755 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:14.259739 2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 30 00:17:14.259832 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:14.259788 2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-10.ec2.internal\" not found" Jul 30 00:17:14.273089 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.273069 2563 container_manager_linux.go:676] "Failed to ensure state" containerName="/system.slice" err="failed to move PID 2599 into the system container \"/system.slice\": " Jul 30 00:17:14.345840 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.345815 2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 30 00:17:14.346911 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.345876 2563 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 30 00:17:14.346911 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.345904 2563 kubelet.go:2448] "Starting kubelet main sync loop" Jul 30 00:17:14.346911 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:14.345948 2563 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 30 00:17:14.348125 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.348102 2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 30 00:17:14.359058 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.359044 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 00:17:14.360414 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.360400 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientMemory" Jul 30 00:17:14.360480 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.360428 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 00:17:14.360480 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.360437 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientPID" Jul 30 00:17:14.360480 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.360462 2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-10.ec2.internal" Jul 30 00:17:14.376437 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.376390 2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-10.ec2.internal" Jul 30 00:17:14.376437 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:14.376413 2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-10.ec2.internal\": node \"ip-10-0-133-10.ec2.internal\" not found" Jul 30 00:17:14.447001 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.446832 2563 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-10.ec2.internal"] Jul 30 00:17:14.447155 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.447038 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 00:17:14.447975 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.447960 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientMemory" Jul 30 00:17:14.448070 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.447992 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 00:17:14.448070 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.448005 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientPID" Jul 30 00:17:14.449398 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:14.449382 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-10.ec2.internal\" not found" Jul 30 00:17:14.449633 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.449623 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 00:17:14.449759 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.449744 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal" Jul 30 00:17:14.449809 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.449774 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 00:17:14.450322 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.450309 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientMemory" Jul 30 00:17:14.450411 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.450324 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientMemory" Jul 30 00:17:14.450411 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.450339 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 00:17:14.450411 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.450347 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 00:17:14.450411 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.450354 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientPID" Jul 30 00:17:14.450411 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.450357 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientPID" Jul 30 00:17:14.451698 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.451683 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-10.ec2.internal" Jul 30 00:17:14.451747 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.451715 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 00:17:14.452325 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.452306 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientMemory" Jul 30 00:17:14.452427 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.452338 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 00:17:14.452427 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.452352 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientPID" Jul 30 00:17:14.476506 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:14.476474 2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-10.ec2.internal\" not found" node="ip-10-0-133-10.ec2.internal" Jul 30 00:17:14.480903 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:14.480887 2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-10.ec2.internal\" not found" node="ip-10-0-133-10.ec2.internal" Jul 30 00:17:14.512784 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.512758 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2eeefd595358be06c0e6dd4416315e17-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal\" (UID: \"2eeefd595358be06c0e6dd4416315e17\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal" Jul 30 00:17:14.512784 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.512783 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2eeefd595358be06c0e6dd4416315e17-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal\" (UID: \"2eeefd595358be06c0e6dd4416315e17\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal" Jul 30 00:17:14.512934 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.512800 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d23058fd1d3df9b79e28eb07292f4443-config\") pod \"kube-apiserver-proxy-ip-10-0-133-10.ec2.internal\" (UID: \"d23058fd1d3df9b79e28eb07292f4443\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-10.ec2.internal" Jul 30 00:17:14.550251 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:14.550221 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-10.ec2.internal\" not found" Jul 30 00:17:14.613685 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.613645 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2eeefd595358be06c0e6dd4416315e17-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal\" (UID: \"2eeefd595358be06c0e6dd4416315e17\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal" Jul 30 00:17:14.613685 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.613687 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2eeefd595358be06c0e6dd4416315e17-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal\" (UID: \"2eeefd595358be06c0e6dd4416315e17\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal" Jul 30 00:17:14.613856 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.613702 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2eeefd595358be06c0e6dd4416315e17-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal\" (UID: \"2eeefd595358be06c0e6dd4416315e17\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal" Jul 30 00:17:14.613856 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.613724 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d23058fd1d3df9b79e28eb07292f4443-config\") pod \"kube-apiserver-proxy-ip-10-0-133-10.ec2.internal\" (UID: \"d23058fd1d3df9b79e28eb07292f4443\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-10.ec2.internal" Jul 30 00:17:14.613856 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.613744 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2eeefd595358be06c0e6dd4416315e17-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal\" (UID: \"2eeefd595358be06c0e6dd4416315e17\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal" Jul 30 00:17:14.613856 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.613759 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d23058fd1d3df9b79e28eb07292f4443-config\") pod \"kube-apiserver-proxy-ip-10-0-133-10.ec2.internal\" (UID: \"d23058fd1d3df9b79e28eb07292f4443\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-10.ec2.internal" Jul 30 00:17:14.650915 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:14.650835 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-10.ec2.internal\" not found" Jul 30 00:17:14.751595 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:14.751554 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-10.ec2.internal\" not found" Jul 30 00:17:14.779758 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.779732 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal" Jul 30 00:17:14.784637 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:14.784603 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-10.ec2.internal" Jul 30 00:17:14.851955 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:14.851919 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-10.ec2.internal\" not found" Jul 30 00:17:14.952521 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:14.952444 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-10.ec2.internal\" not found" Jul 30 00:17:15.052929 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:15.052901 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-10.ec2.internal\" not found" Jul 30 00:17:17.268477 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:17.268448 2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 30 00:17:17.268905 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:17.268587 2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 30 00:17:17.276760 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:17.276724 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-10.ec2.internal\" not found" Jul 30 00:17:17.357769 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:17.357662 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-29 00:12:16 +0000 UTC" deadline="2028-01-04 20:24:17.889494704 +0000 UTC" Jul 30 00:17:17.357769 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:17.357693 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12572h7m0.531804911s" Jul 30 00:17:17.377830 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:17.377809 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-10.ec2.internal\" not found" Jul 30 00:17:17.463027 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:17.462998 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd23058fd1d3df9b79e28eb07292f4443.slice/crio-a4e50c25d0be4bebff04df9ffa9806b09cecbb831d4f5b6bfc1250cf444ba4a2 WatchSource:0}: Error finding container a4e50c25d0be4bebff04df9ffa9806b09cecbb831d4f5b6bfc1250cf444ba4a2: Status 404 returned error can't find the container with id a4e50c25d0be4bebff04df9ffa9806b09cecbb831d4f5b6bfc1250cf444ba4a2 Jul 30 00:17:17.463457 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:17.463439 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2eeefd595358be06c0e6dd4416315e17.slice/crio-fa886ca91f2b79df32a80e06d72a5bbf97fcc4618143811a548df1d455f71710 WatchSource:0}: Error finding container fa886ca91f2b79df32a80e06d72a5bbf97fcc4618143811a548df1d455f71710: Status 404 returned error can't find the container with id fa886ca91f2b79df32a80e06d72a5bbf97fcc4618143811a548df1d455f71710 Jul 30 00:17:17.470534 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:17.470516 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 30 00:17:17.471508 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:17.471462 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal" event={"ID":"2eeefd595358be06c0e6dd4416315e17","Type":"ContainerStarted","Data":"fa886ca91f2b79df32a80e06d72a5bbf97fcc4618143811a548df1d455f71710"} Jul 30 00:17:17.472438 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:17.472418 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-10.ec2.internal" event={"ID":"d23058fd1d3df9b79e28eb07292f4443","Type":"ContainerStarted","Data":"a4e50c25d0be4bebff04df9ffa9806b09cecbb831d4f5b6bfc1250cf444ba4a2"} Jul 30 00:17:17.478932 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:17.478897 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-10.ec2.internal\" not found" Jul 30 00:17:17.532016 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:17.531983 2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 30 00:17:17.579141 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:17.579110 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-10.ec2.internal\" not found" Jul 30 00:17:17.626632 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:17.626583 2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 30 00:17:17.657942 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:17.657922 2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 30 00:17:17.732851 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:17.732818 2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal" Jul 30 00:17:17.746318 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:17.746146 2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 30 00:17:17.749658 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:17.749637 2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-10.ec2.internal" Jul 30 00:17:17.758898 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:17.758877 2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 30 00:17:18.231561 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.231530 2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 30 00:17:18.306560 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.306528 2563 apiserver.go:52] "Watching apiserver" Jul 30 00:17:18.313947 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.313918 2563 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 30 00:17:18.314992 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.314968 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-gp6l9","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal","openshift-multus/multus-additional-cni-plugins-zx7wk","openshift-multus/multus-hglh4","openshift-multus/network-metrics-daemon-zxg4x","kube-system/kube-apiserver-proxy-ip-10-0-133-10.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvgfr","openshift-cluster-node-tuning-operator/tuned-6wfs6","openshift-dns/node-resolver-zlqlw","openshift-network-diagnostics/network-check-target-z7xxn","openshift-network-operator/iptables-alerter-wgvqx","openshift-ovn-kubernetes/ovnkube-node-bpk2r","kube-system/global-pull-secret-syncer-j26vw","kube-system/konnectivity-agent-9m2mr"] Jul 30 00:17:18.316811 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.316785 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvgfr" Jul 30 00:17:18.318379 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.318085 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j26vw" Jul 30 00:17:18.318379 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:18.318184 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j26vw" podUID="f099a95a-e24f-4e05-bba1-bccd334f000c" Jul 30 00:17:18.319221 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.319195 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 30 00:17:18.319442 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.319418 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-mnfll\"" Jul 30 00:17:18.319694 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.319674 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 30 00:17:18.319804 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.319706 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 30 00:17:18.320049 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.320003 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zx7wk" Jul 30 00:17:18.321697 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.321680 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hglh4" Jul 30 00:17:18.322426 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.322211 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 30 00:17:18.322426 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.322369 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 30 00:17:18.322693 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.322673 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 30 00:17:18.322784 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.322700 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-rz2tc\"" Jul 30 00:17:18.322898 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.322879 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 30 00:17:18.322956 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.322917 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 30 00:17:18.323091 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.323076 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zxg4x" Jul 30 00:17:18.323175 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:18.323138 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zxg4x" podUID="8567e26f-2898-4d1b-8000-b8bd3326c9ac" Jul 30 00:17:18.323781 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.323762 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-zf875\"" Jul 30 00:17:18.324045 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.324027 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 30 00:17:18.324788 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.324770 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9m2mr" Jul 30 00:17:18.326049 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.326027 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zlqlw" Jul 30 00:17:18.327000 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.326960 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-4t5zd\"" Jul 30 00:17:18.327091 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.326969 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 30 00:17:18.327147 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.327104 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 30 00:17:18.327607 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.327588 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gp6l9" Jul 30 00:17:18.328153 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.328133 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 30 00:17:18.328241 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.328169 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 30 00:17:18.328307 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.328134 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-4jzh5\"" Jul 30 00:17:18.329297 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.329276 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6wfs6" Jul 30 00:17:18.330224 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.329891 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 30 00:17:18.330224 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.330083 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 30 00:17:18.330224 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.330116 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-kgd84\"" Jul 30 00:17:18.330224 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.330129 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 30 00:17:18.330641 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.330618 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7xxn" Jul 30 00:17:18.330918 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:18.330746 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7xxn" podUID="3101459e-308a-4623-884f-8f54273f1bfd" Jul 30 00:17:18.332061 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.332041 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wgvqx" Jul 30 00:17:18.332466 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.332446 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-stk5q\"" Jul 30 00:17:18.334354 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.333752 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 30 00:17:18.334354 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.333898 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 30 00:17:18.334354 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.334340 2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 30 00:17:18.334668 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.334650 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 30 00:17:18.335006 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.334986 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-2zz5r\"" Jul 30 00:17:18.335227 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.335209 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.337077 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.337009 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 30 00:17:18.337285 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.337263 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 30 00:17:18.339526 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.339507 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 30 00:17:18.340380 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.340356 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 30 00:17:18.340472 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.340455 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-bdwfx\"" Jul 30 00:17:18.340572 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.340555 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 30 00:17:18.340661 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.340628 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 30 00:17:18.341255 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.341222 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 30 00:17:18.341360 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.341343 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 30 00:17:18.343591 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.343522 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f95c8416-e10e-4357-96a7-4b48e1c3be4f-multus-cni-dir\") pod \"multus-hglh4\" (UID: \"f95c8416-e10e-4357-96a7-4b48e1c3be4f\") " pod="openshift-multus/multus-hglh4" Jul 30 00:17:18.343591 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.343546 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b235c4f3-5e9b-4866-99c2-41ec0a64f806-lib-modules\") pod \"tuned-6wfs6\" (UID: \"b235c4f3-5e9b-4866-99c2-41ec0a64f806\") " pod="openshift-cluster-node-tuning-operator/tuned-6wfs6" Jul 30 00:17:18.343591 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.343571 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3806f8c4-b735-4fd6-a240-882dd0c2c70a-iptables-alerter-script\") pod \"iptables-alerter-wgvqx\" (UID: \"3806f8c4-b735-4fd6-a240-882dd0c2c70a\") " pod="openshift-network-operator/iptables-alerter-wgvqx" Jul 30 00:17:18.343775 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.343594 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c882608c-cf7a-4e13-a031-193b7c4ac1dc-socket-dir\") pod \"aws-ebs-csi-driver-node-fvgfr\" (UID: \"c882608c-cf7a-4e13-a031-193b7c4ac1dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvgfr" Jul 30 00:17:18.343775 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.343615 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f95c8416-e10e-4357-96a7-4b48e1c3be4f-host-run-k8s-cni-cncf-io\") pod \"multus-hglh4\" (UID: \"f95c8416-e10e-4357-96a7-4b48e1c3be4f\") " pod="openshift-multus/multus-hglh4" Jul 30 00:17:18.343775 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.343662 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f95c8416-e10e-4357-96a7-4b48e1c3be4f-host-var-lib-cni-bin\") pod \"multus-hglh4\" (UID: \"f95c8416-e10e-4357-96a7-4b48e1c3be4f\") " pod="openshift-multus/multus-hglh4" Jul 30 00:17:18.343775 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.343705 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/52037ff9-396d-4e0d-958b-ff96d3df4855-node-log\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.343775 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.343750 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/52037ff9-396d-4e0d-958b-ff96d3df4855-host-cni-bin\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.343940 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.343774 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/756ee6d1-bf62-49db-954a-ed5d66384c54-host\") pod \"node-ca-gp6l9\" (UID: \"756ee6d1-bf62-49db-954a-ed5d66384c54\") " pod="openshift-image-registry/node-ca-gp6l9" Jul 30 00:17:18.343940 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.343799 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/52037ff9-396d-4e0d-958b-ff96d3df4855-host-kubelet\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.343940 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.343821 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/52037ff9-396d-4e0d-958b-ff96d3df4855-host-run-netns\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.343940 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.343846 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f5cd0fd1-6297-45d6-9823-cd424bce4185-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zx7wk\" (UID: \"f5cd0fd1-6297-45d6-9823-cd424bce4185\") " pod="openshift-multus/multus-additional-cni-plugins-zx7wk" Jul 30 00:17:18.343940 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.343869 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b235c4f3-5e9b-4866-99c2-41ec0a64f806-sys\") pod \"tuned-6wfs6\" (UID: \"b235c4f3-5e9b-4866-99c2-41ec0a64f806\") " pod="openshift-cluster-node-tuning-operator/tuned-6wfs6" Jul 30 00:17:18.343940 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.343891 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b235c4f3-5e9b-4866-99c2-41ec0a64f806-etc-tuned\") pod \"tuned-6wfs6\" (UID: \"b235c4f3-5e9b-4866-99c2-41ec0a64f806\") " pod="openshift-cluster-node-tuning-operator/tuned-6wfs6" Jul 30 00:17:18.344120 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.343937 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2g6bd\" (UniqueName: \"kubernetes.io/projected/c882608c-cf7a-4e13-a031-193b7c4ac1dc-kube-api-access-2g6bd\") pod \"aws-ebs-csi-driver-node-fvgfr\" (UID: \"c882608c-cf7a-4e13-a031-193b7c4ac1dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvgfr" Jul 30 00:17:18.344120 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.343959 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/52037ff9-396d-4e0d-958b-ff96d3df4855-host-run-ovn-kubernetes\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.344120 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.343977 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/52037ff9-396d-4e0d-958b-ff96d3df4855-ovnkube-config\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.344120 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.343994 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/52037ff9-396d-4e0d-958b-ff96d3df4855-ovn-node-metrics-cert\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.344120 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.344017 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b235c4f3-5e9b-4866-99c2-41ec0a64f806-etc-kubernetes\") pod \"tuned-6wfs6\" (UID: \"b235c4f3-5e9b-4866-99c2-41ec0a64f806\") " pod="openshift-cluster-node-tuning-operator/tuned-6wfs6" Jul 30 00:17:18.344120 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.344031 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b235c4f3-5e9b-4866-99c2-41ec0a64f806-etc-sysctl-conf\") pod \"tuned-6wfs6\" (UID: \"b235c4f3-5e9b-4866-99c2-41ec0a64f806\") " pod="openshift-cluster-node-tuning-operator/tuned-6wfs6" Jul 30 00:17:18.344120 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.344045 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b235c4f3-5e9b-4866-99c2-41ec0a64f806-run\") pod \"tuned-6wfs6\" (UID: \"b235c4f3-5e9b-4866-99c2-41ec0a64f806\") " pod="openshift-cluster-node-tuning-operator/tuned-6wfs6" Jul 30 00:17:18.344120 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.344061 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c882608c-cf7a-4e13-a031-193b7c4ac1dc-device-dir\") pod \"aws-ebs-csi-driver-node-fvgfr\" (UID: \"c882608c-cf7a-4e13-a031-193b7c4ac1dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvgfr" Jul 30 00:17:18.344120 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.344089 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f099a95a-e24f-4e05-bba1-bccd334f000c-dbus\") pod \"global-pull-secret-syncer-j26vw\" (UID: \"f099a95a-e24f-4e05-bba1-bccd334f000c\") " pod="kube-system/global-pull-secret-syncer-j26vw" Jul 30 00:17:18.344534 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.344124 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f95c8416-e10e-4357-96a7-4b48e1c3be4f-os-release\") pod \"multus-hglh4\" (UID: \"f95c8416-e10e-4357-96a7-4b48e1c3be4f\") " pod="openshift-multus/multus-hglh4" Jul 30 00:17:18.344534 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.344168 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2nxf\" (UniqueName: \"kubernetes.io/projected/8567e26f-2898-4d1b-8000-b8bd3326c9ac-kube-api-access-x2nxf\") pod \"network-metrics-daemon-zxg4x\" (UID: \"8567e26f-2898-4d1b-8000-b8bd3326c9ac\") " pod="openshift-multus/network-metrics-daemon-zxg4x" Jul 30 00:17:18.344534 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.344202 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/52037ff9-396d-4e0d-958b-ff96d3df4855-host-cni-netd\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.344534 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.344249 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ad9e265e-16ad-428a-a69a-f9a7948e5482-konnectivity-ca\") pod \"konnectivity-agent-9m2mr\" (UID: \"ad9e265e-16ad-428a-a69a-f9a7948e5482\") " pod="kube-system/konnectivity-agent-9m2mr" Jul 30 00:17:18.344534 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.344284 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/add0c4dc-1cb0-4fa3-903a-6ce3b4a8b60a-hosts-file\") pod \"node-resolver-zlqlw\" (UID: \"add0c4dc-1cb0-4fa3-903a-6ce3b4a8b60a\") " pod="openshift-dns/node-resolver-zlqlw" Jul 30 00:17:18.344534 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.344311 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4ch2\" (UniqueName: \"kubernetes.io/projected/f5cd0fd1-6297-45d6-9823-cd424bce4185-kube-api-access-v4ch2\") pod \"multus-additional-cni-plugins-zx7wk\" (UID: \"f5cd0fd1-6297-45d6-9823-cd424bce4185\") " pod="openshift-multus/multus-additional-cni-plugins-zx7wk" Jul 30 00:17:18.344534 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.344338 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f5cd0fd1-6297-45d6-9823-cd424bce4185-cni-binary-copy\") pod \"multus-additional-cni-plugins-zx7wk\" (UID: \"f5cd0fd1-6297-45d6-9823-cd424bce4185\") " pod="openshift-multus/multus-additional-cni-plugins-zx7wk" Jul 30 00:17:18.344534 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.344367 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f95c8416-e10e-4357-96a7-4b48e1c3be4f-system-cni-dir\") pod \"multus-hglh4\" (UID: \"f95c8416-e10e-4357-96a7-4b48e1c3be4f\") " pod="openshift-multus/multus-hglh4" Jul 30 00:17:18.344534 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.344388 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f95c8416-e10e-4357-96a7-4b48e1c3be4f-cnibin\") pod \"multus-hglh4\" (UID: \"f95c8416-e10e-4357-96a7-4b48e1c3be4f\") " pod="openshift-multus/multus-hglh4" Jul 30 00:17:18.344534 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.344411 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f95c8416-e10e-4357-96a7-4b48e1c3be4f-cni-binary-copy\") pod \"multus-hglh4\" (UID: \"f95c8416-e10e-4357-96a7-4b48e1c3be4f\") " pod="openshift-multus/multus-hglh4" Jul 30 00:17:18.344534 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.344441 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/756ee6d1-bf62-49db-954a-ed5d66384c54-serviceca\") pod \"node-ca-gp6l9\" (UID: \"756ee6d1-bf62-49db-954a-ed5d66384c54\") " pod="openshift-image-registry/node-ca-gp6l9" Jul 30 00:17:18.344534 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.344479 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52037ff9-396d-4e0d-958b-ff96d3df4855-var-lib-openvswitch\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.344534 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.344512 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96x4r\" (UniqueName: \"kubernetes.io/projected/52037ff9-396d-4e0d-958b-ff96d3df4855-kube-api-access-96x4r\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.344534 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.344535 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f5cd0fd1-6297-45d6-9823-cd424bce4185-system-cni-dir\") pod \"multus-additional-cni-plugins-zx7wk\" (UID: \"f5cd0fd1-6297-45d6-9823-cd424bce4185\") " pod="openshift-multus/multus-additional-cni-plugins-zx7wk" Jul 30 00:17:18.345163 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.344551 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f95c8416-e10e-4357-96a7-4b48e1c3be4f-host-var-lib-cni-multus\") pod \"multus-hglh4\" (UID: \"f95c8416-e10e-4357-96a7-4b48e1c3be4f\") " pod="openshift-multus/multus-hglh4" Jul 30 00:17:18.345163 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.344583 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f95c8416-e10e-4357-96a7-4b48e1c3be4f-hostroot\") pod \"multus-hglh4\" (UID: \"f95c8416-e10e-4357-96a7-4b48e1c3be4f\") " pod="openshift-multus/multus-hglh4" Jul 30 00:17:18.345163 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.344603 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/52037ff9-396d-4e0d-958b-ff96d3df4855-host-slash\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.345163 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.344620 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52037ff9-396d-4e0d-958b-ff96d3df4855-etc-openvswitch\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.345163 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.344652 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b235c4f3-5e9b-4866-99c2-41ec0a64f806-etc-sysconfig\") pod \"tuned-6wfs6\" (UID: \"b235c4f3-5e9b-4866-99c2-41ec0a64f806\") " pod="openshift-cluster-node-tuning-operator/tuned-6wfs6" Jul 30 00:17:18.345163 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.344706 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f95c8416-e10e-4357-96a7-4b48e1c3be4f-multus-conf-dir\") pod \"multus-hglh4\" (UID: \"f95c8416-e10e-4357-96a7-4b48e1c3be4f\") " pod="openshift-multus/multus-hglh4" Jul 30 00:17:18.345163 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.344747 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/52037ff9-396d-4e0d-958b-ff96d3df4855-run-systemd\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.345163 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.344769 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f5cd0fd1-6297-45d6-9823-cd424bce4185-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zx7wk\" (UID: \"f5cd0fd1-6297-45d6-9823-cd424bce4185\") " pod="openshift-multus/multus-additional-cni-plugins-zx7wk" Jul 30 00:17:18.345163 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.344783 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b235c4f3-5e9b-4866-99c2-41ec0a64f806-host\") pod \"tuned-6wfs6\" (UID: \"b235c4f3-5e9b-4866-99c2-41ec0a64f806\") " pod="openshift-cluster-node-tuning-operator/tuned-6wfs6" Jul 30 00:17:18.345163 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.344796 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f5cd0fd1-6297-45d6-9823-cd424bce4185-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zx7wk\" (UID: \"f5cd0fd1-6297-45d6-9823-cd424bce4185\") " pod="openshift-multus/multus-additional-cni-plugins-zx7wk" Jul 30 00:17:18.345163 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.344811 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knxqr\" (UniqueName: \"kubernetes.io/projected/add0c4dc-1cb0-4fa3-903a-6ce3b4a8b60a-kube-api-access-knxqr\") pod \"node-resolver-zlqlw\" (UID: \"add0c4dc-1cb0-4fa3-903a-6ce3b4a8b60a\") " pod="openshift-dns/node-resolver-zlqlw" Jul 30 00:17:18.345163 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.344834 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtjq7\" (UniqueName: \"kubernetes.io/projected/3806f8c4-b735-4fd6-a240-882dd0c2c70a-kube-api-access-gtjq7\") pod \"iptables-alerter-wgvqx\" (UID: \"3806f8c4-b735-4fd6-a240-882dd0c2c70a\") " pod="openshift-network-operator/iptables-alerter-wgvqx" Jul 30 00:17:18.345163 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.344855 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f099a95a-e24f-4e05-bba1-bccd334f000c-kubelet-config\") pod \"global-pull-secret-syncer-j26vw\" (UID: \"f099a95a-e24f-4e05-bba1-bccd334f000c\") " pod="kube-system/global-pull-secret-syncer-j26vw" Jul 30 00:17:18.345163 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.344871 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f95c8416-e10e-4357-96a7-4b48e1c3be4f-host-run-netns\") pod \"multus-hglh4\" (UID: \"f95c8416-e10e-4357-96a7-4b48e1c3be4f\") " pod="openshift-multus/multus-hglh4" Jul 30 00:17:18.345163 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.344899 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/52037ff9-396d-4e0d-958b-ff96d3df4855-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.345163 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.344924 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/52037ff9-396d-4e0d-958b-ff96d3df4855-ovnkube-script-lib\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.345945 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.344942 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f5cd0fd1-6297-45d6-9823-cd424bce4185-cnibin\") pod \"multus-additional-cni-plugins-zx7wk\" (UID: \"f5cd0fd1-6297-45d6-9823-cd424bce4185\") " pod="openshift-multus/multus-additional-cni-plugins-zx7wk" Jul 30 00:17:18.345945 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.344957 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b235c4f3-5e9b-4866-99c2-41ec0a64f806-etc-sysctl-d\") pod \"tuned-6wfs6\" (UID: \"b235c4f3-5e9b-4866-99c2-41ec0a64f806\") " pod="openshift-cluster-node-tuning-operator/tuned-6wfs6" Jul 30 00:17:18.345945 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.344977 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqcn9\" (UniqueName: \"kubernetes.io/projected/b235c4f3-5e9b-4866-99c2-41ec0a64f806-kube-api-access-sqcn9\") pod \"tuned-6wfs6\" (UID: \"b235c4f3-5e9b-4866-99c2-41ec0a64f806\") " pod="openshift-cluster-node-tuning-operator/tuned-6wfs6" Jul 30 00:17:18.345945 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.345016 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f95c8416-e10e-4357-96a7-4b48e1c3be4f-multus-daemon-config\") pod \"multus-hglh4\" (UID: \"f95c8416-e10e-4357-96a7-4b48e1c3be4f\") " pod="openshift-multus/multus-hglh4" Jul 30 00:17:18.345945 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.345041 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b235c4f3-5e9b-4866-99c2-41ec0a64f806-var-lib-kubelet\") pod \"tuned-6wfs6\" (UID: \"b235c4f3-5e9b-4866-99c2-41ec0a64f806\") " pod="openshift-cluster-node-tuning-operator/tuned-6wfs6" Jul 30 00:17:18.345945 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.345059 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f95c8416-e10e-4357-96a7-4b48e1c3be4f-host-var-lib-kubelet\") pod \"multus-hglh4\" (UID: \"f95c8416-e10e-4357-96a7-4b48e1c3be4f\") " pod="openshift-multus/multus-hglh4" Jul 30 00:17:18.345945 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.345075 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f95c8416-e10e-4357-96a7-4b48e1c3be4f-etc-kubernetes\") pod \"multus-hglh4\" (UID: \"f95c8416-e10e-4357-96a7-4b48e1c3be4f\") " pod="openshift-multus/multus-hglh4" Jul 30 00:17:18.345945 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.345094 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8567e26f-2898-4d1b-8000-b8bd3326c9ac-metrics-certs\") pod \"network-metrics-daemon-zxg4x\" (UID: \"8567e26f-2898-4d1b-8000-b8bd3326c9ac\") " pod="openshift-multus/network-metrics-daemon-zxg4x" Jul 30 00:17:18.345945 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.345144 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxjc4\" (UniqueName: \"kubernetes.io/projected/3101459e-308a-4623-884f-8f54273f1bfd-kube-api-access-qxjc4\") pod \"network-check-target-z7xxn\" (UID: \"3101459e-308a-4623-884f-8f54273f1bfd\") " pod="openshift-network-diagnostics/network-check-target-z7xxn" Jul 30 00:17:18.345945 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.345176 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52037ff9-396d-4e0d-958b-ff96d3df4855-run-openvswitch\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.345945 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.345223 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/add0c4dc-1cb0-4fa3-903a-6ce3b4a8b60a-tmp-dir\") pod \"node-resolver-zlqlw\" (UID: \"add0c4dc-1cb0-4fa3-903a-6ce3b4a8b60a\") " pod="openshift-dns/node-resolver-zlqlw" Jul 30 00:17:18.345945 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.345243 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b235c4f3-5e9b-4866-99c2-41ec0a64f806-etc-systemd\") pod \"tuned-6wfs6\" (UID: \"b235c4f3-5e9b-4866-99c2-41ec0a64f806\") " pod="openshift-cluster-node-tuning-operator/tuned-6wfs6" Jul 30 00:17:18.345945 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.345263 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3806f8c4-b735-4fd6-a240-882dd0c2c70a-host-slash\") pod \"iptables-alerter-wgvqx\" (UID: \"3806f8c4-b735-4fd6-a240-882dd0c2c70a\") " pod="openshift-network-operator/iptables-alerter-wgvqx" Jul 30 00:17:18.345945 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.345287 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f95c8416-e10e-4357-96a7-4b48e1c3be4f-multus-socket-dir-parent\") pod \"multus-hglh4\" (UID: \"f95c8416-e10e-4357-96a7-4b48e1c3be4f\") " pod="openshift-multus/multus-hglh4" Jul 30 00:17:18.345945 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.345310 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j68ms\" (UniqueName: \"kubernetes.io/projected/f95c8416-e10e-4357-96a7-4b48e1c3be4f-kube-api-access-j68ms\") pod \"multus-hglh4\" (UID: \"f95c8416-e10e-4357-96a7-4b48e1c3be4f\") " pod="openshift-multus/multus-hglh4" Jul 30 00:17:18.345945 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.345326 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/52037ff9-396d-4e0d-958b-ff96d3df4855-systemd-units\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.346525 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.345340 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f5cd0fd1-6297-45d6-9823-cd424bce4185-os-release\") pod \"multus-additional-cni-plugins-zx7wk\" (UID: \"f5cd0fd1-6297-45d6-9823-cd424bce4185\") " pod="openshift-multus/multus-additional-cni-plugins-zx7wk" Jul 30 00:17:18.346525 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.345358 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c882608c-cf7a-4e13-a031-193b7c4ac1dc-registration-dir\") pod \"aws-ebs-csi-driver-node-fvgfr\" (UID: \"c882608c-cf7a-4e13-a031-193b7c4ac1dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvgfr" Jul 30 00:17:18.346525 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.345380 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c882608c-cf7a-4e13-a031-193b7c4ac1dc-sys-fs\") pod \"aws-ebs-csi-driver-node-fvgfr\" (UID: \"c882608c-cf7a-4e13-a031-193b7c4ac1dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvgfr" Jul 30 00:17:18.346525 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.345407 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dddts\" (UniqueName: \"kubernetes.io/projected/756ee6d1-bf62-49db-954a-ed5d66384c54-kube-api-access-dddts\") pod \"node-ca-gp6l9\" (UID: \"756ee6d1-bf62-49db-954a-ed5d66384c54\") " pod="openshift-image-registry/node-ca-gp6l9" Jul 30 00:17:18.346525 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.345423 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/52037ff9-396d-4e0d-958b-ff96d3df4855-log-socket\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.346525 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.345437 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b235c4f3-5e9b-4866-99c2-41ec0a64f806-etc-modprobe-d\") pod \"tuned-6wfs6\" (UID: \"b235c4f3-5e9b-4866-99c2-41ec0a64f806\") " pod="openshift-cluster-node-tuning-operator/tuned-6wfs6" Jul 30 00:17:18.346525 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.345457 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b235c4f3-5e9b-4866-99c2-41ec0a64f806-tmp\") pod \"tuned-6wfs6\" (UID: \"b235c4f3-5e9b-4866-99c2-41ec0a64f806\") " pod="openshift-cluster-node-tuning-operator/tuned-6wfs6" Jul 30 00:17:18.346525 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.345492 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c882608c-cf7a-4e13-a031-193b7c4ac1dc-etc-selinux\") pod \"aws-ebs-csi-driver-node-fvgfr\" (UID: \"c882608c-cf7a-4e13-a031-193b7c4ac1dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvgfr" Jul 30 00:17:18.346525 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.345504 2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 30 00:17:18.346525 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.345521 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/52037ff9-396d-4e0d-958b-ff96d3df4855-env-overrides\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.346525 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.345544 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f099a95a-e24f-4e05-bba1-bccd334f000c-original-pull-secret\") pod \"global-pull-secret-syncer-j26vw\" (UID: \"f099a95a-e24f-4e05-bba1-bccd334f000c\") " pod="kube-system/global-pull-secret-syncer-j26vw" Jul 30 00:17:18.346525 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.345560 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f95c8416-e10e-4357-96a7-4b48e1c3be4f-host-run-multus-certs\") pod \"multus-hglh4\" (UID: \"f95c8416-e10e-4357-96a7-4b48e1c3be4f\") " pod="openshift-multus/multus-hglh4" Jul 30 00:17:18.346525 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.345574 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/52037ff9-396d-4e0d-958b-ff96d3df4855-run-ovn\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.346525 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.345591 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ad9e265e-16ad-428a-a69a-f9a7948e5482-agent-certs\") pod \"konnectivity-agent-9m2mr\" (UID: \"ad9e265e-16ad-428a-a69a-f9a7948e5482\") " pod="kube-system/konnectivity-agent-9m2mr" Jul 30 00:17:18.346525 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.345604 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c882608c-cf7a-4e13-a031-193b7c4ac1dc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fvgfr\" (UID: \"c882608c-cf7a-4e13-a031-193b7c4ac1dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvgfr" Jul 30 00:17:18.364873 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.364850 2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-zmlqt" Jul 30 00:17:18.373444 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.373422 2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-zmlqt" Jul 30 00:17:18.433313 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.433284 2563 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 30 00:17:18.446413 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.446387 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96x4r\" (UniqueName: \"kubernetes.io/projected/52037ff9-396d-4e0d-958b-ff96d3df4855-kube-api-access-96x4r\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.446572 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.446422 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f5cd0fd1-6297-45d6-9823-cd424bce4185-system-cni-dir\") pod \"multus-additional-cni-plugins-zx7wk\" (UID: \"f5cd0fd1-6297-45d6-9823-cd424bce4185\") " pod="openshift-multus/multus-additional-cni-plugins-zx7wk" Jul 30 00:17:18.446572 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.446446 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f95c8416-e10e-4357-96a7-4b48e1c3be4f-host-var-lib-cni-multus\") pod \"multus-hglh4\" (UID: \"f95c8416-e10e-4357-96a7-4b48e1c3be4f\") " pod="openshift-multus/multus-hglh4" Jul 30 00:17:18.446572 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.446466 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f95c8416-e10e-4357-96a7-4b48e1c3be4f-hostroot\") pod \"multus-hglh4\" (UID: \"f95c8416-e10e-4357-96a7-4b48e1c3be4f\") " pod="openshift-multus/multus-hglh4" Jul 30 00:17:18.446572 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.446486 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/52037ff9-396d-4e0d-958b-ff96d3df4855-host-slash\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.446572 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.446508 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52037ff9-396d-4e0d-958b-ff96d3df4855-etc-openvswitch\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.446572 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.446509 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f95c8416-e10e-4357-96a7-4b48e1c3be4f-host-var-lib-cni-multus\") pod \"multus-hglh4\" (UID: \"f95c8416-e10e-4357-96a7-4b48e1c3be4f\") " pod="openshift-multus/multus-hglh4" Jul 30 00:17:18.446572 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.446534 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b235c4f3-5e9b-4866-99c2-41ec0a64f806-etc-sysconfig\") pod \"tuned-6wfs6\" (UID: \"b235c4f3-5e9b-4866-99c2-41ec0a64f806\") " pod="openshift-cluster-node-tuning-operator/tuned-6wfs6" Jul 30 00:17:18.446572 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.446558 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f95c8416-e10e-4357-96a7-4b48e1c3be4f-multus-conf-dir\") pod \"multus-hglh4\" (UID: \"f95c8416-e10e-4357-96a7-4b48e1c3be4f\") " pod="openshift-multus/multus-hglh4" Jul 30 00:17:18.446572 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.446568 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f5cd0fd1-6297-45d6-9823-cd424bce4185-system-cni-dir\") pod \"multus-additional-cni-plugins-zx7wk\" (UID: \"f5cd0fd1-6297-45d6-9823-cd424bce4185\") " pod="openshift-multus/multus-additional-cni-plugins-zx7wk" Jul 30 00:17:18.447038 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.446594 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/52037ff9-396d-4e0d-958b-ff96d3df4855-host-slash\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.447038 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.446595 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/52037ff9-396d-4e0d-958b-ff96d3df4855-run-systemd\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.447038 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.446634 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/52037ff9-396d-4e0d-958b-ff96d3df4855-run-systemd\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.447038 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.446647 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f5cd0fd1-6297-45d6-9823-cd424bce4185-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zx7wk\" (UID: \"f5cd0fd1-6297-45d6-9823-cd424bce4185\") " pod="openshift-multus/multus-additional-cni-plugins-zx7wk" Jul 30 00:17:18.447038 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.446663 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b235c4f3-5e9b-4866-99c2-41ec0a64f806-etc-sysconfig\") pod \"tuned-6wfs6\" (UID: \"b235c4f3-5e9b-4866-99c2-41ec0a64f806\") " pod="openshift-cluster-node-tuning-operator/tuned-6wfs6" Jul 30 00:17:18.447038 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.446671 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b235c4f3-5e9b-4866-99c2-41ec0a64f806-host\") pod \"tuned-6wfs6\" (UID: \"b235c4f3-5e9b-4866-99c2-41ec0a64f806\") " pod="openshift-cluster-node-tuning-operator/tuned-6wfs6" Jul 30 00:17:18.447038 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.446678 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f95c8416-e10e-4357-96a7-4b48e1c3be4f-multus-conf-dir\") pod \"multus-hglh4\" (UID: \"f95c8416-e10e-4357-96a7-4b48e1c3be4f\") " pod="openshift-multus/multus-hglh4" Jul 30 00:17:18.447038 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.446697 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52037ff9-396d-4e0d-958b-ff96d3df4855-etc-openvswitch\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.447038 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.446776 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b235c4f3-5e9b-4866-99c2-41ec0a64f806-host\") pod \"tuned-6wfs6\" (UID: \"b235c4f3-5e9b-4866-99c2-41ec0a64f806\") " pod="openshift-cluster-node-tuning-operator/tuned-6wfs6" Jul 30 00:17:18.447038 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.446617 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f95c8416-e10e-4357-96a7-4b48e1c3be4f-hostroot\") pod \"multus-hglh4\" (UID: \"f95c8416-e10e-4357-96a7-4b48e1c3be4f\") " pod="openshift-multus/multus-hglh4" Jul 30 00:17:18.447038 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.446875 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f5cd0fd1-6297-45d6-9823-cd424bce4185-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zx7wk\" (UID: \"f5cd0fd1-6297-45d6-9823-cd424bce4185\") " pod="openshift-multus/multus-additional-cni-plugins-zx7wk" Jul 30 00:17:18.447385 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.446697 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f5cd0fd1-6297-45d6-9823-cd424bce4185-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zx7wk\" (UID: \"f5cd0fd1-6297-45d6-9823-cd424bce4185\") " pod="openshift-multus/multus-additional-cni-plugins-zx7wk" Jul 30 00:17:18.447385 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.447247 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knxqr\" (UniqueName: \"kubernetes.io/projected/add0c4dc-1cb0-4fa3-903a-6ce3b4a8b60a-kube-api-access-knxqr\") pod \"node-resolver-zlqlw\" (UID: \"add0c4dc-1cb0-4fa3-903a-6ce3b4a8b60a\") " pod="openshift-dns/node-resolver-zlqlw" Jul 30 00:17:18.447385 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.447280 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtjq7\" (UniqueName: \"kubernetes.io/projected/3806f8c4-b735-4fd6-a240-882dd0c2c70a-kube-api-access-gtjq7\") pod \"iptables-alerter-wgvqx\" (UID: \"3806f8c4-b735-4fd6-a240-882dd0c2c70a\") " pod="openshift-network-operator/iptables-alerter-wgvqx" Jul 30 00:17:18.447385 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.447306 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f099a95a-e24f-4e05-bba1-bccd334f000c-kubelet-config\") pod \"global-pull-secret-syncer-j26vw\" (UID: \"f099a95a-e24f-4e05-bba1-bccd334f000c\") " pod="kube-system/global-pull-secret-syncer-j26vw" Jul 30 00:17:18.447385 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.447330 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f95c8416-e10e-4357-96a7-4b48e1c3be4f-host-run-netns\") pod \"multus-hglh4\" (UID: \"f95c8416-e10e-4357-96a7-4b48e1c3be4f\") " pod="openshift-multus/multus-hglh4" Jul 30 00:17:18.447612 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.447382 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f099a95a-e24f-4e05-bba1-bccd334f000c-kubelet-config\") pod \"global-pull-secret-syncer-j26vw\" (UID: \"f099a95a-e24f-4e05-bba1-bccd334f000c\") " pod="kube-system/global-pull-secret-syncer-j26vw" Jul 30 00:17:18.447612 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.447384 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f95c8416-e10e-4357-96a7-4b48e1c3be4f-host-run-netns\") pod \"multus-hglh4\" (UID: \"f95c8416-e10e-4357-96a7-4b48e1c3be4f\") " pod="openshift-multus/multus-hglh4" Jul 30 00:17:18.447612 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.447477 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/52037ff9-396d-4e0d-958b-ff96d3df4855-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.447612 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.447528 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/52037ff9-396d-4e0d-958b-ff96d3df4855-ovnkube-script-lib\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.447612 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.447557 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f5cd0fd1-6297-45d6-9823-cd424bce4185-cnibin\") pod \"multus-additional-cni-plugins-zx7wk\" (UID: \"f5cd0fd1-6297-45d6-9823-cd424bce4185\") " pod="openshift-multus/multus-additional-cni-plugins-zx7wk" Jul 30 00:17:18.447612 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.447560 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f5cd0fd1-6297-45d6-9823-cd424bce4185-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zx7wk\" (UID: \"f5cd0fd1-6297-45d6-9823-cd424bce4185\") " pod="openshift-multus/multus-additional-cni-plugins-zx7wk" Jul 30 00:17:18.447612 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.447589 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/52037ff9-396d-4e0d-958b-ff96d3df4855-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.447612 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.447602 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b235c4f3-5e9b-4866-99c2-41ec0a64f806-etc-sysctl-d\") pod \"tuned-6wfs6\" (UID: \"b235c4f3-5e9b-4866-99c2-41ec0a64f806\") " pod="openshift-cluster-node-tuning-operator/tuned-6wfs6" Jul 30 00:17:18.448018 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.447635 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqcn9\" (UniqueName: \"kubernetes.io/projected/b235c4f3-5e9b-4866-99c2-41ec0a64f806-kube-api-access-sqcn9\") pod \"tuned-6wfs6\" (UID: \"b235c4f3-5e9b-4866-99c2-41ec0a64f806\") " pod="openshift-cluster-node-tuning-operator/tuned-6wfs6" Jul 30 00:17:18.448018 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.447642 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f5cd0fd1-6297-45d6-9823-cd424bce4185-cnibin\") pod \"multus-additional-cni-plugins-zx7wk\" (UID: \"f5cd0fd1-6297-45d6-9823-cd424bce4185\") " pod="openshift-multus/multus-additional-cni-plugins-zx7wk" Jul 30 00:17:18.448018 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.447661 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f95c8416-e10e-4357-96a7-4b48e1c3be4f-multus-daemon-config\") pod \"multus-hglh4\" (UID: \"f95c8416-e10e-4357-96a7-4b48e1c3be4f\") " pod="openshift-multus/multus-hglh4" Jul 30 00:17:18.448018 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.447756 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b235c4f3-5e9b-4866-99c2-41ec0a64f806-var-lib-kubelet\") pod \"tuned-6wfs6\" (UID: \"b235c4f3-5e9b-4866-99c2-41ec0a64f806\") " pod="openshift-cluster-node-tuning-operator/tuned-6wfs6" Jul 30 00:17:18.448018 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.447785 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f95c8416-e10e-4357-96a7-4b48e1c3be4f-host-var-lib-kubelet\") pod \"multus-hglh4\" (UID: \"f95c8416-e10e-4357-96a7-4b48e1c3be4f\") " pod="openshift-multus/multus-hglh4" Jul 30 00:17:18.448018 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.447798 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b235c4f3-5e9b-4866-99c2-41ec0a64f806-etc-sysctl-d\") pod \"tuned-6wfs6\" (UID: \"b235c4f3-5e9b-4866-99c2-41ec0a64f806\") " pod="openshift-cluster-node-tuning-operator/tuned-6wfs6" Jul 30 00:17:18.448018 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.447810 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f95c8416-e10e-4357-96a7-4b48e1c3be4f-etc-kubernetes\") pod \"multus-hglh4\" (UID: \"f95c8416-e10e-4357-96a7-4b48e1c3be4f\") " pod="openshift-multus/multus-hglh4" Jul 30 00:17:18.448018 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.447849 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b235c4f3-5e9b-4866-99c2-41ec0a64f806-var-lib-kubelet\") pod \"tuned-6wfs6\" (UID: \"b235c4f3-5e9b-4866-99c2-41ec0a64f806\") " pod="openshift-cluster-node-tuning-operator/tuned-6wfs6" Jul 30 00:17:18.448018 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.447847 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8567e26f-2898-4d1b-8000-b8bd3326c9ac-metrics-certs\") pod \"network-metrics-daemon-zxg4x\" (UID: \"8567e26f-2898-4d1b-8000-b8bd3326c9ac\") " pod="openshift-multus/network-metrics-daemon-zxg4x" Jul 30 00:17:18.448018 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.447876 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxjc4\" (UniqueName: \"kubernetes.io/projected/3101459e-308a-4623-884f-8f54273f1bfd-kube-api-access-qxjc4\") pod \"network-check-target-z7xxn\" (UID: \"3101459e-308a-4623-884f-8f54273f1bfd\") " pod="openshift-network-diagnostics/network-check-target-z7xxn" Jul 30 00:17:18.448018 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.447891 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52037ff9-396d-4e0d-958b-ff96d3df4855-run-openvswitch\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.448018 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.447919 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/add0c4dc-1cb0-4fa3-903a-6ce3b4a8b60a-tmp-dir\") pod \"node-resolver-zlqlw\" (UID: \"add0c4dc-1cb0-4fa3-903a-6ce3b4a8b60a\") " pod="openshift-dns/node-resolver-zlqlw" Jul 30 00:17:18.448018 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.447937 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b235c4f3-5e9b-4866-99c2-41ec0a64f806-etc-systemd\") pod \"tuned-6wfs6\" (UID: \"b235c4f3-5e9b-4866-99c2-41ec0a64f806\") " pod="openshift-cluster-node-tuning-operator/tuned-6wfs6" Jul 30 00:17:18.448018 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.447955 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3806f8c4-b735-4fd6-a240-882dd0c2c70a-host-slash\") pod \"iptables-alerter-wgvqx\" (UID: \"3806f8c4-b735-4fd6-a240-882dd0c2c70a\") " pod="openshift-network-operator/iptables-alerter-wgvqx" Jul 30 00:17:18.448018 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.447978 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f95c8416-e10e-4357-96a7-4b48e1c3be4f-host-var-lib-kubelet\") pod \"multus-hglh4\" (UID: \"f95c8416-e10e-4357-96a7-4b48e1c3be4f\") " pod="openshift-multus/multus-hglh4" Jul 30 00:17:18.448800 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.448028 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f95c8416-e10e-4357-96a7-4b48e1c3be4f-etc-kubernetes\") pod \"multus-hglh4\" (UID: \"f95c8416-e10e-4357-96a7-4b48e1c3be4f\") " pod="openshift-multus/multus-hglh4" Jul 30 00:17:18.448800 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.448005 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f95c8416-e10e-4357-96a7-4b48e1c3be4f-multus-socket-dir-parent\") pod \"multus-hglh4\" (UID: \"f95c8416-e10e-4357-96a7-4b48e1c3be4f\") " pod="openshift-multus/multus-hglh4" Jul 30 00:17:18.448800 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.448061 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b235c4f3-5e9b-4866-99c2-41ec0a64f806-etc-systemd\") pod \"tuned-6wfs6\" (UID: \"b235c4f3-5e9b-4866-99c2-41ec0a64f806\") " pod="openshift-cluster-node-tuning-operator/tuned-6wfs6" Jul 30 00:17:18.448800 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:18.448006 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 00:17:18.448800 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.448075 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j68ms\" (UniqueName: \"kubernetes.io/projected/f95c8416-e10e-4357-96a7-4b48e1c3be4f-kube-api-access-j68ms\") pod \"multus-hglh4\" (UID: \"f95c8416-e10e-4357-96a7-4b48e1c3be4f\") " pod="openshift-multus/multus-hglh4" Jul 30 00:17:18.448800 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.448134 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/52037ff9-396d-4e0d-958b-ff96d3df4855-systemd-units\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.448800 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:18.448175 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8567e26f-2898-4d1b-8000-b8bd3326c9ac-metrics-certs podName:8567e26f-2898-4d1b-8000-b8bd3326c9ac nodeName:}" failed. No retries permitted until 2026-07-30 00:17:18.948134748 +0000 UTC m=+3.006890708 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8567e26f-2898-4d1b-8000-b8bd3326c9ac-metrics-certs") pod "network-metrics-daemon-zxg4x" (UID: "8567e26f-2898-4d1b-8000-b8bd3326c9ac") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 00:17:18.448800 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.448178 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/52037ff9-396d-4e0d-958b-ff96d3df4855-systemd-units\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.448800 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.448044 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3806f8c4-b735-4fd6-a240-882dd0c2c70a-host-slash\") pod \"iptables-alerter-wgvqx\" (UID: \"3806f8c4-b735-4fd6-a240-882dd0c2c70a\") " pod="openshift-network-operator/iptables-alerter-wgvqx" Jul 30 00:17:18.448800 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.448204 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f5cd0fd1-6297-45d6-9823-cd424bce4185-os-release\") pod \"multus-additional-cni-plugins-zx7wk\" (UID: \"f5cd0fd1-6297-45d6-9823-cd424bce4185\") " pod="openshift-multus/multus-additional-cni-plugins-zx7wk" Jul 30 00:17:18.448800 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.448235 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c882608c-cf7a-4e13-a031-193b7c4ac1dc-registration-dir\") pod \"aws-ebs-csi-driver-node-fvgfr\" (UID: \"c882608c-cf7a-4e13-a031-193b7c4ac1dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvgfr" Jul 30 00:17:18.448800 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.448264 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c882608c-cf7a-4e13-a031-193b7c4ac1dc-sys-fs\") pod \"aws-ebs-csi-driver-node-fvgfr\" (UID: \"c882608c-cf7a-4e13-a031-193b7c4ac1dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvgfr" Jul 30 00:17:18.448800 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.448290 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dddts\" (UniqueName: \"kubernetes.io/projected/756ee6d1-bf62-49db-954a-ed5d66384c54-kube-api-access-dddts\") pod \"node-ca-gp6l9\" (UID: \"756ee6d1-bf62-49db-954a-ed5d66384c54\") " pod="openshift-image-registry/node-ca-gp6l9" Jul 30 00:17:18.448800 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.448314 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/52037ff9-396d-4e0d-958b-ff96d3df4855-log-socket\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.448800 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.448343 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b235c4f3-5e9b-4866-99c2-41ec0a64f806-etc-modprobe-d\") pod \"tuned-6wfs6\" (UID: \"b235c4f3-5e9b-4866-99c2-41ec0a64f806\") " pod="openshift-cluster-node-tuning-operator/tuned-6wfs6" Jul 30 00:17:18.448800 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.448354 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f95c8416-e10e-4357-96a7-4b48e1c3be4f-multus-daemon-config\") pod \"multus-hglh4\" (UID: \"f95c8416-e10e-4357-96a7-4b48e1c3be4f\") " pod="openshift-multus/multus-hglh4" Jul 30 00:17:18.448800 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.448370 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b235c4f3-5e9b-4866-99c2-41ec0a64f806-tmp\") pod \"tuned-6wfs6\" (UID: \"b235c4f3-5e9b-4866-99c2-41ec0a64f806\") " pod="openshift-cluster-node-tuning-operator/tuned-6wfs6" Jul 30 00:17:18.449631 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.448376 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52037ff9-396d-4e0d-958b-ff96d3df4855-run-openvswitch\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.449631 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.448396 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c882608c-cf7a-4e13-a031-193b7c4ac1dc-etc-selinux\") pod \"aws-ebs-csi-driver-node-fvgfr\" (UID: \"c882608c-cf7a-4e13-a031-193b7c4ac1dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvgfr" Jul 30 00:17:18.449631 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.448423 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/52037ff9-396d-4e0d-958b-ff96d3df4855-env-overrides\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.449631 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.448423 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/52037ff9-396d-4e0d-958b-ff96d3df4855-log-socket\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.449631 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.448330 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/add0c4dc-1cb0-4fa3-903a-6ce3b4a8b60a-tmp-dir\") pod \"node-resolver-zlqlw\" (UID: \"add0c4dc-1cb0-4fa3-903a-6ce3b4a8b60a\") " pod="openshift-dns/node-resolver-zlqlw" Jul 30 00:17:18.449631 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.448442 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/52037ff9-396d-4e0d-958b-ff96d3df4855-ovnkube-script-lib\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.449631 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.448461 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f099a95a-e24f-4e05-bba1-bccd334f000c-original-pull-secret\") pod \"global-pull-secret-syncer-j26vw\" (UID: \"f099a95a-e24f-4e05-bba1-bccd334f000c\") " pod="kube-system/global-pull-secret-syncer-j26vw" Jul 30 00:17:18.449631 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.448511 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f5cd0fd1-6297-45d6-9823-cd424bce4185-os-release\") pod \"multus-additional-cni-plugins-zx7wk\" (UID: \"f5cd0fd1-6297-45d6-9823-cd424bce4185\") " pod="openshift-multus/multus-additional-cni-plugins-zx7wk" Jul 30 00:17:18.449631 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.448515 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f95c8416-e10e-4357-96a7-4b48e1c3be4f-host-run-multus-certs\") pod \"multus-hglh4\" (UID: \"f95c8416-e10e-4357-96a7-4b48e1c3be4f\") " pod="openshift-multus/multus-hglh4" Jul 30 00:17:18.449631 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.448553 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/52037ff9-396d-4e0d-958b-ff96d3df4855-run-ovn\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.449631 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.448564 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c882608c-cf7a-4e13-a031-193b7c4ac1dc-sys-fs\") pod \"aws-ebs-csi-driver-node-fvgfr\" (UID: \"c882608c-cf7a-4e13-a031-193b7c4ac1dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvgfr" Jul 30 00:17:18.449631 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:18.448576 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 30 00:17:18.449631 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.448581 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ad9e265e-16ad-428a-a69a-f9a7948e5482-agent-certs\") pod \"konnectivity-agent-9m2mr\" (UID: \"ad9e265e-16ad-428a-a69a-f9a7948e5482\") " pod="kube-system/konnectivity-agent-9m2mr" Jul 30 00:17:18.449631 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.448556 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f95c8416-e10e-4357-96a7-4b48e1c3be4f-host-run-multus-certs\") pod \"multus-hglh4\" (UID: \"f95c8416-e10e-4357-96a7-4b48e1c3be4f\") " pod="openshift-multus/multus-hglh4" Jul 30 00:17:18.449631 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.448674 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/52037ff9-396d-4e0d-958b-ff96d3df4855-run-ovn\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.449631 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.448691 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c882608c-cf7a-4e13-a031-193b7c4ac1dc-etc-selinux\") pod \"aws-ebs-csi-driver-node-fvgfr\" (UID: \"c882608c-cf7a-4e13-a031-193b7c4ac1dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvgfr" Jul 30 00:17:18.449631 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.448693 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f95c8416-e10e-4357-96a7-4b48e1c3be4f-multus-socket-dir-parent\") pod \"multus-hglh4\" (UID: \"f95c8416-e10e-4357-96a7-4b48e1c3be4f\") " pod="openshift-multus/multus-hglh4" Jul 30 00:17:18.449631 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.448728 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c882608c-cf7a-4e13-a031-193b7c4ac1dc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fvgfr\" (UID: \"c882608c-cf7a-4e13-a031-193b7c4ac1dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvgfr" Jul 30 00:17:18.450413 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:18.448776 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f099a95a-e24f-4e05-bba1-bccd334f000c-original-pull-secret podName:f099a95a-e24f-4e05-bba1-bccd334f000c nodeName:}" failed. No retries permitted until 2026-07-30 00:17:18.948757875 +0000 UTC m=+3.007513828 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f099a95a-e24f-4e05-bba1-bccd334f000c-original-pull-secret") pod "global-pull-secret-syncer-j26vw" (UID: "f099a95a-e24f-4e05-bba1-bccd334f000c") : object "kube-system"/"original-pull-secret" not registered Jul 30 00:17:18.450413 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.448805 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c882608c-cf7a-4e13-a031-193b7c4ac1dc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fvgfr\" (UID: \"c882608c-cf7a-4e13-a031-193b7c4ac1dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvgfr" Jul 30 00:17:18.450413 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.448824 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/52037ff9-396d-4e0d-958b-ff96d3df4855-env-overrides\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.450413 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.448835 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f95c8416-e10e-4357-96a7-4b48e1c3be4f-multus-cni-dir\") pod \"multus-hglh4\" (UID: \"f95c8416-e10e-4357-96a7-4b48e1c3be4f\") " pod="openshift-multus/multus-hglh4" Jul 30 00:17:18.450413 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.448826 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b235c4f3-5e9b-4866-99c2-41ec0a64f806-etc-modprobe-d\") pod \"tuned-6wfs6\" (UID: \"b235c4f3-5e9b-4866-99c2-41ec0a64f806\") " pod="openshift-cluster-node-tuning-operator/tuned-6wfs6" Jul 30 00:17:18.450413 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.448874 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c882608c-cf7a-4e13-a031-193b7c4ac1dc-registration-dir\") pod \"aws-ebs-csi-driver-node-fvgfr\" (UID: \"c882608c-cf7a-4e13-a031-193b7c4ac1dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvgfr" Jul 30 00:17:18.450413 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.448897 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b235c4f3-5e9b-4866-99c2-41ec0a64f806-lib-modules\") pod \"tuned-6wfs6\" (UID: \"b235c4f3-5e9b-4866-99c2-41ec0a64f806\") " pod="openshift-cluster-node-tuning-operator/tuned-6wfs6" Jul 30 00:17:18.450413 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.448957 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3806f8c4-b735-4fd6-a240-882dd0c2c70a-iptables-alerter-script\") pod \"iptables-alerter-wgvqx\" (UID: \"3806f8c4-b735-4fd6-a240-882dd0c2c70a\") " pod="openshift-network-operator/iptables-alerter-wgvqx" Jul 30 00:17:18.450413 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.448982 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b235c4f3-5e9b-4866-99c2-41ec0a64f806-lib-modules\") pod \"tuned-6wfs6\" (UID: \"b235c4f3-5e9b-4866-99c2-41ec0a64f806\") " pod="openshift-cluster-node-tuning-operator/tuned-6wfs6" Jul 30 00:17:18.450413 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.448969 2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 30 00:17:18.450413 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.449011 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f95c8416-e10e-4357-96a7-4b48e1c3be4f-multus-cni-dir\") pod \"multus-hglh4\" (UID: \"f95c8416-e10e-4357-96a7-4b48e1c3be4f\") " pod="openshift-multus/multus-hglh4" Jul 30 00:17:18.450413 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.449181 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c882608c-cf7a-4e13-a031-193b7c4ac1dc-socket-dir\") pod \"aws-ebs-csi-driver-node-fvgfr\" (UID: \"c882608c-cf7a-4e13-a031-193b7c4ac1dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvgfr" Jul 30 00:17:18.450413 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.449291 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c882608c-cf7a-4e13-a031-193b7c4ac1dc-socket-dir\") pod \"aws-ebs-csi-driver-node-fvgfr\" (UID: \"c882608c-cf7a-4e13-a031-193b7c4ac1dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvgfr" Jul 30 00:17:18.450413 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.449261 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f95c8416-e10e-4357-96a7-4b48e1c3be4f-host-run-k8s-cni-cncf-io\") pod \"multus-hglh4\" (UID: \"f95c8416-e10e-4357-96a7-4b48e1c3be4f\") " pod="openshift-multus/multus-hglh4" Jul 30 00:17:18.450413 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.449340 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f95c8416-e10e-4357-96a7-4b48e1c3be4f-host-var-lib-cni-bin\") pod \"multus-hglh4\" (UID: \"f95c8416-e10e-4357-96a7-4b48e1c3be4f\") " pod="openshift-multus/multus-hglh4" Jul 30 00:17:18.450413 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.450079 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/52037ff9-396d-4e0d-958b-ff96d3df4855-node-log\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.452199 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.451273 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f95c8416-e10e-4357-96a7-4b48e1c3be4f-host-run-k8s-cni-cncf-io\") pod \"multus-hglh4\" (UID: \"f95c8416-e10e-4357-96a7-4b48e1c3be4f\") " pod="openshift-multus/multus-hglh4" Jul 30 00:17:18.452199 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.451357 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/52037ff9-396d-4e0d-958b-ff96d3df4855-host-cni-bin\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.452199 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.451375 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f95c8416-e10e-4357-96a7-4b48e1c3be4f-host-var-lib-cni-bin\") pod \"multus-hglh4\" (UID: \"f95c8416-e10e-4357-96a7-4b48e1c3be4f\") " pod="openshift-multus/multus-hglh4" Jul 30 00:17:18.452199 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.451409 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/756ee6d1-bf62-49db-954a-ed5d66384c54-host\") pod \"node-ca-gp6l9\" (UID: \"756ee6d1-bf62-49db-954a-ed5d66384c54\") " pod="openshift-image-registry/node-ca-gp6l9" Jul 30 00:17:18.452199 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.451442 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/52037ff9-396d-4e0d-958b-ff96d3df4855-host-cni-bin\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.452199 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.451454 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/52037ff9-396d-4e0d-958b-ff96d3df4855-host-kubelet\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.452199 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.451492 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/756ee6d1-bf62-49db-954a-ed5d66384c54-host\") pod \"node-ca-gp6l9\" (UID: \"756ee6d1-bf62-49db-954a-ed5d66384c54\") " pod="openshift-image-registry/node-ca-gp6l9" Jul 30 00:17:18.452199 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.451518 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/52037ff9-396d-4e0d-958b-ff96d3df4855-host-kubelet\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.452199 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.451561 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3806f8c4-b735-4fd6-a240-882dd0c2c70a-iptables-alerter-script\") pod \"iptables-alerter-wgvqx\" (UID: \"3806f8c4-b735-4fd6-a240-882dd0c2c70a\") " pod="openshift-network-operator/iptables-alerter-wgvqx" Jul 30 00:17:18.452199 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.451569 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/52037ff9-396d-4e0d-958b-ff96d3df4855-host-run-netns\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.452199 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.451603 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f5cd0fd1-6297-45d6-9823-cd424bce4185-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zx7wk\" (UID: \"f5cd0fd1-6297-45d6-9823-cd424bce4185\") " pod="openshift-multus/multus-additional-cni-plugins-zx7wk" Jul 30 00:17:18.452199 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.451635 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/52037ff9-396d-4e0d-958b-ff96d3df4855-node-log\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.452199 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.451656 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b235c4f3-5e9b-4866-99c2-41ec0a64f806-sys\") pod \"tuned-6wfs6\" (UID: \"b235c4f3-5e9b-4866-99c2-41ec0a64f806\") " pod="openshift-cluster-node-tuning-operator/tuned-6wfs6" Jul 30 00:17:18.452199 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.451681 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b235c4f3-5e9b-4866-99c2-41ec0a64f806-etc-tuned\") pod \"tuned-6wfs6\" (UID: \"b235c4f3-5e9b-4866-99c2-41ec0a64f806\") " pod="openshift-cluster-node-tuning-operator/tuned-6wfs6" Jul 30 00:17:18.452199 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.451687 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/52037ff9-396d-4e0d-958b-ff96d3df4855-host-run-netns\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.453070 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.453048 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2g6bd\" (UniqueName: \"kubernetes.io/projected/c882608c-cf7a-4e13-a031-193b7c4ac1dc-kube-api-access-2g6bd\") pod \"aws-ebs-csi-driver-node-fvgfr\" (UID: \"c882608c-cf7a-4e13-a031-193b7c4ac1dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvgfr" Jul 30 00:17:18.453147 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.453090 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/52037ff9-396d-4e0d-958b-ff96d3df4855-host-run-ovn-kubernetes\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.453147 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.453125 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/52037ff9-396d-4e0d-958b-ff96d3df4855-ovnkube-config\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.453242 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.453155 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/52037ff9-396d-4e0d-958b-ff96d3df4855-ovn-node-metrics-cert\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.453242 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.453185 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b235c4f3-5e9b-4866-99c2-41ec0a64f806-etc-kubernetes\") pod \"tuned-6wfs6\" (UID: \"b235c4f3-5e9b-4866-99c2-41ec0a64f806\") " pod="openshift-cluster-node-tuning-operator/tuned-6wfs6" Jul 30 00:17:18.453242 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.453214 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b235c4f3-5e9b-4866-99c2-41ec0a64f806-etc-sysctl-conf\") pod \"tuned-6wfs6\" (UID: \"b235c4f3-5e9b-4866-99c2-41ec0a64f806\") " pod="openshift-cluster-node-tuning-operator/tuned-6wfs6" Jul 30 00:17:18.453396 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.453245 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b235c4f3-5e9b-4866-99c2-41ec0a64f806-run\") pod \"tuned-6wfs6\" (UID: \"b235c4f3-5e9b-4866-99c2-41ec0a64f806\") " pod="openshift-cluster-node-tuning-operator/tuned-6wfs6" Jul 30 00:17:18.453396 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.453276 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f5cd0fd1-6297-45d6-9823-cd424bce4185-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zx7wk\" (UID: \"f5cd0fd1-6297-45d6-9823-cd424bce4185\") " pod="openshift-multus/multus-additional-cni-plugins-zx7wk" Jul 30 00:17:18.453396 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.453282 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c882608c-cf7a-4e13-a031-193b7c4ac1dc-device-dir\") pod \"aws-ebs-csi-driver-node-fvgfr\" (UID: \"c882608c-cf7a-4e13-a031-193b7c4ac1dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvgfr" Jul 30 00:17:18.453396 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.453329 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f099a95a-e24f-4e05-bba1-bccd334f000c-dbus\") pod \"global-pull-secret-syncer-j26vw\" (UID: \"f099a95a-e24f-4e05-bba1-bccd334f000c\") " pod="kube-system/global-pull-secret-syncer-j26vw" Jul 30 00:17:18.453396 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.453335 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c882608c-cf7a-4e13-a031-193b7c4ac1dc-device-dir\") pod \"aws-ebs-csi-driver-node-fvgfr\" (UID: \"c882608c-cf7a-4e13-a031-193b7c4ac1dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvgfr" Jul 30 00:17:18.453396 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.453355 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f95c8416-e10e-4357-96a7-4b48e1c3be4f-os-release\") pod \"multus-hglh4\" (UID: \"f95c8416-e10e-4357-96a7-4b48e1c3be4f\") " pod="openshift-multus/multus-hglh4" Jul 30 00:17:18.453396 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.453389 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2nxf\" (UniqueName: \"kubernetes.io/projected/8567e26f-2898-4d1b-8000-b8bd3326c9ac-kube-api-access-x2nxf\") pod \"network-metrics-daemon-zxg4x\" (UID: \"8567e26f-2898-4d1b-8000-b8bd3326c9ac\") " pod="openshift-multus/network-metrics-daemon-zxg4x" Jul 30 00:17:18.453728 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.453419 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/52037ff9-396d-4e0d-958b-ff96d3df4855-host-cni-netd\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.453728 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.453450 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ad9e265e-16ad-428a-a69a-f9a7948e5482-konnectivity-ca\") pod \"konnectivity-agent-9m2mr\" (UID: \"ad9e265e-16ad-428a-a69a-f9a7948e5482\") " pod="kube-system/konnectivity-agent-9m2mr" Jul 30 00:17:18.453728 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.453475 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/add0c4dc-1cb0-4fa3-903a-6ce3b4a8b60a-hosts-file\") pod \"node-resolver-zlqlw\" (UID: \"add0c4dc-1cb0-4fa3-903a-6ce3b4a8b60a\") " pod="openshift-dns/node-resolver-zlqlw" Jul 30 00:17:18.453728 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.453506 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4ch2\" (UniqueName: \"kubernetes.io/projected/f5cd0fd1-6297-45d6-9823-cd424bce4185-kube-api-access-v4ch2\") pod \"multus-additional-cni-plugins-zx7wk\" (UID: \"f5cd0fd1-6297-45d6-9823-cd424bce4185\") " pod="openshift-multus/multus-additional-cni-plugins-zx7wk" Jul 30 00:17:18.453728 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.453537 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f5cd0fd1-6297-45d6-9823-cd424bce4185-cni-binary-copy\") pod \"multus-additional-cni-plugins-zx7wk\" (UID: \"f5cd0fd1-6297-45d6-9823-cd424bce4185\") " pod="openshift-multus/multus-additional-cni-plugins-zx7wk" Jul 30 00:17:18.453728 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.453569 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f95c8416-e10e-4357-96a7-4b48e1c3be4f-system-cni-dir\") pod \"multus-hglh4\" (UID: \"f95c8416-e10e-4357-96a7-4b48e1c3be4f\") " pod="openshift-multus/multus-hglh4" Jul 30 00:17:18.453728 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.453595 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f95c8416-e10e-4357-96a7-4b48e1c3be4f-cnibin\") pod \"multus-hglh4\" (UID: \"f95c8416-e10e-4357-96a7-4b48e1c3be4f\") " pod="openshift-multus/multus-hglh4" Jul 30 00:17:18.453728 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.453622 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f95c8416-e10e-4357-96a7-4b48e1c3be4f-cni-binary-copy\") pod \"multus-hglh4\" (UID: \"f95c8416-e10e-4357-96a7-4b48e1c3be4f\") " pod="openshift-multus/multus-hglh4" Jul 30 00:17:18.453728 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.453652 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/756ee6d1-bf62-49db-954a-ed5d66384c54-serviceca\") pod \"node-ca-gp6l9\" (UID: \"756ee6d1-bf62-49db-954a-ed5d66384c54\") " pod="openshift-image-registry/node-ca-gp6l9" Jul 30 00:17:18.453728 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.453667 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b235c4f3-5e9b-4866-99c2-41ec0a64f806-sys\") pod \"tuned-6wfs6\" (UID: \"b235c4f3-5e9b-4866-99c2-41ec0a64f806\") " pod="openshift-cluster-node-tuning-operator/tuned-6wfs6" Jul 30 00:17:18.453728 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.453680 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52037ff9-396d-4e0d-958b-ff96d3df4855-var-lib-openvswitch\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.454272 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.453814 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f099a95a-e24f-4e05-bba1-bccd334f000c-dbus\") pod \"global-pull-secret-syncer-j26vw\" (UID: \"f099a95a-e24f-4e05-bba1-bccd334f000c\") " pod="kube-system/global-pull-secret-syncer-j26vw" Jul 30 00:17:18.454272 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.453874 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f95c8416-e10e-4357-96a7-4b48e1c3be4f-os-release\") pod \"multus-hglh4\" (UID: \"f95c8416-e10e-4357-96a7-4b48e1c3be4f\") " pod="openshift-multus/multus-hglh4" Jul 30 00:17:18.454272 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.454125 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/52037ff9-396d-4e0d-958b-ff96d3df4855-host-cni-netd\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.454618 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.454593 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ad9e265e-16ad-428a-a69a-f9a7948e5482-konnectivity-ca\") pod \"konnectivity-agent-9m2mr\" (UID: \"ad9e265e-16ad-428a-a69a-f9a7948e5482\") " pod="kube-system/konnectivity-agent-9m2mr" Jul 30 00:17:18.454689 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.454679 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/add0c4dc-1cb0-4fa3-903a-6ce3b4a8b60a-hosts-file\") pod \"node-resolver-zlqlw\" (UID: \"add0c4dc-1cb0-4fa3-903a-6ce3b4a8b60a\") " pod="openshift-dns/node-resolver-zlqlw" Jul 30 00:17:18.455864 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.455129 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f95c8416-e10e-4357-96a7-4b48e1c3be4f-cnibin\") pod \"multus-hglh4\" (UID: \"f95c8416-e10e-4357-96a7-4b48e1c3be4f\") " pod="openshift-multus/multus-hglh4" Jul 30 00:17:18.455864 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.455328 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f95c8416-e10e-4357-96a7-4b48e1c3be4f-cni-binary-copy\") pod \"multus-hglh4\" (UID: \"f95c8416-e10e-4357-96a7-4b48e1c3be4f\") " pod="openshift-multus/multus-hglh4" Jul 30 00:17:18.455864 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.455823 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52037ff9-396d-4e0d-958b-ff96d3df4855-var-lib-openvswitch\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.456071 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.455993 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b235c4f3-5e9b-4866-99c2-41ec0a64f806-tmp\") pod \"tuned-6wfs6\" (UID: \"b235c4f3-5e9b-4866-99c2-41ec0a64f806\") " pod="openshift-cluster-node-tuning-operator/tuned-6wfs6" Jul 30 00:17:18.456071 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.455701 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/756ee6d1-bf62-49db-954a-ed5d66384c54-serviceca\") pod \"node-ca-gp6l9\" (UID: \"756ee6d1-bf62-49db-954a-ed5d66384c54\") " pod="openshift-image-registry/node-ca-gp6l9" Jul 30 00:17:18.456071 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:18.456050 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 30 00:17:18.456222 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:18.456075 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 30 00:17:18.456222 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:18.456099 2563 projected.go:196] Error preparing data for projected volume kube-api-access-qxjc4 for pod openshift-network-diagnostics/network-check-target-z7xxn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 00:17:18.456222 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:18.456164 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3101459e-308a-4623-884f-8f54273f1bfd-kube-api-access-qxjc4 podName:3101459e-308a-4623-884f-8f54273f1bfd nodeName:}" failed. No retries permitted until 2026-07-30 00:17:18.956139405 +0000 UTC m=+3.014895352 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-qxjc4" (UniqueName: "kubernetes.io/projected/3101459e-308a-4623-884f-8f54273f1bfd-kube-api-access-qxjc4") pod "network-check-target-z7xxn" (UID: "3101459e-308a-4623-884f-8f54273f1bfd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 00:17:18.456376 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.456252 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f5cd0fd1-6297-45d6-9823-cd424bce4185-cni-binary-copy\") pod \"multus-additional-cni-plugins-zx7wk\" (UID: \"f5cd0fd1-6297-45d6-9823-cd424bce4185\") " pod="openshift-multus/multus-additional-cni-plugins-zx7wk" Jul 30 00:17:18.456376 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.456263 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f95c8416-e10e-4357-96a7-4b48e1c3be4f-system-cni-dir\") pod \"multus-hglh4\" (UID: \"f95c8416-e10e-4357-96a7-4b48e1c3be4f\") " pod="openshift-multus/multus-hglh4" Jul 30 00:17:18.456376 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.456346 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b235c4f3-5e9b-4866-99c2-41ec0a64f806-etc-kubernetes\") pod \"tuned-6wfs6\" (UID: \"b235c4f3-5e9b-4866-99c2-41ec0a64f806\") " pod="openshift-cluster-node-tuning-operator/tuned-6wfs6" Jul 30 00:17:18.456535 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.456472 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b235c4f3-5e9b-4866-99c2-41ec0a64f806-etc-sysctl-conf\") pod \"tuned-6wfs6\" (UID: \"b235c4f3-5e9b-4866-99c2-41ec0a64f806\") " pod="openshift-cluster-node-tuning-operator/tuned-6wfs6" Jul 30 00:17:18.456594 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.456531 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b235c4f3-5e9b-4866-99c2-41ec0a64f806-run\") pod \"tuned-6wfs6\" (UID: \"b235c4f3-5e9b-4866-99c2-41ec0a64f806\") " pod="openshift-cluster-node-tuning-operator/tuned-6wfs6" Jul 30 00:17:18.456763 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.456727 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/52037ff9-396d-4e0d-958b-ff96d3df4855-host-run-ovn-kubernetes\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.457866 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.457842 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/52037ff9-396d-4e0d-958b-ff96d3df4855-ovnkube-config\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.457990 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.457969 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b235c4f3-5e9b-4866-99c2-41ec0a64f806-etc-tuned\") pod \"tuned-6wfs6\" (UID: \"b235c4f3-5e9b-4866-99c2-41ec0a64f806\") " pod="openshift-cluster-node-tuning-operator/tuned-6wfs6" Jul 30 00:17:18.458308 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.458287 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/52037ff9-396d-4e0d-958b-ff96d3df4855-ovn-node-metrics-cert\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.458386 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.458329 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-knxqr\" (UniqueName: \"kubernetes.io/projected/add0c4dc-1cb0-4fa3-903a-6ce3b4a8b60a-kube-api-access-knxqr\") pod \"node-resolver-zlqlw\" (UID: \"add0c4dc-1cb0-4fa3-903a-6ce3b4a8b60a\") " pod="openshift-dns/node-resolver-zlqlw" Jul 30 00:17:18.458585 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.458541 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96x4r\" (UniqueName: \"kubernetes.io/projected/52037ff9-396d-4e0d-958b-ff96d3df4855-kube-api-access-96x4r\") pod \"ovnkube-node-bpk2r\" (UID: \"52037ff9-396d-4e0d-958b-ff96d3df4855\") " pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.460278 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.460147 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ad9e265e-16ad-428a-a69a-f9a7948e5482-agent-certs\") pod \"konnectivity-agent-9m2mr\" (UID: \"ad9e265e-16ad-428a-a69a-f9a7948e5482\") " pod="kube-system/konnectivity-agent-9m2mr" Jul 30 00:17:18.460278 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.460240 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j68ms\" (UniqueName: \"kubernetes.io/projected/f95c8416-e10e-4357-96a7-4b48e1c3be4f-kube-api-access-j68ms\") pod \"multus-hglh4\" (UID: \"f95c8416-e10e-4357-96a7-4b48e1c3be4f\") " pod="openshift-multus/multus-hglh4" Jul 30 00:17:18.460500 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.460452 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dddts\" (UniqueName: \"kubernetes.io/projected/756ee6d1-bf62-49db-954a-ed5d66384c54-kube-api-access-dddts\") pod \"node-ca-gp6l9\" (UID: \"756ee6d1-bf62-49db-954a-ed5d66384c54\") " pod="openshift-image-registry/node-ca-gp6l9" Jul 30 00:17:18.460608 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.460537 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtjq7\" (UniqueName: \"kubernetes.io/projected/3806f8c4-b735-4fd6-a240-882dd0c2c70a-kube-api-access-gtjq7\") pod \"iptables-alerter-wgvqx\" (UID: \"3806f8c4-b735-4fd6-a240-882dd0c2c70a\") " pod="openshift-network-operator/iptables-alerter-wgvqx" Jul 30 00:17:18.461372 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.461324 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqcn9\" (UniqueName: \"kubernetes.io/projected/b235c4f3-5e9b-4866-99c2-41ec0a64f806-kube-api-access-sqcn9\") pod \"tuned-6wfs6\" (UID: \"b235c4f3-5e9b-4866-99c2-41ec0a64f806\") " pod="openshift-cluster-node-tuning-operator/tuned-6wfs6" Jul 30 00:17:18.464087 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.464065 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4ch2\" (UniqueName: \"kubernetes.io/projected/f5cd0fd1-6297-45d6-9823-cd424bce4185-kube-api-access-v4ch2\") pod \"multus-additional-cni-plugins-zx7wk\" (UID: \"f5cd0fd1-6297-45d6-9823-cd424bce4185\") " pod="openshift-multus/multus-additional-cni-plugins-zx7wk" Jul 30 00:17:18.464664 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.464645 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2nxf\" (UniqueName: \"kubernetes.io/projected/8567e26f-2898-4d1b-8000-b8bd3326c9ac-kube-api-access-x2nxf\") pod \"network-metrics-daemon-zxg4x\" (UID: \"8567e26f-2898-4d1b-8000-b8bd3326c9ac\") " pod="openshift-multus/network-metrics-daemon-zxg4x" Jul 30 00:17:18.464664 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.464655 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2g6bd\" (UniqueName: \"kubernetes.io/projected/c882608c-cf7a-4e13-a031-193b7c4ac1dc-kube-api-access-2g6bd\") pod \"aws-ebs-csi-driver-node-fvgfr\" (UID: \"c882608c-cf7a-4e13-a031-193b7c4ac1dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvgfr" Jul 30 00:17:18.629664 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.629570 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvgfr" Jul 30 00:17:18.637668 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.637647 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zx7wk" Jul 30 00:17:18.647593 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.647572 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hglh4" Jul 30 00:17:18.653330 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.653307 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9m2mr" Jul 30 00:17:18.660041 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.660021 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zlqlw" Jul 30 00:17:18.666581 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.666555 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gp6l9" Jul 30 00:17:18.673147 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.673128 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6wfs6" Jul 30 00:17:18.680634 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.680613 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wgvqx" Jul 30 00:17:18.686174 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.686155 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:18.958268 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.958194 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8567e26f-2898-4d1b-8000-b8bd3326c9ac-metrics-certs\") pod \"network-metrics-daemon-zxg4x\" (UID: \"8567e26f-2898-4d1b-8000-b8bd3326c9ac\") " pod="openshift-multus/network-metrics-daemon-zxg4x" Jul 30 00:17:18.958268 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.958230 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxjc4\" (UniqueName: \"kubernetes.io/projected/3101459e-308a-4623-884f-8f54273f1bfd-kube-api-access-qxjc4\") pod \"network-check-target-z7xxn\" (UID: \"3101459e-308a-4623-884f-8f54273f1bfd\") " pod="openshift-network-diagnostics/network-check-target-z7xxn" Jul 30 00:17:18.958467 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:18.958266 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f099a95a-e24f-4e05-bba1-bccd334f000c-original-pull-secret\") pod \"global-pull-secret-syncer-j26vw\" (UID: \"f099a95a-e24f-4e05-bba1-bccd334f000c\") " pod="kube-system/global-pull-secret-syncer-j26vw" Jul 30 00:17:18.958467 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:18.958374 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 00:17:18.958467 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:18.958387 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 30 00:17:18.958467 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:18.958407 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 30 00:17:18.958467 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:18.958418 2563 projected.go:196] Error preparing data for projected volume kube-api-access-qxjc4 for pod openshift-network-diagnostics/network-check-target-z7xxn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 00:17:18.958467 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:18.958443 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8567e26f-2898-4d1b-8000-b8bd3326c9ac-metrics-certs podName:8567e26f-2898-4d1b-8000-b8bd3326c9ac nodeName:}" failed. No retries permitted until 2026-07-30 00:17:19.958423658 +0000 UTC m=+4.017179622 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8567e26f-2898-4d1b-8000-b8bd3326c9ac-metrics-certs") pod "network-metrics-daemon-zxg4x" (UID: "8567e26f-2898-4d1b-8000-b8bd3326c9ac") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 00:17:18.958467 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:18.958461 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3101459e-308a-4623-884f-8f54273f1bfd-kube-api-access-qxjc4 podName:3101459e-308a-4623-884f-8f54273f1bfd nodeName:}" failed. No retries permitted until 2026-07-30 00:17:19.958451388 +0000 UTC m=+4.017207334 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-qxjc4" (UniqueName: "kubernetes.io/projected/3101459e-308a-4623-884f-8f54273f1bfd-kube-api-access-qxjc4") pod "network-check-target-z7xxn" (UID: "3101459e-308a-4623-884f-8f54273f1bfd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 00:17:18.958467 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:18.958390 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 30 00:17:18.958799 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:18.958490 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f099a95a-e24f-4e05-bba1-bccd334f000c-original-pull-secret podName:f099a95a-e24f-4e05-bba1-bccd334f000c nodeName:}" failed. No retries permitted until 2026-07-30 00:17:19.95848295 +0000 UTC m=+4.017238916 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f099a95a-e24f-4e05-bba1-bccd334f000c-original-pull-secret") pod "global-pull-secret-syncer-j26vw" (UID: "f099a95a-e24f-4e05-bba1-bccd334f000c") : object "kube-system"/"original-pull-secret" not registered Jul 30 00:17:19.123943 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:19.123893 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc882608c_cf7a_4e13_a031_193b7c4ac1dc.slice/crio-5faa5ce6d448a52afcd2b9112461d6872ba65d31b4eaf1b9342b40d5b9970015 WatchSource:0}: Error finding container 5faa5ce6d448a52afcd2b9112461d6872ba65d31b4eaf1b9342b40d5b9970015: Status 404 returned error can't find the container with id 5faa5ce6d448a52afcd2b9112461d6872ba65d31b4eaf1b9342b40d5b9970015 Jul 30 00:17:19.127706 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:19.127587 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3806f8c4_b735_4fd6_a240_882dd0c2c70a.slice/crio-12160bc48cde8de8cbc0e6b294cbd9aacbfd68bf8ccb4b66194f3192f91ee37a WatchSource:0}: Error finding container 12160bc48cde8de8cbc0e6b294cbd9aacbfd68bf8ccb4b66194f3192f91ee37a: Status 404 returned error can't find the container with id 12160bc48cde8de8cbc0e6b294cbd9aacbfd68bf8ccb4b66194f3192f91ee37a Jul 30 00:17:19.130397 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:19.130372 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5cd0fd1_6297_45d6_9823_cd424bce4185.slice/crio-f025427c8008cdc10ba65ae6507ff2cddbdb470218912acf9112b27c51d20a04 WatchSource:0}: Error finding container f025427c8008cdc10ba65ae6507ff2cddbdb470218912acf9112b27c51d20a04: Status 404 returned error can't find the container with id f025427c8008cdc10ba65ae6507ff2cddbdb470218912acf9112b27c51d20a04 Jul 30 00:17:19.131029 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:19.130992 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadd0c4dc_1cb0_4fa3_903a_6ce3b4a8b60a.slice/crio-2431b554f0c10c57cd5fa555347e8fb7c60c767d88281550592676a1c123da64 WatchSource:0}: Error finding container 2431b554f0c10c57cd5fa555347e8fb7c60c767d88281550592676a1c123da64: Status 404 returned error can't find the container with id 2431b554f0c10c57cd5fa555347e8fb7c60c767d88281550592676a1c123da64 Jul 30 00:17:19.132224 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:19.132064 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad9e265e_16ad_428a_a69a_f9a7948e5482.slice/crio-3af873b72ebfab0e6119450b9fa0e23061d731adde56d2cb70c221e5bacc0ed5 WatchSource:0}: Error finding container 3af873b72ebfab0e6119450b9fa0e23061d731adde56d2cb70c221e5bacc0ed5: Status 404 returned error can't find the container with id 3af873b72ebfab0e6119450b9fa0e23061d731adde56d2cb70c221e5bacc0ed5 Jul 30 00:17:19.133038 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:19.132612 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52037ff9_396d_4e0d_958b_ff96d3df4855.slice/crio-3fa63f0d9bdb6776e6700d389a74310ad5deac843c4ad8e7878acb0e776f4e9a WatchSource:0}: Error finding container 3fa63f0d9bdb6776e6700d389a74310ad5deac843c4ad8e7878acb0e776f4e9a: Status 404 returned error can't find the container with id 3fa63f0d9bdb6776e6700d389a74310ad5deac843c4ad8e7878acb0e776f4e9a Jul 30 00:17:19.133962 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:19.133720 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb235c4f3_5e9b_4866_99c2_41ec0a64f806.slice/crio-292f01a825a96e414509bc7d036fbf0822d5e223c70f480804e810fac1572051 WatchSource:0}: Error finding container 292f01a825a96e414509bc7d036fbf0822d5e223c70f480804e810fac1572051: Status 404 returned error can't find the container with id 292f01a825a96e414509bc7d036fbf0822d5e223c70f480804e810fac1572051 Jul 30 00:17:19.134690 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:19.134648 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod756ee6d1_bf62_49db_954a_ed5d66384c54.slice/crio-0ae6485360b44fb32c85d45f3806a26f0e91369d1d992c50975c45802412d9a9 WatchSource:0}: Error finding container 0ae6485360b44fb32c85d45f3806a26f0e91369d1d992c50975c45802412d9a9: Status 404 returned error can't find the container with id 0ae6485360b44fb32c85d45f3806a26f0e91369d1d992c50975c45802412d9a9 Jul 30 00:17:19.135312 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:19.135291 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf95c8416_e10e_4357_96a7_4b48e1c3be4f.slice/crio-b56e82215e31ab30b2b8490514be63bdd78ad55b78fee0210cbdd113ecc48a32 WatchSource:0}: Error finding container b56e82215e31ab30b2b8490514be63bdd78ad55b78fee0210cbdd113ecc48a32: Status 404 returned error can't find the container with id b56e82215e31ab30b2b8490514be63bdd78ad55b78fee0210cbdd113ecc48a32 Jul 30 00:17:19.375990 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:19.375886 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-29 00:12:18 +0000 UTC" deadline="2028-01-03 18:15:41.010095163 +0000 UTC" Jul 30 00:17:19.375990 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:19.375939 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12545h58m21.634172667s" Jul 30 00:17:19.477720 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:19.477682 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-10.ec2.internal" event={"ID":"d23058fd1d3df9b79e28eb07292f4443","Type":"ContainerStarted","Data":"4b3c7933f267915f56e1cf74bdaa6af0879f787e26f43838337958f2dd1278eb"} Jul 30 00:17:19.478704 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:19.478678 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6wfs6" event={"ID":"b235c4f3-5e9b-4866-99c2-41ec0a64f806","Type":"ContainerStarted","Data":"292f01a825a96e414509bc7d036fbf0822d5e223c70f480804e810fac1572051"} Jul 30 00:17:19.479756 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:19.479717 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zx7wk" event={"ID":"f5cd0fd1-6297-45d6-9823-cd424bce4185","Type":"ContainerStarted","Data":"f025427c8008cdc10ba65ae6507ff2cddbdb470218912acf9112b27c51d20a04"} Jul 30 00:17:19.480843 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:19.480820 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hglh4" event={"ID":"f95c8416-e10e-4357-96a7-4b48e1c3be4f","Type":"ContainerStarted","Data":"b56e82215e31ab30b2b8490514be63bdd78ad55b78fee0210cbdd113ecc48a32"} Jul 30 00:17:19.481841 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:19.481809 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gp6l9" event={"ID":"756ee6d1-bf62-49db-954a-ed5d66384c54","Type":"ContainerStarted","Data":"0ae6485360b44fb32c85d45f3806a26f0e91369d1d992c50975c45802412d9a9"} Jul 30 00:17:19.482777 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:19.482757 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" event={"ID":"52037ff9-396d-4e0d-958b-ff96d3df4855","Type":"ContainerStarted","Data":"3fa63f0d9bdb6776e6700d389a74310ad5deac843c4ad8e7878acb0e776f4e9a"} Jul 30 00:17:19.483709 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:19.483690 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9m2mr" event={"ID":"ad9e265e-16ad-428a-a69a-f9a7948e5482","Type":"ContainerStarted","Data":"3af873b72ebfab0e6119450b9fa0e23061d731adde56d2cb70c221e5bacc0ed5"} Jul 30 00:17:19.484660 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:19.484641 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zlqlw" event={"ID":"add0c4dc-1cb0-4fa3-903a-6ce3b4a8b60a","Type":"ContainerStarted","Data":"2431b554f0c10c57cd5fa555347e8fb7c60c767d88281550592676a1c123da64"} Jul 30 00:17:19.485563 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:19.485545 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wgvqx" event={"ID":"3806f8c4-b735-4fd6-a240-882dd0c2c70a","Type":"ContainerStarted","Data":"12160bc48cde8de8cbc0e6b294cbd9aacbfd68bf8ccb4b66194f3192f91ee37a"} Jul 30 00:17:19.486497 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:19.486478 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvgfr" event={"ID":"c882608c-cf7a-4e13-a031-193b7c4ac1dc","Type":"ContainerStarted","Data":"5faa5ce6d448a52afcd2b9112461d6872ba65d31b4eaf1b9342b40d5b9970015"} Jul 30 00:17:19.493907 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:19.493867 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-10.ec2.internal" podStartSLOduration=2.4938495720000002 podStartE2EDuration="2.493849572s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 00:17:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:17:19.493445488 +0000 UTC m=+3.552201508" watchObservedRunningTime="2026-07-30 00:17:19.493849572 +0000 UTC m=+3.552605540" Jul 30 00:17:19.965598 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:19.965555 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8567e26f-2898-4d1b-8000-b8bd3326c9ac-metrics-certs\") pod \"network-metrics-daemon-zxg4x\" (UID: \"8567e26f-2898-4d1b-8000-b8bd3326c9ac\") " pod="openshift-multus/network-metrics-daemon-zxg4x" Jul 30 00:17:19.965784 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:19.965612 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxjc4\" (UniqueName: \"kubernetes.io/projected/3101459e-308a-4623-884f-8f54273f1bfd-kube-api-access-qxjc4\") pod \"network-check-target-z7xxn\" (UID: \"3101459e-308a-4623-884f-8f54273f1bfd\") " pod="openshift-network-diagnostics/network-check-target-z7xxn" Jul 30 00:17:19.965784 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:19.965654 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f099a95a-e24f-4e05-bba1-bccd334f000c-original-pull-secret\") pod \"global-pull-secret-syncer-j26vw\" (UID: \"f099a95a-e24f-4e05-bba1-bccd334f000c\") " pod="kube-system/global-pull-secret-syncer-j26vw" Jul 30 00:17:19.965928 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:19.965824 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 30 00:17:19.965928 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:19.965893 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f099a95a-e24f-4e05-bba1-bccd334f000c-original-pull-secret podName:f099a95a-e24f-4e05-bba1-bccd334f000c nodeName:}" failed. No retries permitted until 2026-07-30 00:17:21.965873145 +0000 UTC m=+6.024629095 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f099a95a-e24f-4e05-bba1-bccd334f000c-original-pull-secret") pod "global-pull-secret-syncer-j26vw" (UID: "f099a95a-e24f-4e05-bba1-bccd334f000c") : object "kube-system"/"original-pull-secret" not registered Jul 30 00:17:19.966323 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:19.966299 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 00:17:19.966400 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:19.966350 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8567e26f-2898-4d1b-8000-b8bd3326c9ac-metrics-certs podName:8567e26f-2898-4d1b-8000-b8bd3326c9ac nodeName:}" failed. No retries permitted until 2026-07-30 00:17:21.966335514 +0000 UTC m=+6.025091465 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8567e26f-2898-4d1b-8000-b8bd3326c9ac-metrics-certs") pod "network-metrics-daemon-zxg4x" (UID: "8567e26f-2898-4d1b-8000-b8bd3326c9ac") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 00:17:19.966462 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:19.966426 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 30 00:17:19.966462 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:19.966444 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 30 00:17:19.966462 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:19.966455 2563 projected.go:196] Error preparing data for projected volume kube-api-access-qxjc4 for pod openshift-network-diagnostics/network-check-target-z7xxn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 00:17:19.966600 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:19.966488 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3101459e-308a-4623-884f-8f54273f1bfd-kube-api-access-qxjc4 podName:3101459e-308a-4623-884f-8f54273f1bfd nodeName:}" failed. No retries permitted until 2026-07-30 00:17:21.966477403 +0000 UTC m=+6.025233350 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-qxjc4" (UniqueName: "kubernetes.io/projected/3101459e-308a-4623-884f-8f54273f1bfd-kube-api-access-qxjc4") pod "network-check-target-z7xxn" (UID: "3101459e-308a-4623-884f-8f54273f1bfd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 00:17:20.376652 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:20.376544 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-29 00:12:18 +0000 UTC" deadline="2028-01-05 23:20:02.969899153 +0000 UTC" Jul 30 00:17:20.376652 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:20.376606 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12599h2m42.593297017s" Jul 30 00:17:20.473910 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:20.473151 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j26vw" Jul 30 00:17:20.473910 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:20.473281 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j26vw" podUID="f099a95a-e24f-4e05-bba1-bccd334f000c" Jul 30 00:17:20.473910 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:20.473679 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zxg4x" Jul 30 00:17:20.473910 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:20.473869 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zxg4x" podUID="8567e26f-2898-4d1b-8000-b8bd3326c9ac" Jul 30 00:17:20.474550 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:20.474408 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7xxn" Jul 30 00:17:20.474550 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:20.474500 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7xxn" podUID="3101459e-308a-4623-884f-8f54273f1bfd" Jul 30 00:17:21.985805 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:21.985563 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8567e26f-2898-4d1b-8000-b8bd3326c9ac-metrics-certs\") pod \"network-metrics-daemon-zxg4x\" (UID: \"8567e26f-2898-4d1b-8000-b8bd3326c9ac\") " pod="openshift-multus/network-metrics-daemon-zxg4x" Jul 30 00:17:21.986255 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:21.985826 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxjc4\" (UniqueName: \"kubernetes.io/projected/3101459e-308a-4623-884f-8f54273f1bfd-kube-api-access-qxjc4\") pod \"network-check-target-z7xxn\" (UID: \"3101459e-308a-4623-884f-8f54273f1bfd\") " pod="openshift-network-diagnostics/network-check-target-z7xxn" Jul 30 00:17:21.986255 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:21.985868 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f099a95a-e24f-4e05-bba1-bccd334f000c-original-pull-secret\") pod \"global-pull-secret-syncer-j26vw\" (UID: \"f099a95a-e24f-4e05-bba1-bccd334f000c\") " pod="kube-system/global-pull-secret-syncer-j26vw" Jul 30 00:17:21.986255 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:21.986020 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 30 00:17:21.986255 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:21.986078 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f099a95a-e24f-4e05-bba1-bccd334f000c-original-pull-secret podName:f099a95a-e24f-4e05-bba1-bccd334f000c nodeName:}" failed. No retries permitted until 2026-07-30 00:17:25.986058312 +0000 UTC m=+10.044814260 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f099a95a-e24f-4e05-bba1-bccd334f000c-original-pull-secret") pod "global-pull-secret-syncer-j26vw" (UID: "f099a95a-e24f-4e05-bba1-bccd334f000c") : object "kube-system"/"original-pull-secret" not registered Jul 30 00:17:21.986484 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:21.986466 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 00:17:21.986538 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:21.986511 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8567e26f-2898-4d1b-8000-b8bd3326c9ac-metrics-certs podName:8567e26f-2898-4d1b-8000-b8bd3326c9ac nodeName:}" failed. No retries permitted until 2026-07-30 00:17:25.986496282 +0000 UTC m=+10.045252233 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8567e26f-2898-4d1b-8000-b8bd3326c9ac-metrics-certs") pod "network-metrics-daemon-zxg4x" (UID: "8567e26f-2898-4d1b-8000-b8bd3326c9ac") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 00:17:21.986603 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:21.986585 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 30 00:17:21.986603 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:21.986598 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 30 00:17:21.986695 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:21.986610 2563 projected.go:196] Error preparing data for projected volume kube-api-access-qxjc4 for pod openshift-network-diagnostics/network-check-target-z7xxn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 00:17:21.986695 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:21.986644 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3101459e-308a-4623-884f-8f54273f1bfd-kube-api-access-qxjc4 podName:3101459e-308a-4623-884f-8f54273f1bfd nodeName:}" failed. No retries permitted until 2026-07-30 00:17:25.986633001 +0000 UTC m=+10.045388956 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-qxjc4" (UniqueName: "kubernetes.io/projected/3101459e-308a-4623-884f-8f54273f1bfd-kube-api-access-qxjc4") pod "network-check-target-z7xxn" (UID: "3101459e-308a-4623-884f-8f54273f1bfd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 00:17:22.469838 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:22.469757 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zxg4x" Jul 30 00:17:22.469838 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:22.469800 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7xxn" Jul 30 00:17:22.470057 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:22.469757 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j26vw" Jul 30 00:17:22.470057 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:22.469909 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zxg4x" podUID="8567e26f-2898-4d1b-8000-b8bd3326c9ac" Jul 30 00:17:22.470057 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:22.470001 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j26vw" podUID="f099a95a-e24f-4e05-bba1-bccd334f000c" Jul 30 00:17:22.470220 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:22.470099 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7xxn" podUID="3101459e-308a-4623-884f-8f54273f1bfd" Jul 30 00:17:24.470519 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:24.469943 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zxg4x" Jul 30 00:17:24.470519 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:24.470087 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zxg4x" podUID="8567e26f-2898-4d1b-8000-b8bd3326c9ac" Jul 30 00:17:24.470519 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:24.470449 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j26vw" Jul 30 00:17:24.471217 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:24.471168 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j26vw" podUID="f099a95a-e24f-4e05-bba1-bccd334f000c" Jul 30 00:17:24.471466 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:24.471449 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7xxn" Jul 30 00:17:24.471611 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:24.471593 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7xxn" podUID="3101459e-308a-4623-884f-8f54273f1bfd" Jul 30 00:17:26.022438 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:26.021684 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8567e26f-2898-4d1b-8000-b8bd3326c9ac-metrics-certs\") pod \"network-metrics-daemon-zxg4x\" (UID: \"8567e26f-2898-4d1b-8000-b8bd3326c9ac\") " pod="openshift-multus/network-metrics-daemon-zxg4x" Jul 30 00:17:26.022438 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:26.021730 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxjc4\" (UniqueName: \"kubernetes.io/projected/3101459e-308a-4623-884f-8f54273f1bfd-kube-api-access-qxjc4\") pod \"network-check-target-z7xxn\" (UID: \"3101459e-308a-4623-884f-8f54273f1bfd\") " pod="openshift-network-diagnostics/network-check-target-z7xxn" Jul 30 00:17:26.022438 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:26.021786 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f099a95a-e24f-4e05-bba1-bccd334f000c-original-pull-secret\") pod \"global-pull-secret-syncer-j26vw\" (UID: \"f099a95a-e24f-4e05-bba1-bccd334f000c\") " pod="kube-system/global-pull-secret-syncer-j26vw" Jul 30 00:17:26.022438 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:26.021922 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 30 00:17:26.022438 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:26.021983 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f099a95a-e24f-4e05-bba1-bccd334f000c-original-pull-secret podName:f099a95a-e24f-4e05-bba1-bccd334f000c nodeName:}" failed. No retries permitted until 2026-07-30 00:17:34.021966051 +0000 UTC m=+18.080721999 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f099a95a-e24f-4e05-bba1-bccd334f000c-original-pull-secret") pod "global-pull-secret-syncer-j26vw" (UID: "f099a95a-e24f-4e05-bba1-bccd334f000c") : object "kube-system"/"original-pull-secret" not registered Jul 30 00:17:26.022438 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:26.022361 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 00:17:26.022438 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:26.022409 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8567e26f-2898-4d1b-8000-b8bd3326c9ac-metrics-certs podName:8567e26f-2898-4d1b-8000-b8bd3326c9ac nodeName:}" failed. No retries permitted until 2026-07-30 00:17:34.022393899 +0000 UTC m=+18.081149845 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8567e26f-2898-4d1b-8000-b8bd3326c9ac-metrics-certs") pod "network-metrics-daemon-zxg4x" (UID: "8567e26f-2898-4d1b-8000-b8bd3326c9ac") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 00:17:26.023215 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:26.022487 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 30 00:17:26.023215 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:26.022502 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 30 00:17:26.023215 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:26.022514 2563 projected.go:196] Error preparing data for projected volume kube-api-access-qxjc4 for pod openshift-network-diagnostics/network-check-target-z7xxn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 00:17:26.023215 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:26.022556 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3101459e-308a-4623-884f-8f54273f1bfd-kube-api-access-qxjc4 podName:3101459e-308a-4623-884f-8f54273f1bfd nodeName:}" failed. No retries permitted until 2026-07-30 00:17:34.022544062 +0000 UTC m=+18.081300022 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-qxjc4" (UniqueName: "kubernetes.io/projected/3101459e-308a-4623-884f-8f54273f1bfd-kube-api-access-qxjc4") pod "network-check-target-z7xxn" (UID: "3101459e-308a-4623-884f-8f54273f1bfd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 00:17:26.470978 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:26.470459 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j26vw" Jul 30 00:17:26.470978 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:26.470555 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zxg4x" Jul 30 00:17:26.470978 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:26.470574 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j26vw" podUID="f099a95a-e24f-4e05-bba1-bccd334f000c" Jul 30 00:17:26.470978 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:26.470712 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zxg4x" podUID="8567e26f-2898-4d1b-8000-b8bd3326c9ac" Jul 30 00:17:26.470978 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:26.470788 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7xxn" Jul 30 00:17:26.470978 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:26.470877 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7xxn" podUID="3101459e-308a-4623-884f-8f54273f1bfd" Jul 30 00:17:28.469789 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:28.469759 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7xxn" Jul 30 00:17:28.470255 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:28.469888 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7xxn" podUID="3101459e-308a-4623-884f-8f54273f1bfd" Jul 30 00:17:28.470320 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:28.470275 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j26vw" Jul 30 00:17:28.470410 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:28.470372 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j26vw" podUID="f099a95a-e24f-4e05-bba1-bccd334f000c" Jul 30 00:17:28.470540 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:28.470443 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zxg4x" Jul 30 00:17:28.470540 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:28.470529 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zxg4x" podUID="8567e26f-2898-4d1b-8000-b8bd3326c9ac" Jul 30 00:17:30.469770 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:30.469723 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j26vw" Jul 30 00:17:30.470177 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:30.469782 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zxg4x" Jul 30 00:17:30.470177 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:30.469876 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j26vw" podUID="f099a95a-e24f-4e05-bba1-bccd334f000c" Jul 30 00:17:30.470177 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:30.469908 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7xxn" Jul 30 00:17:30.470177 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:30.470011 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zxg4x" podUID="8567e26f-2898-4d1b-8000-b8bd3326c9ac" Jul 30 00:17:30.470177 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:30.470099 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7xxn" podUID="3101459e-308a-4623-884f-8f54273f1bfd" Jul 30 00:17:32.469240 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:32.469159 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j26vw" Jul 30 00:17:32.469240 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:32.469195 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7xxn" Jul 30 00:17:32.469810 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:32.469165 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zxg4x" Jul 30 00:17:32.469810 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:32.469301 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j26vw" podUID="f099a95a-e24f-4e05-bba1-bccd334f000c" Jul 30 00:17:32.469810 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:32.469376 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zxg4x" podUID="8567e26f-2898-4d1b-8000-b8bd3326c9ac" Jul 30 00:17:32.469810 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:32.469452 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7xxn" podUID="3101459e-308a-4623-884f-8f54273f1bfd" Jul 30 00:17:34.082663 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:34.082569 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8567e26f-2898-4d1b-8000-b8bd3326c9ac-metrics-certs\") pod \"network-metrics-daemon-zxg4x\" (UID: \"8567e26f-2898-4d1b-8000-b8bd3326c9ac\") " pod="openshift-multus/network-metrics-daemon-zxg4x" Jul 30 00:17:34.082663 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:34.082623 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxjc4\" (UniqueName: \"kubernetes.io/projected/3101459e-308a-4623-884f-8f54273f1bfd-kube-api-access-qxjc4\") pod \"network-check-target-z7xxn\" (UID: \"3101459e-308a-4623-884f-8f54273f1bfd\") " pod="openshift-network-diagnostics/network-check-target-z7xxn" Jul 30 00:17:34.082663 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:34.082660 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f099a95a-e24f-4e05-bba1-bccd334f000c-original-pull-secret\") pod \"global-pull-secret-syncer-j26vw\" (UID: \"f099a95a-e24f-4e05-bba1-bccd334f000c\") " pod="kube-system/global-pull-secret-syncer-j26vw" Jul 30 00:17:34.083200 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:34.082751 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 00:17:34.083200 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:34.082803 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 30 00:17:34.083200 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:34.082825 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8567e26f-2898-4d1b-8000-b8bd3326c9ac-metrics-certs podName:8567e26f-2898-4d1b-8000-b8bd3326c9ac nodeName:}" failed. No retries permitted until 2026-07-30 00:17:50.082804275 +0000 UTC m=+34.141560225 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8567e26f-2898-4d1b-8000-b8bd3326c9ac-metrics-certs") pod "network-metrics-daemon-zxg4x" (UID: "8567e26f-2898-4d1b-8000-b8bd3326c9ac") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 00:17:34.083200 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:34.082857 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f099a95a-e24f-4e05-bba1-bccd334f000c-original-pull-secret podName:f099a95a-e24f-4e05-bba1-bccd334f000c nodeName:}" failed. No retries permitted until 2026-07-30 00:17:50.082840893 +0000 UTC m=+34.141596841 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f099a95a-e24f-4e05-bba1-bccd334f000c-original-pull-secret") pod "global-pull-secret-syncer-j26vw" (UID: "f099a95a-e24f-4e05-bba1-bccd334f000c") : object "kube-system"/"original-pull-secret" not registered Jul 30 00:17:34.083200 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:34.082937 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 30 00:17:34.083200 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:34.082950 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 30 00:17:34.083200 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:34.082963 2563 projected.go:196] Error preparing data for projected volume kube-api-access-qxjc4 for pod openshift-network-diagnostics/network-check-target-z7xxn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 00:17:34.083200 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:34.082995 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3101459e-308a-4623-884f-8f54273f1bfd-kube-api-access-qxjc4 podName:3101459e-308a-4623-884f-8f54273f1bfd nodeName:}" failed. No retries permitted until 2026-07-30 00:17:50.08298491 +0000 UTC m=+34.141740861 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-qxjc4" (UniqueName: "kubernetes.io/projected/3101459e-308a-4623-884f-8f54273f1bfd-kube-api-access-qxjc4") pod "network-check-target-z7xxn" (UID: "3101459e-308a-4623-884f-8f54273f1bfd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 00:17:34.469840 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:34.469748 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zxg4x" Jul 30 00:17:34.469840 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:34.469778 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7xxn" Jul 30 00:17:34.470072 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:34.469748 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j26vw" Jul 30 00:17:34.470072 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:34.469891 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zxg4x" podUID="8567e26f-2898-4d1b-8000-b8bd3326c9ac" Jul 30 00:17:34.470072 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:34.469986 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j26vw" podUID="f099a95a-e24f-4e05-bba1-bccd334f000c" Jul 30 00:17:34.470072 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:34.470059 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7xxn" podUID="3101459e-308a-4623-884f-8f54273f1bfd" Jul 30 00:17:36.471552 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:36.471230 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j26vw" Jul 30 00:17:36.471552 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:36.471399 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j26vw" podUID="f099a95a-e24f-4e05-bba1-bccd334f000c" Jul 30 00:17:36.472090 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:36.471867 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7xxn" Jul 30 00:17:36.472090 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:36.471988 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7xxn" podUID="3101459e-308a-4623-884f-8f54273f1bfd" Jul 30 00:17:36.472090 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:36.472046 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zxg4x" Jul 30 00:17:36.472217 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:36.472161 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zxg4x" podUID="8567e26f-2898-4d1b-8000-b8bd3326c9ac" Jul 30 00:17:38.470134 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:38.469973 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j26vw" Jul 30 00:17:38.470776 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:38.470003 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zxg4x" Jul 30 00:17:38.470776 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:38.470219 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j26vw" podUID="f099a95a-e24f-4e05-bba1-bccd334f000c" Jul 30 00:17:38.470776 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:38.470069 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7xxn" Jul 30 00:17:38.470776 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:38.470326 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zxg4x" podUID="8567e26f-2898-4d1b-8000-b8bd3326c9ac" Jul 30 00:17:38.470776 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:38.470360 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7xxn" podUID="3101459e-308a-4623-884f-8f54273f1bfd" Jul 30 00:17:38.530537 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:38.530503 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gp6l9" event={"ID":"756ee6d1-bf62-49db-954a-ed5d66384c54","Type":"ContainerStarted","Data":"e266e92cc5c19d8048feff129ceb303d4b35e0c8bc445a0ba0ac8f11d11bc91c"} Jul 30 00:17:38.533008 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:38.532983 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" event={"ID":"52037ff9-396d-4e0d-958b-ff96d3df4855","Type":"ContainerStarted","Data":"e4300cfe24570ae2e80f3ae8a10bd7091578dbb073cdd2e7a783802fcf73fa88"} Jul 30 00:17:38.533109 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:38.533049 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" event={"ID":"52037ff9-396d-4e0d-958b-ff96d3df4855","Type":"ContainerStarted","Data":"9eae4f7a57b4f4e81a68d3ef076d903718eace47e3b5209f86fe6c07f0824bfb"} Jul 30 00:17:38.533109 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:38.533085 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" event={"ID":"52037ff9-396d-4e0d-958b-ff96d3df4855","Type":"ContainerStarted","Data":"1797d4968b2656aacc4b9d0ec0cac110bb0c138c19e5616043aa6fd9813e3e7e"} Jul 30 00:17:38.533192 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:38.533121 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" event={"ID":"52037ff9-396d-4e0d-958b-ff96d3df4855","Type":"ContainerStarted","Data":"255ed0307d876dffb3146c1d4156dae29f973e4be4bf66c7d0a72d03a6396fa2"} Jul 30 00:17:38.533192 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:38.533156 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" event={"ID":"52037ff9-396d-4e0d-958b-ff96d3df4855","Type":"ContainerStarted","Data":"63ea76fe2b7690fa78552119267179c49b85b3739965f3abe6b3742cfca67b1b"} Jul 30 00:17:38.534371 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:38.534342 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9m2mr" event={"ID":"ad9e265e-16ad-428a-a69a-f9a7948e5482","Type":"ContainerStarted","Data":"5b123d3ee5b2a75ad211281243affa742d832c9d82e117d5b293441a4d5e5a85"} Jul 30 00:17:38.535622 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:38.535603 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zlqlw" event={"ID":"add0c4dc-1cb0-4fa3-903a-6ce3b4a8b60a","Type":"ContainerStarted","Data":"148df8082d1083cc92b5591b934300296e5a998d23105c8b325cee401712b21c"} Jul 30 00:17:38.536834 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:38.536816 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvgfr" event={"ID":"c882608c-cf7a-4e13-a031-193b7c4ac1dc","Type":"ContainerStarted","Data":"24bf99fa10d54981be67af177576f259d5d2253b6768beb179c0925093ab556b"} Jul 30 00:17:38.538167 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:38.538136 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6wfs6" event={"ID":"b235c4f3-5e9b-4866-99c2-41ec0a64f806","Type":"ContainerStarted","Data":"84e8ccbd954c58126e1181a73ea16df0058ab2173ef84a42d947780140709505"} Jul 30 00:17:38.539684 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:38.539662 2563 generic.go:356] "Generic (PLEG): container finished" podID="f5cd0fd1-6297-45d6-9823-cd424bce4185" containerID="2ce1839642a7de1732b344b76a11a7443a90c8becb2cf2d8033f5f0bf4313e22" exitCode=0 Jul 30 00:17:38.539867 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:38.539724 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zx7wk" event={"ID":"f5cd0fd1-6297-45d6-9823-cd424bce4185","Type":"ContainerDied","Data":"2ce1839642a7de1732b344b76a11a7443a90c8becb2cf2d8033f5f0bf4313e22"} Jul 30 00:17:38.541305 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:38.541271 2563 generic.go:356] "Generic (PLEG): container finished" podID="2eeefd595358be06c0e6dd4416315e17" containerID="c4b5a149864906d583e582cc0aaac380fa80daa63e23b584633a050c6ace696a" exitCode=0 Jul 30 00:17:38.541387 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:38.541335 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal" event={"ID":"2eeefd595358be06c0e6dd4416315e17","Type":"ContainerDied","Data":"c4b5a149864906d583e582cc0aaac380fa80daa63e23b584633a050c6ace696a"} Jul 30 00:17:38.542791 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:38.542772 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hglh4" event={"ID":"f95c8416-e10e-4357-96a7-4b48e1c3be4f","Type":"ContainerStarted","Data":"25a9d1cd07080a5066a60ff48266bb974e085f838c1fb164b1e8f2c5a68806fa"} Jul 30 00:17:38.548138 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:38.548103 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-gp6l9" podStartSLOduration=3.822283282 podStartE2EDuration="22.548092979s" totalImagesPullingTime="18.725809697s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 00:17:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:17:38.547923316 +0000 UTC m=+22.606679276" watchObservedRunningTime="2026-07-30 00:17:38.548092979 +0000 UTC m=+22.606848947" Jul 30 00:17:38.581694 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:38.581654 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-6wfs6" podStartSLOduration=3.837640755 podStartE2EDuration="22.581639013s" totalImagesPullingTime="18.743998258s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 00:17:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:17:38.581538938 +0000 UTC m=+22.640294908" watchObservedRunningTime="2026-07-30 00:17:38.581639013 +0000 UTC m=+22.640394981" Jul 30 00:17:38.581829 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:38.581764 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-zlqlw" podStartSLOduration=3.849210074 podStartE2EDuration="22.581759563s" totalImagesPullingTime="18.732549489s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 00:17:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:17:38.565924842 +0000 UTC m=+22.624680810" watchObservedRunningTime="2026-07-30 00:17:38.581759563 +0000 UTC m=+22.640515532" Jul 30 00:17:38.650583 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:38.650425 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-hglh4" podStartSLOduration=3.874322932 podStartE2EDuration="22.650402578s" totalImagesPullingTime="18.776079646s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 00:17:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:17:38.64879813 +0000 UTC m=+22.707554098" watchObservedRunningTime="2026-07-30 00:17:38.650402578 +0000 UTC m=+22.709158550" Jul 30 00:17:38.664483 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:38.664431 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-9m2mr" podStartSLOduration=8.197735368 podStartE2EDuration="22.664410378s" totalImagesPullingTime="14.46667501s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:17:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:17:38.664124461 +0000 UTC m=+22.722880434" watchObservedRunningTime="2026-07-30 00:17:38.664410378 +0000 UTC m=+22.723166348" Jul 30 00:17:39.525730 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:39.525704 2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 30 00:17:39.546533 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:39.546503 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wgvqx" event={"ID":"3806f8c4-b735-4fd6-a240-882dd0c2c70a","Type":"ContainerStarted","Data":"6e4993a39166f36b2c4addf446c0d312290bdb95d83d99d275e6080f185c2012"} Jul 30 00:17:39.548211 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:39.548186 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvgfr" event={"ID":"c882608c-cf7a-4e13-a031-193b7c4ac1dc","Type":"ContainerStarted","Data":"e0cf49abcff4e283234bd63ab713b19dda0c9e389b755f1f18d66dab56244f0a"} Jul 30 00:17:39.549865 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:39.549844 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal" event={"ID":"2eeefd595358be06c0e6dd4416315e17","Type":"ContainerStarted","Data":"81fdc9c578fa64edc217e849cb287bc657ea431bc07acab264b87787d0b825f3"} Jul 30 00:17:39.552689 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:39.552662 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" event={"ID":"52037ff9-396d-4e0d-958b-ff96d3df4855","Type":"ContainerStarted","Data":"c3bb984719783d9156ab0b38f4798a46bb578e37d1fd87f67b7e2cc054aca4da"} Jul 30 00:17:39.560230 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:39.560194 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-wgvqx" podStartSLOduration=9.089263108 podStartE2EDuration="23.560180737s" totalImagesPullingTime="14.470917629s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 00:17:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:17:39.560178995 +0000 UTC m=+23.618934963" watchObservedRunningTime="2026-07-30 00:17:39.560180737 +0000 UTC m=+23.618936705" Jul 30 00:17:39.573541 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:39.573506 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal" podStartSLOduration=22.57349204 podStartE2EDuration="22.57349204s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 00:17:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:17:39.573057607 +0000 UTC m=+23.631813574" watchObservedRunningTime="2026-07-30 00:17:39.57349204 +0000 UTC m=+23.632248007" Jul 30 00:17:40.396877 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:40.396759 2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-30T00:17:39.525723784Z","UUID":"15ce0620-1496-42a8-bf04-a4e1e968ff32","Handler":null,"Name":"","Endpoint":""} Jul 30 00:17:40.398475 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:40.398451 2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 30 00:17:40.398613 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:40.398496 2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 30 00:17:40.469594 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:40.469565 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zxg4x" Jul 30 00:17:40.469788 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:40.469602 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j26vw" Jul 30 00:17:40.469788 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:40.469565 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7xxn" Jul 30 00:17:40.469788 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:40.469698 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zxg4x" podUID="8567e26f-2898-4d1b-8000-b8bd3326c9ac" Jul 30 00:17:40.469788 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:40.469783 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7xxn" podUID="3101459e-308a-4623-884f-8f54273f1bfd" Jul 30 00:17:40.470072 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:40.469852 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j26vw" podUID="f099a95a-e24f-4e05-bba1-bccd334f000c" Jul 30 00:17:40.743782 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:40.743756 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-9m2mr" Jul 30 00:17:40.744937 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:40.744908 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-9m2mr" Jul 30 00:17:41.561406 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:41.561365 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" event={"ID":"52037ff9-396d-4e0d-958b-ff96d3df4855","Type":"ContainerStarted","Data":"99f3ccee99bff9e3a7cc1677d49783dd9f554635e51c2b6ba5f954b0e0352aa2"} Jul 30 00:17:41.563440 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:41.563407 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvgfr" event={"ID":"c882608c-cf7a-4e13-a031-193b7c4ac1dc","Type":"ContainerStarted","Data":"321fd25c5f997708e8b5e9da49deba8d29fa651595f2388432e1ed0526772b8b"} Jul 30 00:17:41.563711 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:41.563683 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-9m2mr" Jul 30 00:17:41.564250 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:41.564234 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-9m2mr" Jul 30 00:17:41.580576 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:41.580530 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvgfr" podStartSLOduration=4.524534213 podStartE2EDuration="25.58051579s" totalImagesPullingTime="21.055981577s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 00:17:16 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:17:41.580384021 +0000 UTC m=+25.639140046" watchObservedRunningTime="2026-07-30 00:17:41.58051579 +0000 UTC m=+25.639271758" Jul 30 00:17:42.469676 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:42.469646 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j26vw" Jul 30 00:17:42.470136 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:42.469649 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7xxn" Jul 30 00:17:42.470136 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:42.469781 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zxg4x" Jul 30 00:17:42.470136 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:42.469773 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j26vw" podUID="f099a95a-e24f-4e05-bba1-bccd334f000c" Jul 30 00:17:42.470136 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:42.469919 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7xxn" podUID="3101459e-308a-4623-884f-8f54273f1bfd" Jul 30 00:17:42.470136 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:42.470010 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zxg4x" podUID="8567e26f-2898-4d1b-8000-b8bd3326c9ac" Jul 30 00:17:44.470315 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:44.470102 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j26vw" Jul 30 00:17:44.471008 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:44.470102 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zxg4x" Jul 30 00:17:44.471008 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:44.470361 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j26vw" podUID="f099a95a-e24f-4e05-bba1-bccd334f000c" Jul 30 00:17:44.471008 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:44.470453 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zxg4x" podUID="8567e26f-2898-4d1b-8000-b8bd3326c9ac" Jul 30 00:17:44.471008 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:44.470155 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7xxn" Jul 30 00:17:44.471008 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:44.470574 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7xxn" podUID="3101459e-308a-4623-884f-8f54273f1bfd" Jul 30 00:17:44.573412 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:44.573375 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" event={"ID":"52037ff9-396d-4e0d-958b-ff96d3df4855","Type":"ContainerStarted","Data":"b05dd76e8101e8a5367f15cb1a170c2e7df04799ae11d34413e7ef65dc7c957c"} Jul 30 00:17:44.573628 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:44.573608 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:44.573697 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:44.573680 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:44.575226 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:44.575202 2563 generic.go:356] "Generic (PLEG): container finished" podID="f5cd0fd1-6297-45d6-9823-cd424bce4185" containerID="5c4134aa09f1dca462baa2fc594ff9062ccefdb2bd1456e620e6d85f55cd8476" exitCode=0 Jul 30 00:17:44.575346 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:44.575231 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zx7wk" event={"ID":"f5cd0fd1-6297-45d6-9823-cd424bce4185","Type":"ContainerDied","Data":"5c4134aa09f1dca462baa2fc594ff9062ccefdb2bd1456e620e6d85f55cd8476"} Jul 30 00:17:44.589353 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:44.589331 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:44.605392 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:44.605349 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" podStartSLOduration=9.810946074 podStartE2EDuration="28.605335835s" totalImagesPullingTime="18.794389761s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 00:17:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:17:44.604031313 +0000 UTC m=+28.662787281" watchObservedRunningTime="2026-07-30 00:17:44.605335835 +0000 UTC m=+28.664091803" Jul 30 00:17:45.580508 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:45.580055 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:45.594834 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:45.594808 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:17:45.986726 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:45.986619 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-j26vw"] Jul 30 00:17:45.986888 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:45.986758 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j26vw" Jul 30 00:17:45.986888 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:45.986862 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j26vw" podUID="f099a95a-e24f-4e05-bba1-bccd334f000c" Jul 30 00:17:45.990483 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:45.990455 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-z7xxn"] Jul 30 00:17:45.990617 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:45.990570 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7xxn" Jul 30 00:17:45.990695 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:45.990672 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7xxn" podUID="3101459e-308a-4623-884f-8f54273f1bfd" Jul 30 00:17:45.991111 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:45.991085 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zxg4x"] Jul 30 00:17:45.991226 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:45.991186 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zxg4x" Jul 30 00:17:45.991311 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:45.991289 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zxg4x" podUID="8567e26f-2898-4d1b-8000-b8bd3326c9ac" Jul 30 00:17:46.582453 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:46.582418 2563 generic.go:356] "Generic (PLEG): container finished" podID="f5cd0fd1-6297-45d6-9823-cd424bce4185" containerID="28de7ae720ba8139efc2d718af6efa184ff0fd003ad026620729480b03f546b4" exitCode=0 Jul 30 00:17:46.582839 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:46.582481 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zx7wk" event={"ID":"f5cd0fd1-6297-45d6-9823-cd424bce4185","Type":"ContainerDied","Data":"28de7ae720ba8139efc2d718af6efa184ff0fd003ad026620729480b03f546b4"} Jul 30 00:17:47.469741 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:47.469711 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7xxn" Jul 30 00:17:47.469907 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:47.469726 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zxg4x" Jul 30 00:17:47.469907 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:47.469728 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j26vw" Jul 30 00:17:47.469907 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:47.469833 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7xxn" podUID="3101459e-308a-4623-884f-8f54273f1bfd" Jul 30 00:17:47.470018 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:47.469983 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zxg4x" podUID="8567e26f-2898-4d1b-8000-b8bd3326c9ac" Jul 30 00:17:47.470088 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:47.470069 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j26vw" podUID="f099a95a-e24f-4e05-bba1-bccd334f000c" Jul 30 00:17:48.588666 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:48.588633 2563 generic.go:356] "Generic (PLEG): container finished" podID="f5cd0fd1-6297-45d6-9823-cd424bce4185" containerID="ddaceabd5e9bd9425ff7f1d12b20fe9cf4f6e6c92f91fad0f8c646a674875b6b" exitCode=0 Jul 30 00:17:48.589053 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:48.588675 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zx7wk" event={"ID":"f5cd0fd1-6297-45d6-9823-cd424bce4185","Type":"ContainerDied","Data":"ddaceabd5e9bd9425ff7f1d12b20fe9cf4f6e6c92f91fad0f8c646a674875b6b"} Jul 30 00:17:49.469471 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:49.469392 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zxg4x" Jul 30 00:17:49.469471 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:49.469408 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j26vw" Jul 30 00:17:49.469471 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:49.469415 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7xxn" Jul 30 00:17:49.469669 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:49.469494 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j26vw" podUID="f099a95a-e24f-4e05-bba1-bccd334f000c" Jul 30 00:17:49.469669 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:49.469549 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7xxn" podUID="3101459e-308a-4623-884f-8f54273f1bfd" Jul 30 00:17:49.469669 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:49.469634 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zxg4x" podUID="8567e26f-2898-4d1b-8000-b8bd3326c9ac" Jul 30 00:17:50.095468 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.095431 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8567e26f-2898-4d1b-8000-b8bd3326c9ac-metrics-certs\") pod \"network-metrics-daemon-zxg4x\" (UID: \"8567e26f-2898-4d1b-8000-b8bd3326c9ac\") " pod="openshift-multus/network-metrics-daemon-zxg4x" Jul 30 00:17:50.095468 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.095468 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxjc4\" (UniqueName: \"kubernetes.io/projected/3101459e-308a-4623-884f-8f54273f1bfd-kube-api-access-qxjc4\") pod \"network-check-target-z7xxn\" (UID: \"3101459e-308a-4623-884f-8f54273f1bfd\") " pod="openshift-network-diagnostics/network-check-target-z7xxn" Jul 30 00:17:50.095966 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:50.095595 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 00:17:50.095966 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:50.095609 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 30 00:17:50.095966 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:50.095634 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 30 00:17:50.095966 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:50.095647 2563 projected.go:196] Error preparing data for projected volume kube-api-access-qxjc4 for pod openshift-network-diagnostics/network-check-target-z7xxn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 00:17:50.095966 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.095595 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f099a95a-e24f-4e05-bba1-bccd334f000c-original-pull-secret\") pod \"global-pull-secret-syncer-j26vw\" (UID: \"f099a95a-e24f-4e05-bba1-bccd334f000c\") " pod="kube-system/global-pull-secret-syncer-j26vw" Jul 30 00:17:50.095966 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:50.095668 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 30 00:17:50.095966 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:50.095671 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8567e26f-2898-4d1b-8000-b8bd3326c9ac-metrics-certs podName:8567e26f-2898-4d1b-8000-b8bd3326c9ac nodeName:}" failed. No retries permitted until 2026-07-30 00:18:22.095651263 +0000 UTC m=+66.154407222 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8567e26f-2898-4d1b-8000-b8bd3326c9ac-metrics-certs") pod "network-metrics-daemon-zxg4x" (UID: "8567e26f-2898-4d1b-8000-b8bd3326c9ac") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 00:17:50.095966 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:50.095748 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3101459e-308a-4623-884f-8f54273f1bfd-kube-api-access-qxjc4 podName:3101459e-308a-4623-884f-8f54273f1bfd nodeName:}" failed. No retries permitted until 2026-07-30 00:18:22.095714627 +0000 UTC m=+66.154470586 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-qxjc4" (UniqueName: "kubernetes.io/projected/3101459e-308a-4623-884f-8f54273f1bfd-kube-api-access-qxjc4") pod "network-check-target-z7xxn" (UID: "3101459e-308a-4623-884f-8f54273f1bfd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 00:17:50.095966 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:50.095766 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f099a95a-e24f-4e05-bba1-bccd334f000c-original-pull-secret podName:f099a95a-e24f-4e05-bba1-bccd334f000c nodeName:}" failed. No retries permitted until 2026-07-30 00:18:22.095755354 +0000 UTC m=+66.154511300 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f099a95a-e24f-4e05-bba1-bccd334f000c-original-pull-secret") pod "global-pull-secret-syncer-j26vw" (UID: "f099a95a-e24f-4e05-bba1-bccd334f000c") : object "kube-system"/"original-pull-secret" not registered Jul 30 00:17:50.776230 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.776154 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeReady" Jul 30 00:17:50.776394 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.776315 2563 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 30 00:17:50.809795 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.809729 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-ddm85"] Jul 30 00:17:50.811658 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.811637 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-ddm85" Jul 30 00:17:50.813541 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.813508 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-phzr9"] Jul 30 00:17:50.814291 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.814252 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 30 00:17:50.814474 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.814452 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 30 00:17:50.814572 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.814485 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-5bw4s\"" Jul 30 00:17:50.814763 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.814680 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 30 00:17:50.815162 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.814969 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 30 00:17:50.815322 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.815302 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-596d7786c9-nrjxz"] Jul 30 00:17:50.815622 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.815595 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-phzr9" Jul 30 00:17:50.816874 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.816856 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-877bf98d5-xbkfd"] Jul 30 00:17:50.817107 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.817091 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-596d7786c9-nrjxz" Jul 30 00:17:50.818382 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.818363 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c9vdq"] Jul 30 00:17:50.818536 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.818487 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-877bf98d5-xbkfd" Jul 30 00:17:50.819691 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.819548 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 30 00:17:50.819691 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.819606 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 30 00:17:50.820055 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.820034 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-ndhmg\"" Jul 30 00:17:50.820894 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.820871 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 30 00:17:50.821137 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.821119 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 30 00:17:50.821456 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.821185 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 30 00:17:50.821456 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.821185 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 30 00:17:50.821557 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.821505 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 30 00:17:50.821852 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.821827 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 30 00:17:50.822237 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.822136 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-b92tw"] Jul 30 00:17:50.822322 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.822301 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c9vdq" Jul 30 00:17:50.822752 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.822720 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-nspqp\"" Jul 30 00:17:50.823455 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.823434 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-s7c9v\"" Jul 30 00:17:50.824722 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.824135 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-b92tw" Jul 30 00:17:50.824722 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.824183 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 30 00:17:50.824722 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.824275 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 30 00:17:50.824722 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.824375 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 30 00:17:50.824722 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.824645 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 30 00:17:50.825540 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.825521 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 30 00:17:50.825866 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.825850 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 30 00:17:50.826170 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.826155 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 30 00:17:50.826593 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.826577 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-b6cph\"" Jul 30 00:17:50.826834 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.826816 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 30 00:17:50.826916 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.826841 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 30 00:17:50.827331 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.827155 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-vq6z5\"" Jul 30 00:17:50.827331 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.827167 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 30 00:17:50.827331 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.827178 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 30 00:17:50.827331 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.827204 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 30 00:17:50.827567 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.827538 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 30 00:17:50.827931 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.827914 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 30 00:17:50.829707 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.829690 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 30 00:17:50.837519 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.837439 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-ddm85"] Jul 30 00:17:50.837655 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.837552 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c9vdq"] Jul 30 00:17:50.838956 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.838931 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-phzr9"] Jul 30 00:17:50.847394 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.846995 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-6kh2h"] Jul 30 00:17:50.847522 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.847416 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 30 00:17:50.858816 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.853157 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-j2fkq"] Jul 30 00:17:50.858816 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.853323 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6kh2h" Jul 30 00:17:50.858816 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.855850 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-xjl58"] Jul 30 00:17:50.858816 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.856760 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-j2fkq" Jul 30 00:17:50.858816 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.858224 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-ggg5l"] Jul 30 00:17:50.859348 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.859329 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-xjl58" Jul 30 00:17:50.860779 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.859966 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 30 00:17:50.860779 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.860198 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-twxtz\"" Jul 30 00:17:50.860779 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.860456 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 30 00:17:50.860779 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.860485 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 30 00:17:50.861081 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.860879 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 30 00:17:50.861081 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.860940 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-nxhqt\"" Jul 30 00:17:50.861187 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.861107 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-8sgzj"] Jul 30 00:17:50.861303 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.861241 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-ggg5l" Jul 30 00:17:50.862703 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.862652 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 30 00:17:50.863330 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.862942 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 30 00:17:50.863330 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.862980 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 30 00:17:50.863330 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.863075 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 30 00:17:50.863330 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.863140 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-c7m7k\"" Jul 30 00:17:50.863643 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.863367 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-qg9tq"] Jul 30 00:17:50.863643 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.863519 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-8sgzj" Jul 30 00:17:50.863790 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.863685 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 30 00:17:50.864012 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.863993 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 30 00:17:50.864012 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.864010 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-l669n\"" Jul 30 00:17:50.864133 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.864089 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 30 00:17:50.865862 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.865844 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 30 00:17:50.866421 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.866110 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 30 00:17:50.866421 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.866342 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-b8mkt\"" Jul 30 00:17:50.866854 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.866833 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-tx22w"] Jul 30 00:17:50.867814 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.867200 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qg9tq" Jul 30 00:17:50.869173 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.869155 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-877bf98d5-xbkfd"] Jul 30 00:17:50.869338 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.869326 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-596d7786c9-nrjxz"] Jul 30 00:17:50.869462 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.869450 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-j2fkq"] Jul 30 00:17:50.869581 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.869569 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-xjl58"] Jul 30 00:17:50.869722 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.869685 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-8sgzj"] Jul 30 00:17:50.869839 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.869756 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-ggg5l"] Jul 30 00:17:50.869839 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.869791 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qg9tq"] Jul 30 00:17:50.869839 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.869809 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6kh2h"] Jul 30 00:17:50.869839 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.869838 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-tx22w"] Jul 30 00:17:50.870031 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.869855 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-b92tw"] Jul 30 00:17:50.870031 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.869254 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-tx22w" Jul 30 00:17:50.870031 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.869937 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 30 00:17:50.870163 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.870126 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 30 00:17:50.870353 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.869386 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 30 00:17:50.870442 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.870432 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-qvszd\"" Jul 30 00:17:50.871967 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.871948 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 30 00:17:50.872266 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.872249 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 30 00:17:50.872351 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.872333 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-976dl\"" Jul 30 00:17:50.901652 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.901627 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90afa1c1-9879-49ee-9fac-27f2e7e963f3-service-ca-bundle\") pod \"router-default-877bf98d5-xbkfd\" (UID: \"90afa1c1-9879-49ee-9fac-27f2e7e963f3\") " pod="openshift-ingress/router-default-877bf98d5-xbkfd" Jul 30 00:17:50.901783 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.901692 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f7d33879-e015-4cfe-8d19-11cbc98c1de2-registry-certificates\") pod \"image-registry-596d7786c9-nrjxz\" (UID: \"f7d33879-e015-4cfe-8d19-11cbc98c1de2\") " pod="openshift-image-registry/image-registry-596d7786c9-nrjxz" Jul 30 00:17:50.901783 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.901724 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7d591175-5d66-48f9-be3e-c89d48534295-serving-cert\") pod \"service-ca-operator-5d6c75f84f-c9vdq\" (UID: \"7d591175-5d66-48f9-be3e-c89d48534295\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c9vdq" Jul 30 00:17:50.901783 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.901770 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f7d33879-e015-4cfe-8d19-11cbc98c1de2-registry-tls\") pod \"image-registry-596d7786c9-nrjxz\" (UID: \"f7d33879-e015-4cfe-8d19-11cbc98c1de2\") " pod="openshift-image-registry/image-registry-596d7786c9-nrjxz" Jul 30 00:17:50.901933 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.901814 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7d591175-5d66-48f9-be3e-c89d48534295-config\") pod \"service-ca-operator-5d6c75f84f-c9vdq\" (UID: \"7d591175-5d66-48f9-be3e-c89d48534295\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c9vdq" Jul 30 00:17:50.901933 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.901884 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/90afa1c1-9879-49ee-9fac-27f2e7e963f3-stats-auth\") pod \"router-default-877bf98d5-xbkfd\" (UID: \"90afa1c1-9879-49ee-9fac-27f2e7e963f3\") " pod="openshift-ingress/router-default-877bf98d5-xbkfd" Jul 30 00:17:50.901933 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.901917 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f7d33879-e015-4cfe-8d19-11cbc98c1de2-trusted-ca\") pod \"image-registry-596d7786c9-nrjxz\" (UID: \"f7d33879-e015-4cfe-8d19-11cbc98c1de2\") " pod="openshift-image-registry/image-registry-596d7786c9-nrjxz" Jul 30 00:17:50.902041 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.901961 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fa91894-839d-4035-8b46-a787fe957864-service-ca-bundle\") pod \"insights-operator-5885fdb759-ddm85\" (UID: \"3fa91894-839d-4035-8b46-a787fe957864\") " pod="openshift-insights/insights-operator-5885fdb759-ddm85" Jul 30 00:17:50.902041 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.902002 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwgpt\" (UniqueName: \"kubernetes.io/projected/3fa91894-839d-4035-8b46-a787fe957864-kube-api-access-fwgpt\") pod \"insights-operator-5885fdb759-ddm85\" (UID: \"3fa91894-839d-4035-8b46-a787fe957864\") " pod="openshift-insights/insights-operator-5885fdb759-ddm85" Jul 30 00:17:50.902130 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.902043 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/866649cb-1758-4709-82af-f07c8abadf4b-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-phzr9\" (UID: \"866649cb-1758-4709-82af-f07c8abadf4b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-phzr9" Jul 30 00:17:50.902130 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.902070 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/3fa91894-839d-4035-8b46-a787fe957864-snapshots\") pod \"insights-operator-5885fdb759-ddm85\" (UID: \"3fa91894-839d-4035-8b46-a787fe957864\") " pod="openshift-insights/insights-operator-5885fdb759-ddm85" Jul 30 00:17:50.902130 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.902101 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/90afa1c1-9879-49ee-9fac-27f2e7e963f3-default-certificate\") pod \"router-default-877bf98d5-xbkfd\" (UID: \"90afa1c1-9879-49ee-9fac-27f2e7e963f3\") " pod="openshift-ingress/router-default-877bf98d5-xbkfd" Jul 30 00:17:50.902253 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.902126 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fjlv\" (UniqueName: \"kubernetes.io/projected/7d591175-5d66-48f9-be3e-c89d48534295-kube-api-access-7fjlv\") pod \"service-ca-operator-5d6c75f84f-c9vdq\" (UID: \"7d591175-5d66-48f9-be3e-c89d48534295\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c9vdq" Jul 30 00:17:50.902253 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.902202 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f7d33879-e015-4cfe-8d19-11cbc98c1de2-image-registry-private-configuration\") pod \"image-registry-596d7786c9-nrjxz\" (UID: \"f7d33879-e015-4cfe-8d19-11cbc98c1de2\") " pod="openshift-image-registry/image-registry-596d7786c9-nrjxz" Jul 30 00:17:50.902253 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.902232 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0ef8f120-f712-4ed9-adbd-40f785d4aa9d-trusted-ca\") pod \"console-operator-584dcb6b54-b92tw\" (UID: \"0ef8f120-f712-4ed9-adbd-40f785d4aa9d\") " pod="openshift-console-operator/console-operator-584dcb6b54-b92tw" Jul 30 00:17:50.902385 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.902255 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3fa91894-839d-4035-8b46-a787fe957864-serving-cert\") pod \"insights-operator-5885fdb759-ddm85\" (UID: \"3fa91894-839d-4035-8b46-a787fe957864\") " pod="openshift-insights/insights-operator-5885fdb759-ddm85" Jul 30 00:17:50.902385 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.902278 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f7d33879-e015-4cfe-8d19-11cbc98c1de2-installation-pull-secrets\") pod \"image-registry-596d7786c9-nrjxz\" (UID: \"f7d33879-e015-4cfe-8d19-11cbc98c1de2\") " pod="openshift-image-registry/image-registry-596d7786c9-nrjxz" Jul 30 00:17:50.902385 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.902300 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zngl5\" (UniqueName: \"kubernetes.io/projected/f7d33879-e015-4cfe-8d19-11cbc98c1de2-kube-api-access-zngl5\") pod \"image-registry-596d7786c9-nrjxz\" (UID: \"f7d33879-e015-4cfe-8d19-11cbc98c1de2\") " pod="openshift-image-registry/image-registry-596d7786c9-nrjxz" Jul 30 00:17:50.902385 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.902383 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0ef8f120-f712-4ed9-adbd-40f785d4aa9d-config\") pod \"console-operator-584dcb6b54-b92tw\" (UID: \"0ef8f120-f712-4ed9-adbd-40f785d4aa9d\") " pod="openshift-console-operator/console-operator-584dcb6b54-b92tw" Jul 30 00:17:50.902571 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.902417 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0ef8f120-f712-4ed9-adbd-40f785d4aa9d-serving-cert\") pod \"console-operator-584dcb6b54-b92tw\" (UID: \"0ef8f120-f712-4ed9-adbd-40f785d4aa9d\") " pod="openshift-console-operator/console-operator-584dcb6b54-b92tw" Jul 30 00:17:50.902571 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.902462 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fa91894-839d-4035-8b46-a787fe957864-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-ddm85\" (UID: \"3fa91894-839d-4035-8b46-a787fe957864\") " pod="openshift-insights/insights-operator-5885fdb759-ddm85" Jul 30 00:17:50.902571 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.902492 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3fa91894-839d-4035-8b46-a787fe957864-tmp\") pod \"insights-operator-5885fdb759-ddm85\" (UID: \"3fa91894-839d-4035-8b46-a787fe957864\") " pod="openshift-insights/insights-operator-5885fdb759-ddm85" Jul 30 00:17:50.902571 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.902515 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90afa1c1-9879-49ee-9fac-27f2e7e963f3-metrics-certs\") pod \"router-default-877bf98d5-xbkfd\" (UID: \"90afa1c1-9879-49ee-9fac-27f2e7e963f3\") " pod="openshift-ingress/router-default-877bf98d5-xbkfd" Jul 30 00:17:50.902571 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.902546 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f7d33879-e015-4cfe-8d19-11cbc98c1de2-bound-sa-token\") pod \"image-registry-596d7786c9-nrjxz\" (UID: \"f7d33879-e015-4cfe-8d19-11cbc98c1de2\") " pod="openshift-image-registry/image-registry-596d7786c9-nrjxz" Jul 30 00:17:50.902802 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.902577 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/866649cb-1758-4709-82af-f07c8abadf4b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-phzr9\" (UID: \"866649cb-1758-4709-82af-f07c8abadf4b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-phzr9" Jul 30 00:17:50.902802 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.902611 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddp5k\" (UniqueName: \"kubernetes.io/projected/0ef8f120-f712-4ed9-adbd-40f785d4aa9d-kube-api-access-ddp5k\") pod \"console-operator-584dcb6b54-b92tw\" (UID: \"0ef8f120-f712-4ed9-adbd-40f785d4aa9d\") " pod="openshift-console-operator/console-operator-584dcb6b54-b92tw" Jul 30 00:17:50.902802 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.902634 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qdq2\" (UniqueName: \"kubernetes.io/projected/866649cb-1758-4709-82af-f07c8abadf4b-kube-api-access-7qdq2\") pod \"cluster-monitoring-operator-548b489cb8-phzr9\" (UID: \"866649cb-1758-4709-82af-f07c8abadf4b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-phzr9" Jul 30 00:17:50.902802 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.902658 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f7d33879-e015-4cfe-8d19-11cbc98c1de2-ca-trust-extracted\") pod \"image-registry-596d7786c9-nrjxz\" (UID: \"f7d33879-e015-4cfe-8d19-11cbc98c1de2\") " pod="openshift-image-registry/image-registry-596d7786c9-nrjxz" Jul 30 00:17:50.902802 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:50.902685 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58t72\" (UniqueName: \"kubernetes.io/projected/90afa1c1-9879-49ee-9fac-27f2e7e963f3-kube-api-access-58t72\") pod \"router-default-877bf98d5-xbkfd\" (UID: \"90afa1c1-9879-49ee-9fac-27f2e7e963f3\") " pod="openshift-ingress/router-default-877bf98d5-xbkfd" Jul 30 00:17:51.003370 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.003342 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0ef8f120-f712-4ed9-adbd-40f785d4aa9d-config\") pod \"console-operator-584dcb6b54-b92tw\" (UID: \"0ef8f120-f712-4ed9-adbd-40f785d4aa9d\") " pod="openshift-console-operator/console-operator-584dcb6b54-b92tw" Jul 30 00:17:51.003370 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.003373 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0ef8f120-f712-4ed9-adbd-40f785d4aa9d-serving-cert\") pod \"console-operator-584dcb6b54-b92tw\" (UID: \"0ef8f120-f712-4ed9-adbd-40f785d4aa9d\") " pod="openshift-console-operator/console-operator-584dcb6b54-b92tw" Jul 30 00:17:51.003629 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.003392 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fa91894-839d-4035-8b46-a787fe957864-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-ddm85\" (UID: \"3fa91894-839d-4035-8b46-a787fe957864\") " pod="openshift-insights/insights-operator-5885fdb759-ddm85" Jul 30 00:17:51.003629 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.003413 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plbvx\" (UniqueName: \"kubernetes.io/projected/7d7bcf0e-36d5-488a-aa06-817c6cf5831b-kube-api-access-plbvx\") pod \"ingress-canary-qg9tq\" (UID: \"7d7bcf0e-36d5-488a-aa06-817c6cf5831b\") " pod="openshift-ingress-canary/ingress-canary-qg9tq" Jul 30 00:17:51.003629 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.003434 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3fa91894-839d-4035-8b46-a787fe957864-tmp\") pod \"insights-operator-5885fdb759-ddm85\" (UID: \"3fa91894-839d-4035-8b46-a787fe957864\") " pod="openshift-insights/insights-operator-5885fdb759-ddm85" Jul 30 00:17:51.003629 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.003505 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90afa1c1-9879-49ee-9fac-27f2e7e963f3-metrics-certs\") pod \"router-default-877bf98d5-xbkfd\" (UID: \"90afa1c1-9879-49ee-9fac-27f2e7e963f3\") " pod="openshift-ingress/router-default-877bf98d5-xbkfd" Jul 30 00:17:51.003629 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.003545 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f7d33879-e015-4cfe-8d19-11cbc98c1de2-bound-sa-token\") pod \"image-registry-596d7786c9-nrjxz\" (UID: \"f7d33879-e015-4cfe-8d19-11cbc98c1de2\") " pod="openshift-image-registry/image-registry-596d7786c9-nrjxz" Jul 30 00:17:51.003629 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.003563 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/866649cb-1758-4709-82af-f07c8abadf4b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-phzr9\" (UID: \"866649cb-1758-4709-82af-f07c8abadf4b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-phzr9" Jul 30 00:17:51.003629 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.003599 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddp5k\" (UniqueName: \"kubernetes.io/projected/0ef8f120-f712-4ed9-adbd-40f785d4aa9d-kube-api-access-ddp5k\") pod \"console-operator-584dcb6b54-b92tw\" (UID: \"0ef8f120-f712-4ed9-adbd-40f785d4aa9d\") " pod="openshift-console-operator/console-operator-584dcb6b54-b92tw" Jul 30 00:17:51.003629 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.003626 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qdq2\" (UniqueName: \"kubernetes.io/projected/866649cb-1758-4709-82af-f07c8abadf4b-kube-api-access-7qdq2\") pod \"cluster-monitoring-operator-548b489cb8-phzr9\" (UID: \"866649cb-1758-4709-82af-f07c8abadf4b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-phzr9" Jul 30 00:17:51.004052 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.003650 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f7d33879-e015-4cfe-8d19-11cbc98c1de2-ca-trust-extracted\") pod \"image-registry-596d7786c9-nrjxz\" (UID: \"f7d33879-e015-4cfe-8d19-11cbc98c1de2\") " pod="openshift-image-registry/image-registry-596d7786c9-nrjxz" Jul 30 00:17:51.004052 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:51.003753 2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 30 00:17:51.004052 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:51.003821 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90afa1c1-9879-49ee-9fac-27f2e7e963f3-metrics-certs podName:90afa1c1-9879-49ee-9fac-27f2e7e963f3 nodeName:}" failed. No retries permitted until 2026-07-30 00:17:51.503800544 +0000 UTC m=+35.562556506 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/90afa1c1-9879-49ee-9fac-27f2e7e963f3-metrics-certs") pod "router-default-877bf98d5-xbkfd" (UID: "90afa1c1-9879-49ee-9fac-27f2e7e963f3") : secret "router-metrics-certs-default" not found Jul 30 00:17:51.004052 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.003854 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3fa91894-839d-4035-8b46-a787fe957864-tmp\") pod \"insights-operator-5885fdb759-ddm85\" (UID: \"3fa91894-839d-4035-8b46-a787fe957864\") " pod="openshift-insights/insights-operator-5885fdb759-ddm85" Jul 30 00:17:51.004052 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:51.003886 2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 30 00:17:51.004052 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:51.003936 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/866649cb-1758-4709-82af-f07c8abadf4b-cluster-monitoring-operator-tls podName:866649cb-1758-4709-82af-f07c8abadf4b nodeName:}" failed. No retries permitted until 2026-07-30 00:17:51.503923623 +0000 UTC m=+35.562679586 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/866649cb-1758-4709-82af-f07c8abadf4b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-phzr9" (UID: "866649cb-1758-4709-82af-f07c8abadf4b") : secret "cluster-monitoring-operator-tls" not found Jul 30 00:17:51.004052 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.003961 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ffe2ebcf-0fa0-4863-ac2f-568e509d8103-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-ggg5l\" (UID: \"ffe2ebcf-0fa0-4863-ac2f-568e509d8103\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-ggg5l" Jul 30 00:17:51.004052 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.003977 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f7d33879-e015-4cfe-8d19-11cbc98c1de2-ca-trust-extracted\") pod \"image-registry-596d7786c9-nrjxz\" (UID: \"f7d33879-e015-4cfe-8d19-11cbc98c1de2\") " pod="openshift-image-registry/image-registry-596d7786c9-nrjxz" Jul 30 00:17:51.004052 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.003997 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7d7bcf0e-36d5-488a-aa06-817c6cf5831b-cert\") pod \"ingress-canary-qg9tq\" (UID: \"7d7bcf0e-36d5-488a-aa06-817c6cf5831b\") " pod="openshift-ingress-canary/ingress-canary-qg9tq" Jul 30 00:17:51.004052 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.004045 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58t72\" (UniqueName: \"kubernetes.io/projected/90afa1c1-9879-49ee-9fac-27f2e7e963f3-kube-api-access-58t72\") pod \"router-default-877bf98d5-xbkfd\" (UID: \"90afa1c1-9879-49ee-9fac-27f2e7e963f3\") " pod="openshift-ingress/router-default-877bf98d5-xbkfd" Jul 30 00:17:51.004550 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.004077 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c9caa632-edc8-43e9-ab2a-b29758f8150f-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-xjl58\" (UID: \"c9caa632-edc8-43e9-ab2a-b29758f8150f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-xjl58" Jul 30 00:17:51.004550 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.004101 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5ca5f4f2-1c36-4eeb-a43c-8533b4d52e84-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-tx22w\" (UID: \"5ca5f4f2-1c36-4eeb-a43c-8533b4d52e84\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-tx22w" Jul 30 00:17:51.004550 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.004132 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpq5n\" (UniqueName: \"kubernetes.io/projected/ffe2ebcf-0fa0-4863-ac2f-568e509d8103-kube-api-access-gpq5n\") pod \"cluster-samples-operator-6748489656-ggg5l\" (UID: \"ffe2ebcf-0fa0-4863-ac2f-568e509d8103\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-ggg5l" Jul 30 00:17:51.004550 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.004159 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90afa1c1-9879-49ee-9fac-27f2e7e963f3-service-ca-bundle\") pod \"router-default-877bf98d5-xbkfd\" (UID: \"90afa1c1-9879-49ee-9fac-27f2e7e963f3\") " pod="openshift-ingress/router-default-877bf98d5-xbkfd" Jul 30 00:17:51.004550 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.004192 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f7d33879-e015-4cfe-8d19-11cbc98c1de2-registry-certificates\") pod \"image-registry-596d7786c9-nrjxz\" (UID: \"f7d33879-e015-4cfe-8d19-11cbc98c1de2\") " pod="openshift-image-registry/image-registry-596d7786c9-nrjxz" Jul 30 00:17:51.004550 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.004247 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7d591175-5d66-48f9-be3e-c89d48534295-serving-cert\") pod \"service-ca-operator-5d6c75f84f-c9vdq\" (UID: \"7d591175-5d66-48f9-be3e-c89d48534295\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c9vdq" Jul 30 00:17:51.004550 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:51.004257 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/90afa1c1-9879-49ee-9fac-27f2e7e963f3-service-ca-bundle podName:90afa1c1-9879-49ee-9fac-27f2e7e963f3 nodeName:}" failed. No retries permitted until 2026-07-30 00:17:51.504244199 +0000 UTC m=+35.563000172 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/90afa1c1-9879-49ee-9fac-27f2e7e963f3-service-ca-bundle") pod "router-default-877bf98d5-xbkfd" (UID: "90afa1c1-9879-49ee-9fac-27f2e7e963f3") : configmap references non-existent config key: service-ca.crt Jul 30 00:17:51.004550 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.004375 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f7d33879-e015-4cfe-8d19-11cbc98c1de2-registry-tls\") pod \"image-registry-596d7786c9-nrjxz\" (UID: \"f7d33879-e015-4cfe-8d19-11cbc98c1de2\") " pod="openshift-image-registry/image-registry-596d7786c9-nrjxz" Jul 30 00:17:51.004550 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.004410 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7d591175-5d66-48f9-be3e-c89d48534295-config\") pod \"service-ca-operator-5d6c75f84f-c9vdq\" (UID: \"7d591175-5d66-48f9-be3e-c89d48534295\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c9vdq" Jul 30 00:17:51.004550 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.004419 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fa91894-839d-4035-8b46-a787fe957864-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-ddm85\" (UID: \"3fa91894-839d-4035-8b46-a787fe957864\") " pod="openshift-insights/insights-operator-5885fdb759-ddm85" Jul 30 00:17:51.004550 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.004455 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/90afa1c1-9879-49ee-9fac-27f2e7e963f3-stats-auth\") pod \"router-default-877bf98d5-xbkfd\" (UID: \"90afa1c1-9879-49ee-9fac-27f2e7e963f3\") " pod="openshift-ingress/router-default-877bf98d5-xbkfd" Jul 30 00:17:51.005099 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:51.004468 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 30 00:17:51.005099 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:51.004673 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-596d7786c9-nrjxz: secret "image-registry-tls" not found Jul 30 00:17:51.005099 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:51.004717 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f7d33879-e015-4cfe-8d19-11cbc98c1de2-registry-tls podName:f7d33879-e015-4cfe-8d19-11cbc98c1de2 nodeName:}" failed. No retries permitted until 2026-07-30 00:17:51.504703959 +0000 UTC m=+35.563459928 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f7d33879-e015-4cfe-8d19-11cbc98c1de2-registry-tls") pod "image-registry-596d7786c9-nrjxz" (UID: "f7d33879-e015-4cfe-8d19-11cbc98c1de2") : secret "image-registry-tls" not found Jul 30 00:17:51.005099 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.004480 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f7d33879-e015-4cfe-8d19-11cbc98c1de2-trusted-ca\") pod \"image-registry-596d7786c9-nrjxz\" (UID: \"f7d33879-e015-4cfe-8d19-11cbc98c1de2\") " pod="openshift-image-registry/image-registry-596d7786c9-nrjxz" Jul 30 00:17:51.005099 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.004774 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166-tmp-dir\") pod \"dns-default-6kh2h\" (UID: \"bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166\") " pod="openshift-dns/dns-default-6kh2h" Jul 30 00:17:51.005099 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.004802 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5ca5f4f2-1c36-4eeb-a43c-8533b4d52e84-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-tx22w\" (UID: \"5ca5f4f2-1c36-4eeb-a43c-8533b4d52e84\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-tx22w" Jul 30 00:17:51.005099 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.004833 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f7d33879-e015-4cfe-8d19-11cbc98c1de2-registry-certificates\") pod \"image-registry-596d7786c9-nrjxz\" (UID: \"f7d33879-e015-4cfe-8d19-11cbc98c1de2\") " pod="openshift-image-registry/image-registry-596d7786c9-nrjxz" Jul 30 00:17:51.005099 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.004842 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fa91894-839d-4035-8b46-a787fe957864-service-ca-bundle\") pod \"insights-operator-5885fdb759-ddm85\" (UID: \"3fa91894-839d-4035-8b46-a787fe957864\") " pod="openshift-insights/insights-operator-5885fdb759-ddm85" Jul 30 00:17:51.005099 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.004562 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0ef8f120-f712-4ed9-adbd-40f785d4aa9d-config\") pod \"console-operator-584dcb6b54-b92tw\" (UID: \"0ef8f120-f712-4ed9-adbd-40f785d4aa9d\") " pod="openshift-console-operator/console-operator-584dcb6b54-b92tw" Jul 30 00:17:51.005099 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.004874 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwgpt\" (UniqueName: \"kubernetes.io/projected/3fa91894-839d-4035-8b46-a787fe957864-kube-api-access-fwgpt\") pod \"insights-operator-5885fdb759-ddm85\" (UID: \"3fa91894-839d-4035-8b46-a787fe957864\") " pod="openshift-insights/insights-operator-5885fdb759-ddm85" Jul 30 00:17:51.005099 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.004918 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6qxj\" (UniqueName: \"kubernetes.io/projected/a9fed7ee-e7db-4403-98d6-37274ab4f0e8-kube-api-access-q6qxj\") pod \"volume-data-source-validator-549c6d54cf-j2fkq\" (UID: \"a9fed7ee-e7db-4403-98d6-37274ab4f0e8\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-j2fkq" Jul 30 00:17:51.005099 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.004964 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/866649cb-1758-4709-82af-f07c8abadf4b-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-phzr9\" (UID: \"866649cb-1758-4709-82af-f07c8abadf4b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-phzr9" Jul 30 00:17:51.005099 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.005000 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/3fa91894-839d-4035-8b46-a787fe957864-snapshots\") pod \"insights-operator-5885fdb759-ddm85\" (UID: \"3fa91894-839d-4035-8b46-a787fe957864\") " pod="openshift-insights/insights-operator-5885fdb759-ddm85" Jul 30 00:17:51.005099 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.005024 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166-config-volume\") pod \"dns-default-6kh2h\" (UID: \"bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166\") " pod="openshift-dns/dns-default-6kh2h" Jul 30 00:17:51.005099 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.005050 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/90afa1c1-9879-49ee-9fac-27f2e7e963f3-default-certificate\") pod \"router-default-877bf98d5-xbkfd\" (UID: \"90afa1c1-9879-49ee-9fac-27f2e7e963f3\") " pod="openshift-ingress/router-default-877bf98d5-xbkfd" Jul 30 00:17:51.005099 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.005075 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fjlv\" (UniqueName: \"kubernetes.io/projected/7d591175-5d66-48f9-be3e-c89d48534295-kube-api-access-7fjlv\") pod \"service-ca-operator-5d6c75f84f-c9vdq\" (UID: \"7d591175-5d66-48f9-be3e-c89d48534295\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c9vdq" Jul 30 00:17:51.005845 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.005104 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcwjj\" (UniqueName: \"kubernetes.io/projected/c9caa632-edc8-43e9-ab2a-b29758f8150f-kube-api-access-fcwjj\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-xjl58\" (UID: \"c9caa632-edc8-43e9-ab2a-b29758f8150f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-xjl58" Jul 30 00:17:51.005845 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.005107 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7d591175-5d66-48f9-be3e-c89d48534295-config\") pod \"service-ca-operator-5d6c75f84f-c9vdq\" (UID: \"7d591175-5d66-48f9-be3e-c89d48534295\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c9vdq" Jul 30 00:17:51.005845 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.005131 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpjj4\" (UniqueName: \"kubernetes.io/projected/bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166-kube-api-access-bpjj4\") pod \"dns-default-6kh2h\" (UID: \"bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166\") " pod="openshift-dns/dns-default-6kh2h" Jul 30 00:17:51.005845 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.005181 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c9caa632-edc8-43e9-ab2a-b29758f8150f-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-xjl58\" (UID: \"c9caa632-edc8-43e9-ab2a-b29758f8150f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-xjl58" Jul 30 00:17:51.005845 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.005220 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f7d33879-e015-4cfe-8d19-11cbc98c1de2-image-registry-private-configuration\") pod \"image-registry-596d7786c9-nrjxz\" (UID: \"f7d33879-e015-4cfe-8d19-11cbc98c1de2\") " pod="openshift-image-registry/image-registry-596d7786c9-nrjxz" Jul 30 00:17:51.005845 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.005251 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0ef8f120-f712-4ed9-adbd-40f785d4aa9d-trusted-ca\") pod \"console-operator-584dcb6b54-b92tw\" (UID: \"0ef8f120-f712-4ed9-adbd-40f785d4aa9d\") " pod="openshift-console-operator/console-operator-584dcb6b54-b92tw" Jul 30 00:17:51.005845 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.005282 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3fa91894-839d-4035-8b46-a787fe957864-serving-cert\") pod \"insights-operator-5885fdb759-ddm85\" (UID: \"3fa91894-839d-4035-8b46-a787fe957864\") " pod="openshift-insights/insights-operator-5885fdb759-ddm85" Jul 30 00:17:51.005845 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.005341 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fa91894-839d-4035-8b46-a787fe957864-service-ca-bundle\") pod \"insights-operator-5885fdb759-ddm85\" (UID: \"3fa91894-839d-4035-8b46-a787fe957864\") " pod="openshift-insights/insights-operator-5885fdb759-ddm85" Jul 30 00:17:51.005845 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.005365 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f7d33879-e015-4cfe-8d19-11cbc98c1de2-trusted-ca\") pod \"image-registry-596d7786c9-nrjxz\" (UID: \"f7d33879-e015-4cfe-8d19-11cbc98c1de2\") " pod="openshift-image-registry/image-registry-596d7786c9-nrjxz" Jul 30 00:17:51.005845 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.005619 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f7d33879-e015-4cfe-8d19-11cbc98c1de2-installation-pull-secrets\") pod \"image-registry-596d7786c9-nrjxz\" (UID: \"f7d33879-e015-4cfe-8d19-11cbc98c1de2\") " pod="openshift-image-registry/image-registry-596d7786c9-nrjxz" Jul 30 00:17:51.005845 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.005654 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zngl5\" (UniqueName: \"kubernetes.io/projected/f7d33879-e015-4cfe-8d19-11cbc98c1de2-kube-api-access-zngl5\") pod \"image-registry-596d7786c9-nrjxz\" (UID: \"f7d33879-e015-4cfe-8d19-11cbc98c1de2\") " pod="openshift-image-registry/image-registry-596d7786c9-nrjxz" Jul 30 00:17:51.005845 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.005680 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166-metrics-tls\") pod \"dns-default-6kh2h\" (UID: \"bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166\") " pod="openshift-dns/dns-default-6kh2h" Jul 30 00:17:51.005845 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.005807 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/3fa91894-839d-4035-8b46-a787fe957864-snapshots\") pod \"insights-operator-5885fdb759-ddm85\" (UID: \"3fa91894-839d-4035-8b46-a787fe957864\") " pod="openshift-insights/insights-operator-5885fdb759-ddm85" Jul 30 00:17:51.006392 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.006035 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/866649cb-1758-4709-82af-f07c8abadf4b-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-phzr9\" (UID: \"866649cb-1758-4709-82af-f07c8abadf4b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-phzr9" Jul 30 00:17:51.006392 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.006097 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mghlm\" (UniqueName: \"kubernetes.io/projected/cc1eb41e-5e04-4e76-8924-363508ec319b-kube-api-access-mghlm\") pod \"network-check-source-7d88cb55d8-8sgzj\" (UID: \"cc1eb41e-5e04-4e76-8924-363508ec319b\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-8sgzj" Jul 30 00:17:51.006392 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.006118 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0ef8f120-f712-4ed9-adbd-40f785d4aa9d-trusted-ca\") pod \"console-operator-584dcb6b54-b92tw\" (UID: \"0ef8f120-f712-4ed9-adbd-40f785d4aa9d\") " pod="openshift-console-operator/console-operator-584dcb6b54-b92tw" Jul 30 00:17:51.008858 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.008813 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3fa91894-839d-4035-8b46-a787fe957864-serving-cert\") pod \"insights-operator-5885fdb759-ddm85\" (UID: \"3fa91894-839d-4035-8b46-a787fe957864\") " pod="openshift-insights/insights-operator-5885fdb759-ddm85" Jul 30 00:17:51.008858 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.008824 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/90afa1c1-9879-49ee-9fac-27f2e7e963f3-default-certificate\") pod \"router-default-877bf98d5-xbkfd\" (UID: \"90afa1c1-9879-49ee-9fac-27f2e7e963f3\") " pod="openshift-ingress/router-default-877bf98d5-xbkfd" Jul 30 00:17:51.009057 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.008903 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7d591175-5d66-48f9-be3e-c89d48534295-serving-cert\") pod \"service-ca-operator-5d6c75f84f-c9vdq\" (UID: \"7d591175-5d66-48f9-be3e-c89d48534295\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c9vdq" Jul 30 00:17:51.009057 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.008919 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/90afa1c1-9879-49ee-9fac-27f2e7e963f3-stats-auth\") pod \"router-default-877bf98d5-xbkfd\" (UID: \"90afa1c1-9879-49ee-9fac-27f2e7e963f3\") " pod="openshift-ingress/router-default-877bf98d5-xbkfd" Jul 30 00:17:51.009158 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.009077 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f7d33879-e015-4cfe-8d19-11cbc98c1de2-image-registry-private-configuration\") pod \"image-registry-596d7786c9-nrjxz\" (UID: \"f7d33879-e015-4cfe-8d19-11cbc98c1de2\") " pod="openshift-image-registry/image-registry-596d7786c9-nrjxz" Jul 30 00:17:51.009234 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.009217 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f7d33879-e015-4cfe-8d19-11cbc98c1de2-installation-pull-secrets\") pod \"image-registry-596d7786c9-nrjxz\" (UID: \"f7d33879-e015-4cfe-8d19-11cbc98c1de2\") " pod="openshift-image-registry/image-registry-596d7786c9-nrjxz" Jul 30 00:17:51.009710 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.009688 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0ef8f120-f712-4ed9-adbd-40f785d4aa9d-serving-cert\") pod \"console-operator-584dcb6b54-b92tw\" (UID: \"0ef8f120-f712-4ed9-adbd-40f785d4aa9d\") " pod="openshift-console-operator/console-operator-584dcb6b54-b92tw" Jul 30 00:17:51.012936 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.012896 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-58t72\" (UniqueName: \"kubernetes.io/projected/90afa1c1-9879-49ee-9fac-27f2e7e963f3-kube-api-access-58t72\") pod \"router-default-877bf98d5-xbkfd\" (UID: \"90afa1c1-9879-49ee-9fac-27f2e7e963f3\") " pod="openshift-ingress/router-default-877bf98d5-xbkfd" Jul 30 00:17:51.013102 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.013059 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f7d33879-e015-4cfe-8d19-11cbc98c1de2-bound-sa-token\") pod \"image-registry-596d7786c9-nrjxz\" (UID: \"f7d33879-e015-4cfe-8d19-11cbc98c1de2\") " pod="openshift-image-registry/image-registry-596d7786c9-nrjxz" Jul 30 00:17:51.013211 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.013145 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qdq2\" (UniqueName: \"kubernetes.io/projected/866649cb-1758-4709-82af-f07c8abadf4b-kube-api-access-7qdq2\") pod \"cluster-monitoring-operator-548b489cb8-phzr9\" (UID: \"866649cb-1758-4709-82af-f07c8abadf4b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-phzr9" Jul 30 00:17:51.015514 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.015471 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwgpt\" (UniqueName: \"kubernetes.io/projected/3fa91894-839d-4035-8b46-a787fe957864-kube-api-access-fwgpt\") pod \"insights-operator-5885fdb759-ddm85\" (UID: \"3fa91894-839d-4035-8b46-a787fe957864\") " pod="openshift-insights/insights-operator-5885fdb759-ddm85" Jul 30 00:17:51.015514 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.015471 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zngl5\" (UniqueName: \"kubernetes.io/projected/f7d33879-e015-4cfe-8d19-11cbc98c1de2-kube-api-access-zngl5\") pod \"image-registry-596d7786c9-nrjxz\" (UID: \"f7d33879-e015-4cfe-8d19-11cbc98c1de2\") " pod="openshift-image-registry/image-registry-596d7786c9-nrjxz" Jul 30 00:17:51.016040 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.016019 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddp5k\" (UniqueName: \"kubernetes.io/projected/0ef8f120-f712-4ed9-adbd-40f785d4aa9d-kube-api-access-ddp5k\") pod \"console-operator-584dcb6b54-b92tw\" (UID: \"0ef8f120-f712-4ed9-adbd-40f785d4aa9d\") " pod="openshift-console-operator/console-operator-584dcb6b54-b92tw" Jul 30 00:17:51.016456 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.016420 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fjlv\" (UniqueName: \"kubernetes.io/projected/7d591175-5d66-48f9-be3e-c89d48534295-kube-api-access-7fjlv\") pod \"service-ca-operator-5d6c75f84f-c9vdq\" (UID: \"7d591175-5d66-48f9-be3e-c89d48534295\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c9vdq" Jul 30 00:17:51.106637 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.106552 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6qxj\" (UniqueName: \"kubernetes.io/projected/a9fed7ee-e7db-4403-98d6-37274ab4f0e8-kube-api-access-q6qxj\") pod \"volume-data-source-validator-549c6d54cf-j2fkq\" (UID: \"a9fed7ee-e7db-4403-98d6-37274ab4f0e8\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-j2fkq" Jul 30 00:17:51.106637 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.106614 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166-config-volume\") pod \"dns-default-6kh2h\" (UID: \"bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166\") " pod="openshift-dns/dns-default-6kh2h" Jul 30 00:17:51.107116 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.106643 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcwjj\" (UniqueName: \"kubernetes.io/projected/c9caa632-edc8-43e9-ab2a-b29758f8150f-kube-api-access-fcwjj\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-xjl58\" (UID: \"c9caa632-edc8-43e9-ab2a-b29758f8150f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-xjl58" Jul 30 00:17:51.107116 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.106670 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpjj4\" (UniqueName: \"kubernetes.io/projected/bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166-kube-api-access-bpjj4\") pod \"dns-default-6kh2h\" (UID: \"bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166\") " pod="openshift-dns/dns-default-6kh2h" Jul 30 00:17:51.107116 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.106911 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c9caa632-edc8-43e9-ab2a-b29758f8150f-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-xjl58\" (UID: \"c9caa632-edc8-43e9-ab2a-b29758f8150f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-xjl58" Jul 30 00:17:51.107116 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.107001 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166-metrics-tls\") pod \"dns-default-6kh2h\" (UID: \"bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166\") " pod="openshift-dns/dns-default-6kh2h" Jul 30 00:17:51.107116 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.107036 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mghlm\" (UniqueName: \"kubernetes.io/projected/cc1eb41e-5e04-4e76-8924-363508ec319b-kube-api-access-mghlm\") pod \"network-check-source-7d88cb55d8-8sgzj\" (UID: \"cc1eb41e-5e04-4e76-8924-363508ec319b\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-8sgzj" Jul 30 00:17:51.107349 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:51.107141 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 00:17:51.107349 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.107185 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plbvx\" (UniqueName: \"kubernetes.io/projected/7d7bcf0e-36d5-488a-aa06-817c6cf5831b-kube-api-access-plbvx\") pod \"ingress-canary-qg9tq\" (UID: \"7d7bcf0e-36d5-488a-aa06-817c6cf5831b\") " pod="openshift-ingress-canary/ingress-canary-qg9tq" Jul 30 00:17:51.107349 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:51.107219 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166-metrics-tls podName:bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166 nodeName:}" failed. No retries permitted until 2026-07-30 00:17:51.607197735 +0000 UTC m=+35.665953695 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166-metrics-tls") pod "dns-default-6kh2h" (UID: "bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166") : secret "dns-default-metrics-tls" not found Jul 30 00:17:51.107349 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.107300 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ffe2ebcf-0fa0-4863-ac2f-568e509d8103-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-ggg5l\" (UID: \"ffe2ebcf-0fa0-4863-ac2f-568e509d8103\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-ggg5l" Jul 30 00:17:51.107349 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.107328 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7d7bcf0e-36d5-488a-aa06-817c6cf5831b-cert\") pod \"ingress-canary-qg9tq\" (UID: \"7d7bcf0e-36d5-488a-aa06-817c6cf5831b\") " pod="openshift-ingress-canary/ingress-canary-qg9tq" Jul 30 00:17:51.107596 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.107359 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c9caa632-edc8-43e9-ab2a-b29758f8150f-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-xjl58\" (UID: \"c9caa632-edc8-43e9-ab2a-b29758f8150f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-xjl58" Jul 30 00:17:51.107596 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:51.107382 2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 30 00:17:51.107596 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:51.107530 2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 30 00:17:51.107596 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.107387 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5ca5f4f2-1c36-4eeb-a43c-8533b4d52e84-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-tx22w\" (UID: \"5ca5f4f2-1c36-4eeb-a43c-8533b4d52e84\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-tx22w" Jul 30 00:17:51.107596 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:51.107565 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffe2ebcf-0fa0-4863-ac2f-568e509d8103-samples-operator-tls podName:ffe2ebcf-0fa0-4863-ac2f-568e509d8103 nodeName:}" failed. No retries permitted until 2026-07-30 00:17:51.607546243 +0000 UTC m=+35.666302199 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ffe2ebcf-0fa0-4863-ac2f-568e509d8103-samples-operator-tls") pod "cluster-samples-operator-6748489656-ggg5l" (UID: "ffe2ebcf-0fa0-4863-ac2f-568e509d8103") : secret "samples-operator-tls" not found Jul 30 00:17:51.107596 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:51.107595 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ca5f4f2-1c36-4eeb-a43c-8533b4d52e84-networking-console-plugin-cert podName:5ca5f4f2-1c36-4eeb-a43c-8533b4d52e84 nodeName:}" failed. No retries permitted until 2026-07-30 00:17:51.607579259 +0000 UTC m=+35.666335221 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/5ca5f4f2-1c36-4eeb-a43c-8533b4d52e84-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-tx22w" (UID: "5ca5f4f2-1c36-4eeb-a43c-8533b4d52e84") : secret "networking-console-plugin-cert" not found Jul 30 00:17:51.107921 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.107646 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpq5n\" (UniqueName: \"kubernetes.io/projected/ffe2ebcf-0fa0-4863-ac2f-568e509d8103-kube-api-access-gpq5n\") pod \"cluster-samples-operator-6748489656-ggg5l\" (UID: \"ffe2ebcf-0fa0-4863-ac2f-568e509d8103\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-ggg5l" Jul 30 00:17:51.107921 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.107749 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166-tmp-dir\") pod \"dns-default-6kh2h\" (UID: \"bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166\") " pod="openshift-dns/dns-default-6kh2h" Jul 30 00:17:51.107921 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.107779 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5ca5f4f2-1c36-4eeb-a43c-8533b4d52e84-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-tx22w\" (UID: \"5ca5f4f2-1c36-4eeb-a43c-8533b4d52e84\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-tx22w" Jul 30 00:17:51.108073 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.107974 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c9caa632-edc8-43e9-ab2a-b29758f8150f-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-xjl58\" (UID: \"c9caa632-edc8-43e9-ab2a-b29758f8150f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-xjl58" Jul 30 00:17:51.108073 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.107975 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166-config-volume\") pod \"dns-default-6kh2h\" (UID: \"bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166\") " pod="openshift-dns/dns-default-6kh2h" Jul 30 00:17:51.108189 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.108165 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166-tmp-dir\") pod \"dns-default-6kh2h\" (UID: \"bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166\") " pod="openshift-dns/dns-default-6kh2h" Jul 30 00:17:51.108479 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.108439 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5ca5f4f2-1c36-4eeb-a43c-8533b4d52e84-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-tx22w\" (UID: \"5ca5f4f2-1c36-4eeb-a43c-8533b4d52e84\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-tx22w" Jul 30 00:17:51.109728 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.109705 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c9caa632-edc8-43e9-ab2a-b29758f8150f-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-xjl58\" (UID: \"c9caa632-edc8-43e9-ab2a-b29758f8150f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-xjl58" Jul 30 00:17:51.110365 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.110328 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7d7bcf0e-36d5-488a-aa06-817c6cf5831b-cert\") pod \"ingress-canary-qg9tq\" (UID: \"7d7bcf0e-36d5-488a-aa06-817c6cf5831b\") " pod="openshift-ingress-canary/ingress-canary-qg9tq" Jul 30 00:17:51.116049 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.116003 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpjj4\" (UniqueName: \"kubernetes.io/projected/bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166-kube-api-access-bpjj4\") pod \"dns-default-6kh2h\" (UID: \"bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166\") " pod="openshift-dns/dns-default-6kh2h" Jul 30 00:17:51.116852 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.116809 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-plbvx\" (UniqueName: \"kubernetes.io/projected/7d7bcf0e-36d5-488a-aa06-817c6cf5831b-kube-api-access-plbvx\") pod \"ingress-canary-qg9tq\" (UID: \"7d7bcf0e-36d5-488a-aa06-817c6cf5831b\") " pod="openshift-ingress-canary/ingress-canary-qg9tq" Jul 30 00:17:51.118142 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.118121 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcwjj\" (UniqueName: \"kubernetes.io/projected/c9caa632-edc8-43e9-ab2a-b29758f8150f-kube-api-access-fcwjj\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-xjl58\" (UID: \"c9caa632-edc8-43e9-ab2a-b29758f8150f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-xjl58" Jul 30 00:17:51.118243 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.118142 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6qxj\" (UniqueName: \"kubernetes.io/projected/a9fed7ee-e7db-4403-98d6-37274ab4f0e8-kube-api-access-q6qxj\") pod \"volume-data-source-validator-549c6d54cf-j2fkq\" (UID: \"a9fed7ee-e7db-4403-98d6-37274ab4f0e8\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-j2fkq" Jul 30 00:17:51.118301 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.118289 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpq5n\" (UniqueName: \"kubernetes.io/projected/ffe2ebcf-0fa0-4863-ac2f-568e509d8103-kube-api-access-gpq5n\") pod \"cluster-samples-operator-6748489656-ggg5l\" (UID: \"ffe2ebcf-0fa0-4863-ac2f-568e509d8103\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-ggg5l" Jul 30 00:17:51.118888 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.118864 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mghlm\" (UniqueName: \"kubernetes.io/projected/cc1eb41e-5e04-4e76-8924-363508ec319b-kube-api-access-mghlm\") pod \"network-check-source-7d88cb55d8-8sgzj\" (UID: \"cc1eb41e-5e04-4e76-8924-363508ec319b\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-8sgzj" Jul 30 00:17:51.126571 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.126555 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-ddm85" Jul 30 00:17:51.172073 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.172042 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c9vdq" Jul 30 00:17:51.181113 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.180921 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-b92tw" Jul 30 00:17:51.195043 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.194911 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-j2fkq" Jul 30 00:17:51.201878 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.201581 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-xjl58" Jul 30 00:17:51.216729 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.216687 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-8sgzj" Jul 30 00:17:51.226759 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.224994 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qg9tq" Jul 30 00:17:51.337895 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.335676 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-ddm85"] Jul 30 00:17:51.414413 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.414237 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:17:51.421657 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.421626 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c9vdq"] Jul 30 00:17:51.426554 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:51.426523 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d591175_5d66_48f9_be3e_c89d48534295.slice/crio-923d85036d08ac5c16612919aefaf5b8581b9f196aa323b6455ebb4a0d859489 WatchSource:0}: Error finding container 923d85036d08ac5c16612919aefaf5b8581b9f196aa323b6455ebb4a0d859489: Status 404 returned error can't find the container with id 923d85036d08ac5c16612919aefaf5b8581b9f196aa323b6455ebb4a0d859489 Jul 30 00:17:51.435805 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.435778 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-b92tw"] Jul 30 00:17:51.439235 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:51.439203 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ef8f120_f712_4ed9_adbd_40f785d4aa9d.slice/crio-02b56542a9ae2bb6be0582421e589a4b1f8d56c2f3ef9afaae5e072697905df6 WatchSource:0}: Error finding container 02b56542a9ae2bb6be0582421e589a4b1f8d56c2f3ef9afaae5e072697905df6: Status 404 returned error can't find the container with id 02b56542a9ae2bb6be0582421e589a4b1f8d56c2f3ef9afaae5e072697905df6 Jul 30 00:17:51.455421 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.455381 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-j2fkq"] Jul 30 00:17:51.458546 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.458517 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-xjl58"] Jul 30 00:17:51.462394 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:51.462357 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9caa632_edc8_43e9_ab2a_b29758f8150f.slice/crio-f509ca17e58b30cd314077d650f8d267270700e959794d370ef33c22cd108636 WatchSource:0}: Error finding container f509ca17e58b30cd314077d650f8d267270700e959794d370ef33c22cd108636: Status 404 returned error can't find the container with id f509ca17e58b30cd314077d650f8d267270700e959794d370ef33c22cd108636 Jul 30 00:17:51.463295 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:51.463259 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9fed7ee_e7db_4403_98d6_37274ab4f0e8.slice/crio-2167eae3b75172cbddb47e70e847a928f9eae978da595313f13acd78d88a19ce WatchSource:0}: Error finding container 2167eae3b75172cbddb47e70e847a928f9eae978da595313f13acd78d88a19ce: Status 404 returned error can't find the container with id 2167eae3b75172cbddb47e70e847a928f9eae978da595313f13acd78d88a19ce Jul 30 00:17:51.468541 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.468483 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-8sgzj"] Jul 30 00:17:51.469936 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.469810 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7xxn" Jul 30 00:17:51.470583 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.469967 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zxg4x" Jul 30 00:17:51.470583 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.469983 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j26vw" Jul 30 00:17:51.471674 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.471653 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qg9tq"] Jul 30 00:17:51.472310 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.472194 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 30 00:17:51.472453 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.472405 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qvw7x\"" Jul 30 00:17:51.472830 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.472758 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-f7pzr\"" Jul 30 00:17:51.472920 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.472876 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 30 00:17:51.473393 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:51.473285 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc1eb41e_5e04_4e76_8924_363508ec319b.slice/crio-ef3c78d489e086f6494ffa0709864b1bf61ee9e6ed6036bb18aaf3c2d8132db3 WatchSource:0}: Error finding container ef3c78d489e086f6494ffa0709864b1bf61ee9e6ed6036bb18aaf3c2d8132db3: Status 404 returned error can't find the container with id ef3c78d489e086f6494ffa0709864b1bf61ee9e6ed6036bb18aaf3c2d8132db3 Jul 30 00:17:51.475354 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:17:51.475333 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d7bcf0e_36d5_488a_aa06_817c6cf5831b.slice/crio-207feb13f7640fdc588939a063c6b6fe177ae5026e13b4ec19551d0b9060ef5e WatchSource:0}: Error finding container 207feb13f7640fdc588939a063c6b6fe177ae5026e13b4ec19551d0b9060ef5e: Status 404 returned error can't find the container with id 207feb13f7640fdc588939a063c6b6fe177ae5026e13b4ec19551d0b9060ef5e Jul 30 00:17:51.498368 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.498339 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:17:51.511931 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.511891 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f7d33879-e015-4cfe-8d19-11cbc98c1de2-registry-tls\") pod \"image-registry-596d7786c9-nrjxz\" (UID: \"f7d33879-e015-4cfe-8d19-11cbc98c1de2\") " pod="openshift-image-registry/image-registry-596d7786c9-nrjxz" Jul 30 00:17:51.512111 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.512054 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90afa1c1-9879-49ee-9fac-27f2e7e963f3-metrics-certs\") pod \"router-default-877bf98d5-xbkfd\" (UID: \"90afa1c1-9879-49ee-9fac-27f2e7e963f3\") " pod="openshift-ingress/router-default-877bf98d5-xbkfd" Jul 30 00:17:51.512111 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.512080 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/866649cb-1758-4709-82af-f07c8abadf4b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-phzr9\" (UID: \"866649cb-1758-4709-82af-f07c8abadf4b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-phzr9" Jul 30 00:17:51.512111 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:51.512085 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 30 00:17:51.512111 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:51.512107 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-596d7786c9-nrjxz: secret "image-registry-tls" not found Jul 30 00:17:51.512293 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.512125 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90afa1c1-9879-49ee-9fac-27f2e7e963f3-service-ca-bundle\") pod \"router-default-877bf98d5-xbkfd\" (UID: \"90afa1c1-9879-49ee-9fac-27f2e7e963f3\") " pod="openshift-ingress/router-default-877bf98d5-xbkfd" Jul 30 00:17:51.512293 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:51.512172 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f7d33879-e015-4cfe-8d19-11cbc98c1de2-registry-tls podName:f7d33879-e015-4cfe-8d19-11cbc98c1de2 nodeName:}" failed. No retries permitted until 2026-07-30 00:17:52.51215043 +0000 UTC m=+36.570906388 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f7d33879-e015-4cfe-8d19-11cbc98c1de2-registry-tls") pod "image-registry-596d7786c9-nrjxz" (UID: "f7d33879-e015-4cfe-8d19-11cbc98c1de2") : secret "image-registry-tls" not found Jul 30 00:17:51.512293 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:51.512204 2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 30 00:17:51.512293 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:51.512241 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/90afa1c1-9879-49ee-9fac-27f2e7e963f3-service-ca-bundle podName:90afa1c1-9879-49ee-9fac-27f2e7e963f3 nodeName:}" failed. No retries permitted until 2026-07-30 00:17:52.51222533 +0000 UTC m=+36.570981279 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/90afa1c1-9879-49ee-9fac-27f2e7e963f3-service-ca-bundle") pod "router-default-877bf98d5-xbkfd" (UID: "90afa1c1-9879-49ee-9fac-27f2e7e963f3") : configmap references non-existent config key: service-ca.crt Jul 30 00:17:51.512293 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:51.512260 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90afa1c1-9879-49ee-9fac-27f2e7e963f3-metrics-certs podName:90afa1c1-9879-49ee-9fac-27f2e7e963f3 nodeName:}" failed. No retries permitted until 2026-07-30 00:17:52.512250654 +0000 UTC m=+36.571006621 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/90afa1c1-9879-49ee-9fac-27f2e7e963f3-metrics-certs") pod "router-default-877bf98d5-xbkfd" (UID: "90afa1c1-9879-49ee-9fac-27f2e7e963f3") : secret "router-metrics-certs-default" not found Jul 30 00:17:51.512293 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:51.512271 2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 30 00:17:51.512522 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:51.512325 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/866649cb-1758-4709-82af-f07c8abadf4b-cluster-monitoring-operator-tls podName:866649cb-1758-4709-82af-f07c8abadf4b nodeName:}" failed. No retries permitted until 2026-07-30 00:17:52.512312827 +0000 UTC m=+36.571068789 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/866649cb-1758-4709-82af-f07c8abadf4b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-phzr9" (UID: "866649cb-1758-4709-82af-f07c8abadf4b") : secret "cluster-monitoring-operator-tls" not found Jul 30 00:17:51.595807 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.595770 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-xjl58" event={"ID":"c9caa632-edc8-43e9-ab2a-b29758f8150f","Type":"ContainerStarted","Data":"f509ca17e58b30cd314077d650f8d267270700e959794d370ef33c22cd108636"} Jul 30 00:17:51.596829 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.596802 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-b92tw" event={"ID":"0ef8f120-f712-4ed9-adbd-40f785d4aa9d","Type":"ContainerStarted","Data":"02b56542a9ae2bb6be0582421e589a4b1f8d56c2f3ef9afaae5e072697905df6"} Jul 30 00:17:51.597909 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.597880 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qg9tq" event={"ID":"7d7bcf0e-36d5-488a-aa06-817c6cf5831b","Type":"ContainerStarted","Data":"207feb13f7640fdc588939a063c6b6fe177ae5026e13b4ec19551d0b9060ef5e"} Jul 30 00:17:51.598950 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.598928 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-8sgzj" event={"ID":"cc1eb41e-5e04-4e76-8924-363508ec319b","Type":"ContainerStarted","Data":"ef3c78d489e086f6494ffa0709864b1bf61ee9e6ed6036bb18aaf3c2d8132db3"} Jul 30 00:17:51.599996 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.599969 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c9vdq" event={"ID":"7d591175-5d66-48f9-be3e-c89d48534295","Type":"ContainerStarted","Data":"923d85036d08ac5c16612919aefaf5b8581b9f196aa323b6455ebb4a0d859489"} Jul 30 00:17:51.600935 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.600916 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-ddm85" event={"ID":"3fa91894-839d-4035-8b46-a787fe957864","Type":"ContainerStarted","Data":"cbf63fc53021799a4885b809b79f1163233cd450a9e6fd08de8e4f17f28f4676"} Jul 30 00:17:51.601872 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.601851 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-j2fkq" event={"ID":"a9fed7ee-e7db-4403-98d6-37274ab4f0e8","Type":"ContainerStarted","Data":"2167eae3b75172cbddb47e70e847a928f9eae978da595313f13acd78d88a19ce"} Jul 30 00:17:51.613228 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.613205 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166-metrics-tls\") pod \"dns-default-6kh2h\" (UID: \"bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166\") " pod="openshift-dns/dns-default-6kh2h" Jul 30 00:17:51.613325 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.613276 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ffe2ebcf-0fa0-4863-ac2f-568e509d8103-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-ggg5l\" (UID: \"ffe2ebcf-0fa0-4863-ac2f-568e509d8103\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-ggg5l" Jul 30 00:17:51.613325 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.613299 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5ca5f4f2-1c36-4eeb-a43c-8533b4d52e84-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-tx22w\" (UID: \"5ca5f4f2-1c36-4eeb-a43c-8533b4d52e84\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-tx22w" Jul 30 00:17:51.613509 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:51.613357 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 00:17:51.613509 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:51.613411 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166-metrics-tls podName:bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166 nodeName:}" failed. No retries permitted until 2026-07-30 00:17:52.613397296 +0000 UTC m=+36.672153241 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166-metrics-tls") pod "dns-default-6kh2h" (UID: "bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166") : secret "dns-default-metrics-tls" not found Jul 30 00:17:51.613509 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:51.613456 2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 30 00:17:51.613509 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:51.613504 2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 30 00:17:51.613704 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:51.613526 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffe2ebcf-0fa0-4863-ac2f-568e509d8103-samples-operator-tls podName:ffe2ebcf-0fa0-4863-ac2f-568e509d8103 nodeName:}" failed. No retries permitted until 2026-07-30 00:17:52.61350809 +0000 UTC m=+36.672264065 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ffe2ebcf-0fa0-4863-ac2f-568e509d8103-samples-operator-tls") pod "cluster-samples-operator-6748489656-ggg5l" (UID: "ffe2ebcf-0fa0-4863-ac2f-568e509d8103") : secret "samples-operator-tls" not found Jul 30 00:17:51.613704 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:51.613546 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ca5f4f2-1c36-4eeb-a43c-8533b4d52e84-networking-console-plugin-cert podName:5ca5f4f2-1c36-4eeb-a43c-8533b4d52e84 nodeName:}" failed. No retries permitted until 2026-07-30 00:17:52.613536231 +0000 UTC m=+36.672292177 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/5ca5f4f2-1c36-4eeb-a43c-8533b4d52e84-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-tx22w" (UID: "5ca5f4f2-1c36-4eeb-a43c-8533b4d52e84") : secret "networking-console-plugin-cert" not found Jul 30 00:17:51.792423 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.792336 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:17:51.890848 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:51.890816 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:17:52.522891 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:52.522647 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90afa1c1-9879-49ee-9fac-27f2e7e963f3-metrics-certs\") pod \"router-default-877bf98d5-xbkfd\" (UID: \"90afa1c1-9879-49ee-9fac-27f2e7e963f3\") " pod="openshift-ingress/router-default-877bf98d5-xbkfd" Jul 30 00:17:52.523693 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:52.522916 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/866649cb-1758-4709-82af-f07c8abadf4b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-phzr9\" (UID: \"866649cb-1758-4709-82af-f07c8abadf4b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-phzr9" Jul 30 00:17:52.523693 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:52.522994 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90afa1c1-9879-49ee-9fac-27f2e7e963f3-service-ca-bundle\") pod \"router-default-877bf98d5-xbkfd\" (UID: \"90afa1c1-9879-49ee-9fac-27f2e7e963f3\") " pod="openshift-ingress/router-default-877bf98d5-xbkfd" Jul 30 00:17:52.523693 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:52.523031 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f7d33879-e015-4cfe-8d19-11cbc98c1de2-registry-tls\") pod \"image-registry-596d7786c9-nrjxz\" (UID: \"f7d33879-e015-4cfe-8d19-11cbc98c1de2\") " pod="openshift-image-registry/image-registry-596d7786c9-nrjxz" Jul 30 00:17:52.523693 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:52.522821 2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 30 00:17:52.523693 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:52.523262 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90afa1c1-9879-49ee-9fac-27f2e7e963f3-metrics-certs podName:90afa1c1-9879-49ee-9fac-27f2e7e963f3 nodeName:}" failed. No retries permitted until 2026-07-30 00:17:54.523240292 +0000 UTC m=+38.581996242 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/90afa1c1-9879-49ee-9fac-27f2e7e963f3-metrics-certs") pod "router-default-877bf98d5-xbkfd" (UID: "90afa1c1-9879-49ee-9fac-27f2e7e963f3") : secret "router-metrics-certs-default" not found Jul 30 00:17:52.523693 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:52.523670 2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 30 00:17:52.524230 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:52.523720 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/866649cb-1758-4709-82af-f07c8abadf4b-cluster-monitoring-operator-tls podName:866649cb-1758-4709-82af-f07c8abadf4b nodeName:}" failed. No retries permitted until 2026-07-30 00:17:54.523703556 +0000 UTC m=+38.582459516 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/866649cb-1758-4709-82af-f07c8abadf4b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-phzr9" (UID: "866649cb-1758-4709-82af-f07c8abadf4b") : secret "cluster-monitoring-operator-tls" not found Jul 30 00:17:52.524230 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:52.523197 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 30 00:17:52.524230 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:52.523793 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-596d7786c9-nrjxz: secret "image-registry-tls" not found Jul 30 00:17:52.524230 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:52.523809 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/90afa1c1-9879-49ee-9fac-27f2e7e963f3-service-ca-bundle podName:90afa1c1-9879-49ee-9fac-27f2e7e963f3 nodeName:}" failed. No retries permitted until 2026-07-30 00:17:54.523797276 +0000 UTC m=+38.582553225 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/90afa1c1-9879-49ee-9fac-27f2e7e963f3-service-ca-bundle") pod "router-default-877bf98d5-xbkfd" (UID: "90afa1c1-9879-49ee-9fac-27f2e7e963f3") : configmap references non-existent config key: service-ca.crt Jul 30 00:17:52.524230 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:52.523865 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f7d33879-e015-4cfe-8d19-11cbc98c1de2-registry-tls podName:f7d33879-e015-4cfe-8d19-11cbc98c1de2 nodeName:}" failed. No retries permitted until 2026-07-30 00:17:54.523843153 +0000 UTC m=+38.582599109 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f7d33879-e015-4cfe-8d19-11cbc98c1de2-registry-tls") pod "image-registry-596d7786c9-nrjxz" (UID: "f7d33879-e015-4cfe-8d19-11cbc98c1de2") : secret "image-registry-tls" not found Jul 30 00:17:52.625296 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:52.624221 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166-metrics-tls\") pod \"dns-default-6kh2h\" (UID: \"bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166\") " pod="openshift-dns/dns-default-6kh2h" Jul 30 00:17:52.625296 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:52.624311 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ffe2ebcf-0fa0-4863-ac2f-568e509d8103-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-ggg5l\" (UID: \"ffe2ebcf-0fa0-4863-ac2f-568e509d8103\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-ggg5l" Jul 30 00:17:52.625296 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:52.624342 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5ca5f4f2-1c36-4eeb-a43c-8533b4d52e84-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-tx22w\" (UID: \"5ca5f4f2-1c36-4eeb-a43c-8533b4d52e84\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-tx22w" Jul 30 00:17:52.625296 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:52.624539 2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 30 00:17:52.625296 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:52.624610 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ca5f4f2-1c36-4eeb-a43c-8533b4d52e84-networking-console-plugin-cert podName:5ca5f4f2-1c36-4eeb-a43c-8533b4d52e84 nodeName:}" failed. No retries permitted until 2026-07-30 00:17:54.624590406 +0000 UTC m=+38.683346376 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/5ca5f4f2-1c36-4eeb-a43c-8533b4d52e84-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-tx22w" (UID: "5ca5f4f2-1c36-4eeb-a43c-8533b4d52e84") : secret "networking-console-plugin-cert" not found Jul 30 00:17:52.625296 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:52.625123 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 00:17:52.625296 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:52.625169 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166-metrics-tls podName:bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166 nodeName:}" failed. No retries permitted until 2026-07-30 00:17:54.625155121 +0000 UTC m=+38.683911081 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166-metrics-tls") pod "dns-default-6kh2h" (UID: "bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166") : secret "dns-default-metrics-tls" not found Jul 30 00:17:52.625296 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:52.625223 2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 30 00:17:52.625296 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:52.625253 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffe2ebcf-0fa0-4863-ac2f-568e509d8103-samples-operator-tls podName:ffe2ebcf-0fa0-4863-ac2f-568e509d8103 nodeName:}" failed. No retries permitted until 2026-07-30 00:17:54.625243885 +0000 UTC m=+38.683999834 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ffe2ebcf-0fa0-4863-ac2f-568e509d8103-samples-operator-tls") pod "cluster-samples-operator-6748489656-ggg5l" (UID: "ffe2ebcf-0fa0-4863-ac2f-568e509d8103") : secret "samples-operator-tls" not found Jul 30 00:17:54.543667 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:54.543625 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90afa1c1-9879-49ee-9fac-27f2e7e963f3-metrics-certs\") pod \"router-default-877bf98d5-xbkfd\" (UID: \"90afa1c1-9879-49ee-9fac-27f2e7e963f3\") " pod="openshift-ingress/router-default-877bf98d5-xbkfd" Jul 30 00:17:54.544141 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:54.543676 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/866649cb-1758-4709-82af-f07c8abadf4b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-phzr9\" (UID: \"866649cb-1758-4709-82af-f07c8abadf4b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-phzr9" Jul 30 00:17:54.544141 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:54.543806 2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 30 00:17:54.544141 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:54.543827 2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 30 00:17:54.544141 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:54.543858 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90afa1c1-9879-49ee-9fac-27f2e7e963f3-service-ca-bundle\") pod \"router-default-877bf98d5-xbkfd\" (UID: \"90afa1c1-9879-49ee-9fac-27f2e7e963f3\") " pod="openshift-ingress/router-default-877bf98d5-xbkfd" Jul 30 00:17:54.544141 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:54.543879 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90afa1c1-9879-49ee-9fac-27f2e7e963f3-metrics-certs podName:90afa1c1-9879-49ee-9fac-27f2e7e963f3 nodeName:}" failed. No retries permitted until 2026-07-30 00:17:58.54386039 +0000 UTC m=+42.602616351 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/90afa1c1-9879-49ee-9fac-27f2e7e963f3-metrics-certs") pod "router-default-877bf98d5-xbkfd" (UID: "90afa1c1-9879-49ee-9fac-27f2e7e963f3") : secret "router-metrics-certs-default" not found Jul 30 00:17:54.544141 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:54.543952 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/866649cb-1758-4709-82af-f07c8abadf4b-cluster-monitoring-operator-tls podName:866649cb-1758-4709-82af-f07c8abadf4b nodeName:}" failed. No retries permitted until 2026-07-30 00:17:58.543921206 +0000 UTC m=+42.602677171 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/866649cb-1758-4709-82af-f07c8abadf4b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-phzr9" (UID: "866649cb-1758-4709-82af-f07c8abadf4b") : secret "cluster-monitoring-operator-tls" not found Jul 30 00:17:54.544141 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:54.543993 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/90afa1c1-9879-49ee-9fac-27f2e7e963f3-service-ca-bundle podName:90afa1c1-9879-49ee-9fac-27f2e7e963f3 nodeName:}" failed. No retries permitted until 2026-07-30 00:17:58.543976233 +0000 UTC m=+42.602732178 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/90afa1c1-9879-49ee-9fac-27f2e7e963f3-service-ca-bundle") pod "router-default-877bf98d5-xbkfd" (UID: "90afa1c1-9879-49ee-9fac-27f2e7e963f3") : configmap references non-existent config key: service-ca.crt Jul 30 00:17:54.544141 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:54.544044 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f7d33879-e015-4cfe-8d19-11cbc98c1de2-registry-tls\") pod \"image-registry-596d7786c9-nrjxz\" (UID: \"f7d33879-e015-4cfe-8d19-11cbc98c1de2\") " pod="openshift-image-registry/image-registry-596d7786c9-nrjxz" Jul 30 00:17:54.544462 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:54.544191 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 30 00:17:54.544462 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:54.544205 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-596d7786c9-nrjxz: secret "image-registry-tls" not found Jul 30 00:17:54.544462 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:54.544250 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f7d33879-e015-4cfe-8d19-11cbc98c1de2-registry-tls podName:f7d33879-e015-4cfe-8d19-11cbc98c1de2 nodeName:}" failed. No retries permitted until 2026-07-30 00:17:58.544236456 +0000 UTC m=+42.602992407 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f7d33879-e015-4cfe-8d19-11cbc98c1de2-registry-tls") pod "image-registry-596d7786c9-nrjxz" (UID: "f7d33879-e015-4cfe-8d19-11cbc98c1de2") : secret "image-registry-tls" not found Jul 30 00:17:54.645151 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:54.645118 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166-metrics-tls\") pod \"dns-default-6kh2h\" (UID: \"bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166\") " pod="openshift-dns/dns-default-6kh2h" Jul 30 00:17:54.645331 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:54.645211 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ffe2ebcf-0fa0-4863-ac2f-568e509d8103-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-ggg5l\" (UID: \"ffe2ebcf-0fa0-4863-ac2f-568e509d8103\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-ggg5l" Jul 30 00:17:54.645331 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:54.645246 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5ca5f4f2-1c36-4eeb-a43c-8533b4d52e84-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-tx22w\" (UID: \"5ca5f4f2-1c36-4eeb-a43c-8533b4d52e84\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-tx22w" Jul 30 00:17:54.645331 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:54.645280 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 00:17:54.645504 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:54.645354 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166-metrics-tls podName:bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166 nodeName:}" failed. No retries permitted until 2026-07-30 00:17:58.645331911 +0000 UTC m=+42.704087880 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166-metrics-tls") pod "dns-default-6kh2h" (UID: "bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166") : secret "dns-default-metrics-tls" not found Jul 30 00:17:54.645504 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:54.645378 2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 30 00:17:54.645504 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:54.645390 2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 30 00:17:54.645504 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:54.645436 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffe2ebcf-0fa0-4863-ac2f-568e509d8103-samples-operator-tls podName:ffe2ebcf-0fa0-4863-ac2f-568e509d8103 nodeName:}" failed. No retries permitted until 2026-07-30 00:17:58.645419143 +0000 UTC m=+42.704175118 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ffe2ebcf-0fa0-4863-ac2f-568e509d8103-samples-operator-tls") pod "cluster-samples-operator-6748489656-ggg5l" (UID: "ffe2ebcf-0fa0-4863-ac2f-568e509d8103") : secret "samples-operator-tls" not found Jul 30 00:17:54.645504 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:54.645456 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ca5f4f2-1c36-4eeb-a43c-8533b4d52e84-networking-console-plugin-cert podName:5ca5f4f2-1c36-4eeb-a43c-8533b4d52e84 nodeName:}" failed. No retries permitted until 2026-07-30 00:17:58.645445944 +0000 UTC m=+42.704201894 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/5ca5f4f2-1c36-4eeb-a43c-8533b4d52e84-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-tx22w" (UID: "5ca5f4f2-1c36-4eeb-a43c-8533b4d52e84") : secret "networking-console-plugin-cert" not found Jul 30 00:17:55.231066 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:55.231032 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:17:55.285304 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:55.285255 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:17:55.956605 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:55.956528 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:17:56.031038 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:56.031000 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:17:56.132168 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:56.132134 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:17:56.232265 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:56.232132 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:17:56.332437 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:56.332403 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:17:58.584712 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:58.584670 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90afa1c1-9879-49ee-9fac-27f2e7e963f3-metrics-certs\") pod \"router-default-877bf98d5-xbkfd\" (UID: \"90afa1c1-9879-49ee-9fac-27f2e7e963f3\") " pod="openshift-ingress/router-default-877bf98d5-xbkfd" Jul 30 00:17:58.584712 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:58.584714 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/866649cb-1758-4709-82af-f07c8abadf4b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-phzr9\" (UID: \"866649cb-1758-4709-82af-f07c8abadf4b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-phzr9" Jul 30 00:17:58.585152 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:58.584853 2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 30 00:17:58.585152 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:58.584856 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90afa1c1-9879-49ee-9fac-27f2e7e963f3-service-ca-bundle\") pod \"router-default-877bf98d5-xbkfd\" (UID: \"90afa1c1-9879-49ee-9fac-27f2e7e963f3\") " pod="openshift-ingress/router-default-877bf98d5-xbkfd" Jul 30 00:17:58.585152 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:58.584922 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90afa1c1-9879-49ee-9fac-27f2e7e963f3-metrics-certs podName:90afa1c1-9879-49ee-9fac-27f2e7e963f3 nodeName:}" failed. No retries permitted until 2026-07-30 00:18:06.584900273 +0000 UTC m=+50.643656225 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/90afa1c1-9879-49ee-9fac-27f2e7e963f3-metrics-certs") pod "router-default-877bf98d5-xbkfd" (UID: "90afa1c1-9879-49ee-9fac-27f2e7e963f3") : secret "router-metrics-certs-default" not found Jul 30 00:17:58.585152 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:58.584947 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/90afa1c1-9879-49ee-9fac-27f2e7e963f3-service-ca-bundle podName:90afa1c1-9879-49ee-9fac-27f2e7e963f3 nodeName:}" failed. No retries permitted until 2026-07-30 00:18:06.584938604 +0000 UTC m=+50.643694550 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/90afa1c1-9879-49ee-9fac-27f2e7e963f3-service-ca-bundle") pod "router-default-877bf98d5-xbkfd" (UID: "90afa1c1-9879-49ee-9fac-27f2e7e963f3") : configmap references non-existent config key: service-ca.crt Jul 30 00:17:58.585152 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:58.584976 2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 30 00:17:58.585152 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:58.584998 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f7d33879-e015-4cfe-8d19-11cbc98c1de2-registry-tls\") pod \"image-registry-596d7786c9-nrjxz\" (UID: \"f7d33879-e015-4cfe-8d19-11cbc98c1de2\") " pod="openshift-image-registry/image-registry-596d7786c9-nrjxz" Jul 30 00:17:58.585152 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:58.585018 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/866649cb-1758-4709-82af-f07c8abadf4b-cluster-monitoring-operator-tls podName:866649cb-1758-4709-82af-f07c8abadf4b nodeName:}" failed. No retries permitted until 2026-07-30 00:18:06.585006326 +0000 UTC m=+50.643762280 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/866649cb-1758-4709-82af-f07c8abadf4b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-phzr9" (UID: "866649cb-1758-4709-82af-f07c8abadf4b") : secret "cluster-monitoring-operator-tls" not found Jul 30 00:17:58.585152 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:58.585063 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 30 00:17:58.585152 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:58.585074 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-596d7786c9-nrjxz: secret "image-registry-tls" not found Jul 30 00:17:58.585152 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:58.585101 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f7d33879-e015-4cfe-8d19-11cbc98c1de2-registry-tls podName:f7d33879-e015-4cfe-8d19-11cbc98c1de2 nodeName:}" failed. No retries permitted until 2026-07-30 00:18:06.585094849 +0000 UTC m=+50.643850795 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f7d33879-e015-4cfe-8d19-11cbc98c1de2-registry-tls") pod "image-registry-596d7786c9-nrjxz" (UID: "f7d33879-e015-4cfe-8d19-11cbc98c1de2") : secret "image-registry-tls" not found Jul 30 00:17:58.685468 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:58.685426 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166-metrics-tls\") pod \"dns-default-6kh2h\" (UID: \"bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166\") " pod="openshift-dns/dns-default-6kh2h" Jul 30 00:17:58.685659 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:58.685489 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ffe2ebcf-0fa0-4863-ac2f-568e509d8103-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-ggg5l\" (UID: \"ffe2ebcf-0fa0-4863-ac2f-568e509d8103\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-ggg5l" Jul 30 00:17:58.685659 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:58.685511 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5ca5f4f2-1c36-4eeb-a43c-8533b4d52e84-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-tx22w\" (UID: \"5ca5f4f2-1c36-4eeb-a43c-8533b4d52e84\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-tx22w" Jul 30 00:17:58.685659 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:58.685575 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 00:17:58.685659 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:58.685620 2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 30 00:17:58.685659 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:58.685640 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166-metrics-tls podName:bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166 nodeName:}" failed. No retries permitted until 2026-07-30 00:18:06.685620133 +0000 UTC m=+50.744376089 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166-metrics-tls") pod "dns-default-6kh2h" (UID: "bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166") : secret "dns-default-metrics-tls" not found Jul 30 00:17:58.685659 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:58.685646 2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 30 00:17:58.685903 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:58.685669 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffe2ebcf-0fa0-4863-ac2f-568e509d8103-samples-operator-tls podName:ffe2ebcf-0fa0-4863-ac2f-568e509d8103 nodeName:}" failed. No retries permitted until 2026-07-30 00:18:06.685657426 +0000 UTC m=+50.744413371 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ffe2ebcf-0fa0-4863-ac2f-568e509d8103-samples-operator-tls") pod "cluster-samples-operator-6748489656-ggg5l" (UID: "ffe2ebcf-0fa0-4863-ac2f-568e509d8103") : secret "samples-operator-tls" not found Jul 30 00:17:58.685903 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:17:58.685687 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ca5f4f2-1c36-4eeb-a43c-8533b4d52e84-networking-console-plugin-cert podName:5ca5f4f2-1c36-4eeb-a43c-8533b4d52e84 nodeName:}" failed. No retries permitted until 2026-07-30 00:18:06.685676446 +0000 UTC m=+50.744432398 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/5ca5f4f2-1c36-4eeb-a43c-8533b4d52e84-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-tx22w" (UID: "5ca5f4f2-1c36-4eeb-a43c-8533b4d52e84") : secret "networking-console-plugin-cert" not found Jul 30 00:17:59.431812 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:59.431779 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:17:59.531932 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:17:59.531893 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:01.632196 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:01.632157 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-8sgzj" event={"ID":"cc1eb41e-5e04-4e76-8924-363508ec319b","Type":"ContainerStarted","Data":"79f33dd3bd4737f06003e389be3a55567a411be8b4210b7cd96e962c500c60b9"} Jul 30 00:18:01.634103 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:01.634023 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c9vdq" event={"ID":"7d591175-5d66-48f9-be3e-c89d48534295","Type":"ContainerStarted","Data":"6fee375cdddfb6067431338f61470964f87dbd15d41c3fcb599e5d6520ccc54b"} Jul 30 00:18:01.635622 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:01.635597 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-ddm85" event={"ID":"3fa91894-839d-4035-8b46-a787fe957864","Type":"ContainerStarted","Data":"7f6e0b98d84ae97afadf008e3b6734b2c3e9f8f08c1a6a2d2cad89c727b69f8b"} Jul 30 00:18:01.637413 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:01.637387 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-j2fkq" event={"ID":"a9fed7ee-e7db-4403-98d6-37274ab4f0e8","Type":"ContainerStarted","Data":"eb0b42d60a3ad38ab20c1b88ee7fdfa1697befed998f93be3a62a78fd47d7935"} Jul 30 00:18:01.638947 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:01.638926 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-xjl58" event={"ID":"c9caa632-edc8-43e9-ab2a-b29758f8150f","Type":"ContainerStarted","Data":"6e43df617a22688b997aaa58e093dd57ac4d3991b02f4edd8f9dfe74f431e909"} Jul 30 00:18:01.640672 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:01.640653 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-b92tw_0ef8f120-f712-4ed9-adbd-40f785d4aa9d/console-operator/0.log" Jul 30 00:18:01.640789 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:01.640692 2563 generic.go:356] "Generic (PLEG): container finished" podID="0ef8f120-f712-4ed9-adbd-40f785d4aa9d" containerID="2c0a5aee6fc7f68b613eb3a8da52d99f1dee31fe587ae4fa8788cdb31f619492" exitCode=255 Jul 30 00:18:01.640789 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:01.640717 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-b92tw" event={"ID":"0ef8f120-f712-4ed9-adbd-40f785d4aa9d","Type":"ContainerDied","Data":"2c0a5aee6fc7f68b613eb3a8da52d99f1dee31fe587ae4fa8788cdb31f619492"} Jul 30 00:18:01.640972 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:01.640955 2563 scope.go:117] "RemoveContainer" containerID="2c0a5aee6fc7f68b613eb3a8da52d99f1dee31fe587ae4fa8788cdb31f619492" Jul 30 00:18:01.644091 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:01.644069 2563 generic.go:356] "Generic (PLEG): container finished" podID="f5cd0fd1-6297-45d6-9823-cd424bce4185" containerID="e501cdecfe2d6d3db0b84ffc80d9594d8fe1da42754bee762596cffd2db50492" exitCode=0 Jul 30 00:18:01.644198 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:01.644155 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zx7wk" event={"ID":"f5cd0fd1-6297-45d6-9823-cd424bce4185","Type":"ContainerDied","Data":"e501cdecfe2d6d3db0b84ffc80d9594d8fe1da42754bee762596cffd2db50492"} Jul 30 00:18:01.646490 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:01.646116 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qg9tq" event={"ID":"7d7bcf0e-36d5-488a-aa06-817c6cf5831b","Type":"ContainerStarted","Data":"35709d3ef5fd80f6a33a221d59ea578f99589c1ac3c1da645aa50d095a8f9a24"} Jul 30 00:18:01.648846 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:01.648485 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-8sgzj" podStartSLOduration=27.519946865 podStartE2EDuration="36.648470896s" totalImagesPullingTime="9.128524031s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:17:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:18:01.648246889 +0000 UTC m=+45.707002868" watchObservedRunningTime="2026-07-30 00:18:01.648470896 +0000 UTC m=+45.707226866" Jul 30 00:18:01.666002 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:01.665912 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c9vdq" podStartSLOduration=27.495336607 podStartE2EDuration="36.665897841s" totalImagesPullingTime="9.170561234s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:17:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:18:01.665524854 +0000 UTC m=+45.724280823" watchObservedRunningTime="2026-07-30 00:18:01.665897841 +0000 UTC m=+45.724653809" Jul 30 00:18:01.722580 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:01.720483 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-j2fkq" podStartSLOduration=27.583073325 podStartE2EDuration="36.720463376s" totalImagesPullingTime="9.137390051s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:17:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:18:01.716501181 +0000 UTC m=+45.775257150" watchObservedRunningTime="2026-07-30 00:18:01.720463376 +0000 UTC m=+45.779219349" Jul 30 00:18:01.735679 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:01.735200 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-qg9tq" podStartSLOduration=2.601731787 podStartE2EDuration="11.735179982s" totalImagesPullingTime="9.133448195s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:17:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:18:01.734560082 +0000 UTC m=+45.793316048" watchObservedRunningTime="2026-07-30 00:18:01.735179982 +0000 UTC m=+45.793935951" Jul 30 00:18:01.781348 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:01.781173 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-ddm85" podStartSLOduration=27.538810915 podStartE2EDuration="36.781151955s" totalImagesPullingTime="9.24234104s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:17:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:18:01.780644538 +0000 UTC m=+45.839400507" watchObservedRunningTime="2026-07-30 00:18:01.781151955 +0000 UTC m=+45.839907924" Jul 30 00:18:01.800799 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:01.800449 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-xjl58" podStartSLOduration=27.662027746 podStartE2EDuration="36.800424972s" totalImagesPullingTime="9.138397226s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:17:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:18:01.797270911 +0000 UTC m=+45.856026879" watchObservedRunningTime="2026-07-30 00:18:01.800424972 +0000 UTC m=+45.859180941" Jul 30 00:18:01.808911 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:01.808836 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:01.877527 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:01.876081 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-t7sqw"] Jul 30 00:18:01.889207 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:01.889123 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-t7sqw" Jul 30 00:18:01.922601 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:01.922572 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/78222dee-1367-47b9-a2bc-e97144bbeadc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t7sqw\" (UID: \"78222dee-1367-47b9-a2bc-e97144bbeadc\") " pod="openshift-insights/insights-runtime-extractor-t7sqw" Jul 30 00:18:01.922774 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:01.922629 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/78222dee-1367-47b9-a2bc-e97144bbeadc-data-volume\") pod \"insights-runtime-extractor-t7sqw\" (UID: \"78222dee-1367-47b9-a2bc-e97144bbeadc\") " pod="openshift-insights/insights-runtime-extractor-t7sqw" Jul 30 00:18:01.922774 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:01.922717 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nt5r\" (UniqueName: \"kubernetes.io/projected/78222dee-1367-47b9-a2bc-e97144bbeadc-kube-api-access-6nt5r\") pod \"insights-runtime-extractor-t7sqw\" (UID: \"78222dee-1367-47b9-a2bc-e97144bbeadc\") " pod="openshift-insights/insights-runtime-extractor-t7sqw" Jul 30 00:18:01.922894 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:01.922815 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/78222dee-1367-47b9-a2bc-e97144bbeadc-crio-socket\") pod \"insights-runtime-extractor-t7sqw\" (UID: \"78222dee-1367-47b9-a2bc-e97144bbeadc\") " pod="openshift-insights/insights-runtime-extractor-t7sqw" Jul 30 00:18:01.922894 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:01.922849 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/78222dee-1367-47b9-a2bc-e97144bbeadc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t7sqw\" (UID: \"78222dee-1367-47b9-a2bc-e97144bbeadc\") " pod="openshift-insights/insights-runtime-extractor-t7sqw" Jul 30 00:18:01.928709 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:01.923672 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 30 00:18:01.928709 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:01.923684 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-mbp9q\"" Jul 30 00:18:01.929022 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:01.928983 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 30 00:18:01.942796 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:01.942763 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-t7sqw"] Jul 30 00:18:01.969686 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:01.969652 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:02.023333 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:02.023302 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/78222dee-1367-47b9-a2bc-e97144bbeadc-crio-socket\") pod \"insights-runtime-extractor-t7sqw\" (UID: \"78222dee-1367-47b9-a2bc-e97144bbeadc\") " pod="openshift-insights/insights-runtime-extractor-t7sqw" Jul 30 00:18:02.023526 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:02.023344 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/78222dee-1367-47b9-a2bc-e97144bbeadc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t7sqw\" (UID: \"78222dee-1367-47b9-a2bc-e97144bbeadc\") " pod="openshift-insights/insights-runtime-extractor-t7sqw" Jul 30 00:18:02.023526 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:02.023455 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/78222dee-1367-47b9-a2bc-e97144bbeadc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t7sqw\" (UID: \"78222dee-1367-47b9-a2bc-e97144bbeadc\") " pod="openshift-insights/insights-runtime-extractor-t7sqw" Jul 30 00:18:02.023526 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:02.023498 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/78222dee-1367-47b9-a2bc-e97144bbeadc-data-volume\") pod \"insights-runtime-extractor-t7sqw\" (UID: \"78222dee-1367-47b9-a2bc-e97144bbeadc\") " pod="openshift-insights/insights-runtime-extractor-t7sqw" Jul 30 00:18:02.023682 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:02.023527 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/78222dee-1367-47b9-a2bc-e97144bbeadc-crio-socket\") pod \"insights-runtime-extractor-t7sqw\" (UID: \"78222dee-1367-47b9-a2bc-e97144bbeadc\") " pod="openshift-insights/insights-runtime-extractor-t7sqw" Jul 30 00:18:02.023682 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:02.023540 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:02.023682 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:02.023580 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nt5r\" (UniqueName: \"kubernetes.io/projected/78222dee-1367-47b9-a2bc-e97144bbeadc-kube-api-access-6nt5r\") pod \"insights-runtime-extractor-t7sqw\" (UID: \"78222dee-1367-47b9-a2bc-e97144bbeadc\") " pod="openshift-insights/insights-runtime-extractor-t7sqw" Jul 30 00:18:02.023682 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:18:02.023600 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 30 00:18:02.023682 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:18:02.023661 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/78222dee-1367-47b9-a2bc-e97144bbeadc-kube-rbac-proxy-cm podName:78222dee-1367-47b9-a2bc-e97144bbeadc nodeName:}" failed. No retries permitted until 2026-07-30 00:18:02.523642486 +0000 UTC m=+46.582398448 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/78222dee-1367-47b9-a2bc-e97144bbeadc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t7sqw" (UID: "78222dee-1367-47b9-a2bc-e97144bbeadc") : configmap "kube-rbac-proxy" not found Jul 30 00:18:02.023682 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:18:02.023667 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 30 00:18:02.024015 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:18:02.023702 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78222dee-1367-47b9-a2bc-e97144bbeadc-insights-runtime-extractor-tls podName:78222dee-1367-47b9-a2bc-e97144bbeadc nodeName:}" failed. No retries permitted until 2026-07-30 00:18:02.523691736 +0000 UTC m=+46.582447693 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/78222dee-1367-47b9-a2bc-e97144bbeadc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t7sqw" (UID: "78222dee-1367-47b9-a2bc-e97144bbeadc") : secret "insights-runtime-extractor-tls" not found Jul 30 00:18:02.024078 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:02.024042 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/78222dee-1367-47b9-a2bc-e97144bbeadc-data-volume\") pod \"insights-runtime-extractor-t7sqw\" (UID: \"78222dee-1367-47b9-a2bc-e97144bbeadc\") " pod="openshift-insights/insights-runtime-extractor-t7sqw" Jul 30 00:18:02.033500 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:02.033479 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nt5r\" (UniqueName: \"kubernetes.io/projected/78222dee-1367-47b9-a2bc-e97144bbeadc-kube-api-access-6nt5r\") pod \"insights-runtime-extractor-t7sqw\" (UID: \"78222dee-1367-47b9-a2bc-e97144bbeadc\") " pod="openshift-insights/insights-runtime-extractor-t7sqw" Jul 30 00:18:02.395719 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:02.395686 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:02.506379 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:02.506338 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:02.512832 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:02.512808 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-t4mlm"] Jul 30 00:18:02.528703 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:02.528673 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/78222dee-1367-47b9-a2bc-e97144bbeadc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t7sqw\" (UID: \"78222dee-1367-47b9-a2bc-e97144bbeadc\") " pod="openshift-insights/insights-runtime-extractor-t7sqw" Jul 30 00:18:02.528882 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:18:02.528826 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 30 00:18:02.528933 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:18:02.528887 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/78222dee-1367-47b9-a2bc-e97144bbeadc-kube-rbac-proxy-cm podName:78222dee-1367-47b9-a2bc-e97144bbeadc nodeName:}" failed. No retries permitted until 2026-07-30 00:18:03.528869505 +0000 UTC m=+47.587625470 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/78222dee-1367-47b9-a2bc-e97144bbeadc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t7sqw" (UID: "78222dee-1367-47b9-a2bc-e97144bbeadc") : configmap "kube-rbac-proxy" not found Jul 30 00:18:02.528933 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:02.528822 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/78222dee-1367-47b9-a2bc-e97144bbeadc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t7sqw\" (UID: \"78222dee-1367-47b9-a2bc-e97144bbeadc\") " pod="openshift-insights/insights-runtime-extractor-t7sqw" Jul 30 00:18:02.528933 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:18:02.528898 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 30 00:18:02.529056 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:18:02.528937 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78222dee-1367-47b9-a2bc-e97144bbeadc-insights-runtime-extractor-tls podName:78222dee-1367-47b9-a2bc-e97144bbeadc nodeName:}" failed. No retries permitted until 2026-07-30 00:18:03.528926338 +0000 UTC m=+47.587682304 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/78222dee-1367-47b9-a2bc-e97144bbeadc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t7sqw" (UID: "78222dee-1367-47b9-a2bc-e97144bbeadc") : secret "insights-runtime-extractor-tls" not found Jul 30 00:18:02.559013 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:02.558982 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-t4mlm"] Jul 30 00:18:02.559151 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:02.559091 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-t4mlm" Jul 30 00:18:02.561654 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:02.561637 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 30 00:18:02.562603 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:02.562587 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 30 00:18:02.562664 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:02.562610 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-58cgr\"" Jul 30 00:18:02.581971 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:02.581943 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:02.629706 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:02.629669 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvtps\" (UniqueName: \"kubernetes.io/projected/6dd73a3d-6fdf-4ae6-982e-cfea1dffdd79-kube-api-access-vvtps\") pod \"migrator-6865c9967-t4mlm\" (UID: \"6dd73a3d-6fdf-4ae6-982e-cfea1dffdd79\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-t4mlm" Jul 30 00:18:02.644025 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:02.643991 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:02.653573 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:02.653511 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-b92tw_0ef8f120-f712-4ed9-adbd-40f785d4aa9d/console-operator/1.log" Jul 30 00:18:02.654001 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:02.653986 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-b92tw_0ef8f120-f712-4ed9-adbd-40f785d4aa9d/console-operator/0.log" Jul 30 00:18:02.654086 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:02.654023 2563 generic.go:356] "Generic (PLEG): container finished" podID="0ef8f120-f712-4ed9-adbd-40f785d4aa9d" containerID="0f52c9523949e67b478fa6e8b423f78d6e26d166b63e2f80b94b5b5341740fcc" exitCode=255 Jul 30 00:18:02.654147 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:02.654100 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-b92tw" event={"ID":"0ef8f120-f712-4ed9-adbd-40f785d4aa9d","Type":"ContainerDied","Data":"0f52c9523949e67b478fa6e8b423f78d6e26d166b63e2f80b94b5b5341740fcc"} Jul 30 00:18:02.654188 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:02.654152 2563 scope.go:117] "RemoveContainer" containerID="2c0a5aee6fc7f68b613eb3a8da52d99f1dee31fe587ae4fa8788cdb31f619492" Jul 30 00:18:02.654406 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:02.654362 2563 scope.go:117] "RemoveContainer" containerID="0f52c9523949e67b478fa6e8b423f78d6e26d166b63e2f80b94b5b5341740fcc" Jul 30 00:18:02.654593 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:18:02.654569 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-b92tw_openshift-console-operator(0ef8f120-f712-4ed9-adbd-40f785d4aa9d)\"" pod="openshift-console-operator/console-operator-584dcb6b54-b92tw" podUID="0ef8f120-f712-4ed9-adbd-40f785d4aa9d" Jul 30 00:18:02.657574 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:02.657553 2563 generic.go:356] "Generic (PLEG): container finished" podID="f5cd0fd1-6297-45d6-9823-cd424bce4185" containerID="b818736cc29441b4ed1b88061bcf01e48df832002e3852ce7c5674628a655e43" exitCode=0 Jul 30 00:18:02.657662 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:02.657645 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zx7wk" event={"ID":"f5cd0fd1-6297-45d6-9823-cd424bce4185","Type":"ContainerDied","Data":"b818736cc29441b4ed1b88061bcf01e48df832002e3852ce7c5674628a655e43"} Jul 30 00:18:02.731305 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:02.731268 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvtps\" (UniqueName: \"kubernetes.io/projected/6dd73a3d-6fdf-4ae6-982e-cfea1dffdd79-kube-api-access-vvtps\") pod \"migrator-6865c9967-t4mlm\" (UID: \"6dd73a3d-6fdf-4ae6-982e-cfea1dffdd79\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-t4mlm" Jul 30 00:18:02.740135 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:02.740106 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvtps\" (UniqueName: \"kubernetes.io/projected/6dd73a3d-6fdf-4ae6-982e-cfea1dffdd79-kube-api-access-vvtps\") pod \"migrator-6865c9967-t4mlm\" (UID: \"6dd73a3d-6fdf-4ae6-982e-cfea1dffdd79\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-t4mlm" Jul 30 00:18:02.868297 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:02.868263 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-t4mlm" Jul 30 00:18:03.016314 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:03.016280 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-t4mlm"] Jul 30 00:18:03.019143 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:18:03.019118 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dd73a3d_6fdf_4ae6_982e_cfea1dffdd79.slice/crio-fb9caa7a18bb2d9edf093c2ed9e0d0f7a0dd6eab61e851491d23665d54e1f6f1 WatchSource:0}: Error finding container fb9caa7a18bb2d9edf093c2ed9e0d0f7a0dd6eab61e851491d23665d54e1f6f1: Status 404 returned error can't find the container with id fb9caa7a18bb2d9edf093c2ed9e0d0f7a0dd6eab61e851491d23665d54e1f6f1 Jul 30 00:18:03.104529 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:03.104492 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:03.207674 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:03.207589 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:03.373127 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:03.373099 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zlqlw_add0c4dc-1cb0-4fa3-903a-6ce3b4a8b60a/dns-node-resolver/0.log" Jul 30 00:18:03.539224 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:03.539194 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/78222dee-1367-47b9-a2bc-e97144bbeadc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t7sqw\" (UID: \"78222dee-1367-47b9-a2bc-e97144bbeadc\") " pod="openshift-insights/insights-runtime-extractor-t7sqw" Jul 30 00:18:03.539369 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:03.539322 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/78222dee-1367-47b9-a2bc-e97144bbeadc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t7sqw\" (UID: \"78222dee-1367-47b9-a2bc-e97144bbeadc\") " pod="openshift-insights/insights-runtime-extractor-t7sqw" Jul 30 00:18:03.539423 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:18:03.539388 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 30 00:18:03.539458 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:18:03.539444 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 30 00:18:03.539494 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:18:03.539459 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78222dee-1367-47b9-a2bc-e97144bbeadc-insights-runtime-extractor-tls podName:78222dee-1367-47b9-a2bc-e97144bbeadc nodeName:}" failed. No retries permitted until 2026-07-30 00:18:05.539440765 +0000 UTC m=+49.598196728 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/78222dee-1367-47b9-a2bc-e97144bbeadc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t7sqw" (UID: "78222dee-1367-47b9-a2bc-e97144bbeadc") : secret "insights-runtime-extractor-tls" not found Jul 30 00:18:03.539535 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:18:03.539495 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/78222dee-1367-47b9-a2bc-e97144bbeadc-kube-rbac-proxy-cm podName:78222dee-1367-47b9-a2bc-e97144bbeadc nodeName:}" failed. No retries permitted until 2026-07-30 00:18:05.539482018 +0000 UTC m=+49.598237964 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/78222dee-1367-47b9-a2bc-e97144bbeadc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t7sqw" (UID: "78222dee-1367-47b9-a2bc-e97144bbeadc") : configmap "kube-rbac-proxy" not found Jul 30 00:18:03.645288 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:03.645254 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:03.662697 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:03.662672 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-b92tw_0ef8f120-f712-4ed9-adbd-40f785d4aa9d/console-operator/1.log" Jul 30 00:18:03.663081 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:03.663052 2563 scope.go:117] "RemoveContainer" containerID="0f52c9523949e67b478fa6e8b423f78d6e26d166b63e2f80b94b5b5341740fcc" Jul 30 00:18:03.663259 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:18:03.663237 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-b92tw_openshift-console-operator(0ef8f120-f712-4ed9-adbd-40f785d4aa9d)\"" pod="openshift-console-operator/console-operator-584dcb6b54-b92tw" podUID="0ef8f120-f712-4ed9-adbd-40f785d4aa9d" Jul 30 00:18:03.670584 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:03.670556 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zx7wk" event={"ID":"f5cd0fd1-6297-45d6-9823-cd424bce4185","Type":"ContainerStarted","Data":"75e8d1f4dbce481efcf6b25b0234f49e8eec48225fd2949b42c76d089846fc72"} Jul 30 00:18:03.671689 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:03.671666 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-t4mlm" event={"ID":"6dd73a3d-6fdf-4ae6-982e-cfea1dffdd79","Type":"ContainerStarted","Data":"fb9caa7a18bb2d9edf093c2ed9e0d0f7a0dd6eab61e851491d23665d54e1f6f1"} Jul 30 00:18:03.703940 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:03.703898 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-zx7wk" podStartSLOduration=9.385196716 podStartE2EDuration="47.703885524s" totalImagesPullingTime="38.318688808s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 00:17:16 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:18:03.703618122 +0000 UTC m=+47.762374090" watchObservedRunningTime="2026-07-30 00:18:03.703885524 +0000 UTC m=+47.762641553" Jul 30 00:18:03.706292 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:03.706271 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:03.855791 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:03.855684 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:03.977168 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:03.977134 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gp6l9_756ee6d1-bf62-49db-954a-ed5d66384c54/node-ca/0.log" Jul 30 00:18:03.977414 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:03.977389 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:04.706655 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:04.706624 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-dkn5p"] Jul 30 00:18:04.727854 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:04.727829 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-dkn5p"] Jul 30 00:18:04.727998 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:04.727918 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-dkn5p" Jul 30 00:18:04.730305 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:04.730284 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 30 00:18:04.731230 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:04.731210 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 30 00:18:04.731324 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:04.731230 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-4xqnq\"" Jul 30 00:18:04.731324 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:04.731211 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 30 00:18:04.731324 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:04.731215 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 30 00:18:04.745101 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:04.745074 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:04.773962 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:04.773932 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qg9tq_7d7bcf0e-36d5-488a-aa06-817c6cf5831b/serve-healthcheck-canary/0.log" Jul 30 00:18:04.814235 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:04.814197 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:04.852761 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:04.852698 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/7f3b90ea-8eca-4a10-b0e4-c85ea7887be2-signing-key\") pod \"service-ca-658d7b7c78-dkn5p\" (UID: \"7f3b90ea-8eca-4a10-b0e4-c85ea7887be2\") " pod="openshift-service-ca/service-ca-658d7b7c78-dkn5p" Jul 30 00:18:04.852925 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:04.852907 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vv6sj\" (UniqueName: \"kubernetes.io/projected/7f3b90ea-8eca-4a10-b0e4-c85ea7887be2-kube-api-access-vv6sj\") pod \"service-ca-658d7b7c78-dkn5p\" (UID: \"7f3b90ea-8eca-4a10-b0e4-c85ea7887be2\") " pod="openshift-service-ca/service-ca-658d7b7c78-dkn5p" Jul 30 00:18:04.852983 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:04.852965 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/7f3b90ea-8eca-4a10-b0e4-c85ea7887be2-signing-cabundle\") pod \"service-ca-658d7b7c78-dkn5p\" (UID: \"7f3b90ea-8eca-4a10-b0e4-c85ea7887be2\") " pod="openshift-service-ca/service-ca-658d7b7c78-dkn5p" Jul 30 00:18:04.953957 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:04.953918 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/7f3b90ea-8eca-4a10-b0e4-c85ea7887be2-signing-key\") pod \"service-ca-658d7b7c78-dkn5p\" (UID: \"7f3b90ea-8eca-4a10-b0e4-c85ea7887be2\") " pod="openshift-service-ca/service-ca-658d7b7c78-dkn5p" Jul 30 00:18:04.954101 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:04.954018 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vv6sj\" (UniqueName: \"kubernetes.io/projected/7f3b90ea-8eca-4a10-b0e4-c85ea7887be2-kube-api-access-vv6sj\") pod \"service-ca-658d7b7c78-dkn5p\" (UID: \"7f3b90ea-8eca-4a10-b0e4-c85ea7887be2\") " pod="openshift-service-ca/service-ca-658d7b7c78-dkn5p" Jul 30 00:18:04.954101 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:04.954055 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/7f3b90ea-8eca-4a10-b0e4-c85ea7887be2-signing-cabundle\") pod \"service-ca-658d7b7c78-dkn5p\" (UID: \"7f3b90ea-8eca-4a10-b0e4-c85ea7887be2\") " pod="openshift-service-ca/service-ca-658d7b7c78-dkn5p" Jul 30 00:18:04.954782 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:04.954764 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/7f3b90ea-8eca-4a10-b0e4-c85ea7887be2-signing-cabundle\") pod \"service-ca-658d7b7c78-dkn5p\" (UID: \"7f3b90ea-8eca-4a10-b0e4-c85ea7887be2\") " pod="openshift-service-ca/service-ca-658d7b7c78-dkn5p" Jul 30 00:18:04.956322 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:04.956297 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/7f3b90ea-8eca-4a10-b0e4-c85ea7887be2-signing-key\") pod \"service-ca-658d7b7c78-dkn5p\" (UID: \"7f3b90ea-8eca-4a10-b0e4-c85ea7887be2\") " pod="openshift-service-ca/service-ca-658d7b7c78-dkn5p" Jul 30 00:18:04.961748 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:04.961680 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vv6sj\" (UniqueName: \"kubernetes.io/projected/7f3b90ea-8eca-4a10-b0e4-c85ea7887be2-kube-api-access-vv6sj\") pod \"service-ca-658d7b7c78-dkn5p\" (UID: \"7f3b90ea-8eca-4a10-b0e4-c85ea7887be2\") " pod="openshift-service-ca/service-ca-658d7b7c78-dkn5p" Jul 30 00:18:05.037574 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:05.037533 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-dkn5p" Jul 30 00:18:05.170265 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:05.170233 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-dkn5p"] Jul 30 00:18:05.173484 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:18:05.173456 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f3b90ea_8eca_4a10_b0e4_c85ea7887be2.slice/crio-692f260b2bdaef2e5f78c725d460acb64d3fa1b224fafd410dc907e25575dc66 WatchSource:0}: Error finding container 692f260b2bdaef2e5f78c725d460acb64d3fa1b224fafd410dc907e25575dc66: Status 404 returned error can't find the container with id 692f260b2bdaef2e5f78c725d460acb64d3fa1b224fafd410dc907e25575dc66 Jul 30 00:18:05.560549 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:05.560513 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/78222dee-1367-47b9-a2bc-e97144bbeadc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t7sqw\" (UID: \"78222dee-1367-47b9-a2bc-e97144bbeadc\") " pod="openshift-insights/insights-runtime-extractor-t7sqw" Jul 30 00:18:05.560788 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:18:05.560695 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 30 00:18:05.560788 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:05.560765 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/78222dee-1367-47b9-a2bc-e97144bbeadc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t7sqw\" (UID: \"78222dee-1367-47b9-a2bc-e97144bbeadc\") " pod="openshift-insights/insights-runtime-extractor-t7sqw" Jul 30 00:18:05.560917 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:18:05.560790 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78222dee-1367-47b9-a2bc-e97144bbeadc-insights-runtime-extractor-tls podName:78222dee-1367-47b9-a2bc-e97144bbeadc nodeName:}" failed. No retries permitted until 2026-07-30 00:18:09.560766935 +0000 UTC m=+53.619522897 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/78222dee-1367-47b9-a2bc-e97144bbeadc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t7sqw" (UID: "78222dee-1367-47b9-a2bc-e97144bbeadc") : secret "insights-runtime-extractor-tls" not found Jul 30 00:18:05.560917 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:18:05.560835 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 30 00:18:05.560917 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:18:05.560888 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/78222dee-1367-47b9-a2bc-e97144bbeadc-kube-rbac-proxy-cm podName:78222dee-1367-47b9-a2bc-e97144bbeadc nodeName:}" failed. No retries permitted until 2026-07-30 00:18:09.560871408 +0000 UTC m=+53.619627371 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/78222dee-1367-47b9-a2bc-e97144bbeadc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t7sqw" (UID: "78222dee-1367-47b9-a2bc-e97144bbeadc") : configmap "kube-rbac-proxy" not found Jul 30 00:18:05.680411 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:05.680372 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-dkn5p" event={"ID":"7f3b90ea-8eca-4a10-b0e4-c85ea7887be2","Type":"ContainerStarted","Data":"c0fa5442b940761e02d5c1134ee6b21df65f3b6f1c7eaa44733e3cbc79e6e7ed"} Jul 30 00:18:05.680589 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:05.680430 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-dkn5p" event={"ID":"7f3b90ea-8eca-4a10-b0e4-c85ea7887be2","Type":"ContainerStarted","Data":"692f260b2bdaef2e5f78c725d460acb64d3fa1b224fafd410dc907e25575dc66"} Jul 30 00:18:05.698011 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:05.697958 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-658d7b7c78-dkn5p" podStartSLOduration=1.697942625 podStartE2EDuration="1.697942625s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:18:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:18:05.69697529 +0000 UTC m=+49.755731264" watchObservedRunningTime="2026-07-30 00:18:05.697942625 +0000 UTC m=+49.756698611" Jul 30 00:18:05.827710 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:05.827624 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:06.603334 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:06.603293 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:06.672572 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:06.672535 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90afa1c1-9879-49ee-9fac-27f2e7e963f3-metrics-certs\") pod \"router-default-877bf98d5-xbkfd\" (UID: \"90afa1c1-9879-49ee-9fac-27f2e7e963f3\") " pod="openshift-ingress/router-default-877bf98d5-xbkfd" Jul 30 00:18:06.672798 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:06.672780 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/866649cb-1758-4709-82af-f07c8abadf4b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-phzr9\" (UID: \"866649cb-1758-4709-82af-f07c8abadf4b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-phzr9" Jul 30 00:18:06.673003 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:06.672983 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90afa1c1-9879-49ee-9fac-27f2e7e963f3-service-ca-bundle\") pod \"router-default-877bf98d5-xbkfd\" (UID: \"90afa1c1-9879-49ee-9fac-27f2e7e963f3\") " pod="openshift-ingress/router-default-877bf98d5-xbkfd" Jul 30 00:18:06.673104 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:06.673092 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f7d33879-e015-4cfe-8d19-11cbc98c1de2-registry-tls\") pod \"image-registry-596d7786c9-nrjxz\" (UID: \"f7d33879-e015-4cfe-8d19-11cbc98c1de2\") " pod="openshift-image-registry/image-registry-596d7786c9-nrjxz" Jul 30 00:18:06.673364 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:18:06.672799 2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 30 00:18:06.673950 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:18:06.672859 2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 30 00:18:06.674076 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:18:06.673351 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 30 00:18:06.674167 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:18:06.674158 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-596d7786c9-nrjxz: secret "image-registry-tls" not found Jul 30 00:18:06.674253 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:18:06.673536 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/90afa1c1-9879-49ee-9fac-27f2e7e963f3-service-ca-bundle podName:90afa1c1-9879-49ee-9fac-27f2e7e963f3 nodeName:}" failed. No retries permitted until 2026-07-30 00:18:22.673518558 +0000 UTC m=+66.732274511 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/90afa1c1-9879-49ee-9fac-27f2e7e963f3-service-ca-bundle") pod "router-default-877bf98d5-xbkfd" (UID: "90afa1c1-9879-49ee-9fac-27f2e7e963f3") : configmap references non-existent config key: service-ca.crt Jul 30 00:18:06.674348 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:18:06.674340 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90afa1c1-9879-49ee-9fac-27f2e7e963f3-metrics-certs podName:90afa1c1-9879-49ee-9fac-27f2e7e963f3 nodeName:}" failed. No retries permitted until 2026-07-30 00:18:22.674324555 +0000 UTC m=+66.733080508 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/90afa1c1-9879-49ee-9fac-27f2e7e963f3-metrics-certs") pod "router-default-877bf98d5-xbkfd" (UID: "90afa1c1-9879-49ee-9fac-27f2e7e963f3") : secret "router-metrics-certs-default" not found Jul 30 00:18:06.674428 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:18:06.674420 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/866649cb-1758-4709-82af-f07c8abadf4b-cluster-monitoring-operator-tls podName:866649cb-1758-4709-82af-f07c8abadf4b nodeName:}" failed. No retries permitted until 2026-07-30 00:18:22.674409383 +0000 UTC m=+66.733165337 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/866649cb-1758-4709-82af-f07c8abadf4b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-phzr9" (UID: "866649cb-1758-4709-82af-f07c8abadf4b") : secret "cluster-monitoring-operator-tls" not found Jul 30 00:18:06.674524 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:18:06.674516 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f7d33879-e015-4cfe-8d19-11cbc98c1de2-registry-tls podName:f7d33879-e015-4cfe-8d19-11cbc98c1de2 nodeName:}" failed. No retries permitted until 2026-07-30 00:18:22.674506317 +0000 UTC m=+66.733262270 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f7d33879-e015-4cfe-8d19-11cbc98c1de2-registry-tls") pod "image-registry-596d7786c9-nrjxz" (UID: "f7d33879-e015-4cfe-8d19-11cbc98c1de2") : secret "image-registry-tls" not found Jul 30 00:18:06.687533 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:06.687500 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-t4mlm" event={"ID":"6dd73a3d-6fdf-4ae6-982e-cfea1dffdd79","Type":"ContainerStarted","Data":"7ccad8b9b5dc8bac66ffcf86da2415b559e1f7d997e79091f7e6240a518fba66"} Jul 30 00:18:06.687649 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:06.687554 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-t4mlm" event={"ID":"6dd73a3d-6fdf-4ae6-982e-cfea1dffdd79","Type":"ContainerStarted","Data":"bd8f43e3b17e2770261ede31288975fd7594731a9996b265bbbb98f7dae0edf5"} Jul 30 00:18:06.703730 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:06.703681 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-t4mlm" podStartSLOduration=1.694810762 podStartE2EDuration="4.703662883s" totalImagesPullingTime="3.008852121s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:18:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:18:06.70254513 +0000 UTC m=+50.761301098" watchObservedRunningTime="2026-07-30 00:18:06.703662883 +0000 UTC m=+50.762418852" Jul 30 00:18:06.774618 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:06.774581 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5ca5f4f2-1c36-4eeb-a43c-8533b4d52e84-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-tx22w\" (UID: \"5ca5f4f2-1c36-4eeb-a43c-8533b4d52e84\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-tx22w" Jul 30 00:18:06.774833 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:06.774798 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166-metrics-tls\") pod \"dns-default-6kh2h\" (UID: \"bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166\") " pod="openshift-dns/dns-default-6kh2h" Jul 30 00:18:06.774906 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:06.774892 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ffe2ebcf-0fa0-4863-ac2f-568e509d8103-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-ggg5l\" (UID: \"ffe2ebcf-0fa0-4863-ac2f-568e509d8103\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-ggg5l" Jul 30 00:18:06.775053 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:18:06.775034 2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 30 00:18:06.775167 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:18:06.775146 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 00:18:06.775233 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:18:06.775225 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166-metrics-tls podName:bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166 nodeName:}" failed. No retries permitted until 2026-07-30 00:18:22.775205872 +0000 UTC m=+66.833961825 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166-metrics-tls") pod "dns-default-6kh2h" (UID: "bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166") : secret "dns-default-metrics-tls" not found Jul 30 00:18:06.775334 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:18:06.775322 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffe2ebcf-0fa0-4863-ac2f-568e509d8103-samples-operator-tls podName:ffe2ebcf-0fa0-4863-ac2f-568e509d8103 nodeName:}" failed. No retries permitted until 2026-07-30 00:18:22.77530444 +0000 UTC m=+66.834060390 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ffe2ebcf-0fa0-4863-ac2f-568e509d8103-samples-operator-tls") pod "cluster-samples-operator-6748489656-ggg5l" (UID: "ffe2ebcf-0fa0-4863-ac2f-568e509d8103") : secret "samples-operator-tls" not found Jul 30 00:18:06.775684 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:18:06.775081 2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 30 00:18:06.775846 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:18:06.775836 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ca5f4f2-1c36-4eeb-a43c-8533b4d52e84-networking-console-plugin-cert podName:5ca5f4f2-1c36-4eeb-a43c-8533b4d52e84 nodeName:}" failed. No retries permitted until 2026-07-30 00:18:22.775821945 +0000 UTC m=+66.834577890 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/5ca5f4f2-1c36-4eeb-a43c-8533b4d52e84-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-tx22w" (UID: "5ca5f4f2-1c36-4eeb-a43c-8533b4d52e84") : secret "networking-console-plugin-cert" not found Jul 30 00:18:06.962687 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:06.962595 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:07.069204 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:07.069163 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:07.162533 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:07.162496 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:07.262495 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:07.262456 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:07.364078 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:07.364044 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:07.550584 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:07.550486 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:07.765872 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:07.765839 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:08.583776 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:08.583729 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:08.765930 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:08.765894 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:08.966728 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:08.966644 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:09.179333 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:09.179298 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:09.563483 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:09.563451 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:09.603462 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:09.603438 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/78222dee-1367-47b9-a2bc-e97144bbeadc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t7sqw\" (UID: \"78222dee-1367-47b9-a2bc-e97144bbeadc\") " pod="openshift-insights/insights-runtime-extractor-t7sqw" Jul 30 00:18:09.603786 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:09.603545 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/78222dee-1367-47b9-a2bc-e97144bbeadc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t7sqw\" (UID: \"78222dee-1367-47b9-a2bc-e97144bbeadc\") " pod="openshift-insights/insights-runtime-extractor-t7sqw" Jul 30 00:18:09.603786 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:18:09.603573 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 30 00:18:09.603786 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:18:09.603625 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78222dee-1367-47b9-a2bc-e97144bbeadc-insights-runtime-extractor-tls podName:78222dee-1367-47b9-a2bc-e97144bbeadc nodeName:}" failed. No retries permitted until 2026-07-30 00:18:17.603611284 +0000 UTC m=+61.662367230 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/78222dee-1367-47b9-a2bc-e97144bbeadc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t7sqw" (UID: "78222dee-1367-47b9-a2bc-e97144bbeadc") : secret "insights-runtime-extractor-tls" not found Jul 30 00:18:09.604020 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:09.604002 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/78222dee-1367-47b9-a2bc-e97144bbeadc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t7sqw\" (UID: \"78222dee-1367-47b9-a2bc-e97144bbeadc\") " pod="openshift-insights/insights-runtime-extractor-t7sqw" Jul 30 00:18:10.164232 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:10.164199 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:10.290459 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:10.290425 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:10.633427 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:10.633393 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:10.762309 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:10.762281 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:10.961590 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:10.961499 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:11.158398 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:11.158359 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:11.182269 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:11.182243 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-b92tw" Jul 30 00:18:11.182425 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:11.182318 2563 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-b92tw" Jul 30 00:18:11.182692 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:11.182679 2563 scope.go:117] "RemoveContainer" containerID="0f52c9523949e67b478fa6e8b423f78d6e26d166b63e2f80b94b5b5341740fcc" Jul 30 00:18:11.182922 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:18:11.182900 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-b92tw_openshift-console-operator(0ef8f120-f712-4ed9-adbd-40f785d4aa9d)\"" pod="openshift-console-operator/console-operator-584dcb6b54-b92tw" podUID="0ef8f120-f712-4ed9-adbd-40f785d4aa9d" Jul 30 00:18:11.258754 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:11.258705 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:11.359219 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:11.359186 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:11.702585 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:11.702519 2563 scope.go:117] "RemoveContainer" containerID="0f52c9523949e67b478fa6e8b423f78d6e26d166b63e2f80b94b5b5341740fcc" Jul 30 00:18:11.702940 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:18:11.702664 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-b92tw_openshift-console-operator(0ef8f120-f712-4ed9-adbd-40f785d4aa9d)\"" pod="openshift-console-operator/console-operator-584dcb6b54-b92tw" podUID="0ef8f120-f712-4ed9-adbd-40f785d4aa9d" Jul 30 00:18:12.134243 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:12.134210 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:12.361291 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:12.361253 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:12.764969 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:12.764935 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:12.865253 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:12.865202 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:12.966109 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:12.966059 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:13.158262 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:13.158173 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:13.358327 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:13.358269 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:14.558782 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:14.558730 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:14.754150 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:14.754109 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:14.955532 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:14.955457 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:15.160140 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:15.160112 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:15.358409 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:15.358377 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:16.359592 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:16.359561 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:16.760452 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:16.760421 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:16.954726 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:16.954694 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:17.152841 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:17.152747 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:17.352038 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:17.351997 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:17.595300 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:17.595275 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bpk2r" Jul 30 00:18:17.683068 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:17.683032 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/78222dee-1367-47b9-a2bc-e97144bbeadc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t7sqw\" (UID: \"78222dee-1367-47b9-a2bc-e97144bbeadc\") " pod="openshift-insights/insights-runtime-extractor-t7sqw" Jul 30 00:18:17.685302 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:17.685276 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/78222dee-1367-47b9-a2bc-e97144bbeadc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t7sqw\" (UID: \"78222dee-1367-47b9-a2bc-e97144bbeadc\") " pod="openshift-insights/insights-runtime-extractor-t7sqw" Jul 30 00:18:17.878072 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:17.877987 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-mbp9q\"" Jul 30 00:18:17.886171 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:17.886146 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-t7sqw" Jul 30 00:18:18.010226 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:18.010194 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-t7sqw"] Jul 30 00:18:18.013988 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:18:18.013962 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78222dee_1367_47b9_a2bc_e97144bbeadc.slice/crio-4efeb7c0ce72f2107764950417c63afa6e1976646976c8225a04c8bad26864cf WatchSource:0}: Error finding container 4efeb7c0ce72f2107764950417c63afa6e1976646976c8225a04c8bad26864cf: Status 404 returned error can't find the container with id 4efeb7c0ce72f2107764950417c63afa6e1976646976c8225a04c8bad26864cf Jul 30 00:18:18.639996 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:18.639953 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:18.729068 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:18.728977 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t7sqw" event={"ID":"78222dee-1367-47b9-a2bc-e97144bbeadc","Type":"ContainerStarted","Data":"4efeb7c0ce72f2107764950417c63afa6e1976646976c8225a04c8bad26864cf"} Jul 30 00:18:18.733080 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:18.732947 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:22.130831 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:22.130791 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8567e26f-2898-4d1b-8000-b8bd3326c9ac-metrics-certs\") pod \"network-metrics-daemon-zxg4x\" (UID: \"8567e26f-2898-4d1b-8000-b8bd3326c9ac\") " pod="openshift-multus/network-metrics-daemon-zxg4x" Jul 30 00:18:22.131187 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:22.130848 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxjc4\" (UniqueName: \"kubernetes.io/projected/3101459e-308a-4623-884f-8f54273f1bfd-kube-api-access-qxjc4\") pod \"network-check-target-z7xxn\" (UID: \"3101459e-308a-4623-884f-8f54273f1bfd\") " pod="openshift-network-diagnostics/network-check-target-z7xxn" Jul 30 00:18:22.131187 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:22.131018 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f099a95a-e24f-4e05-bba1-bccd334f000c-original-pull-secret\") pod \"global-pull-secret-syncer-j26vw\" (UID: \"f099a95a-e24f-4e05-bba1-bccd334f000c\") " pod="kube-system/global-pull-secret-syncer-j26vw" Jul 30 00:18:22.133300 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:22.133274 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxjc4\" (UniqueName: \"kubernetes.io/projected/3101459e-308a-4623-884f-8f54273f1bfd-kube-api-access-qxjc4\") pod \"network-check-target-z7xxn\" (UID: \"3101459e-308a-4623-884f-8f54273f1bfd\") " pod="openshift-network-diagnostics/network-check-target-z7xxn" Jul 30 00:18:22.133589 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:22.133575 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 30 00:18:22.133761 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:22.133729 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 30 00:18:22.143854 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:22.143825 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8567e26f-2898-4d1b-8000-b8bd3326c9ac-metrics-certs\") pod \"network-metrics-daemon-zxg4x\" (UID: \"8567e26f-2898-4d1b-8000-b8bd3326c9ac\") " pod="openshift-multus/network-metrics-daemon-zxg4x" Jul 30 00:18:22.143968 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:22.143900 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f099a95a-e24f-4e05-bba1-bccd334f000c-original-pull-secret\") pod \"global-pull-secret-syncer-j26vw\" (UID: \"f099a95a-e24f-4e05-bba1-bccd334f000c\") " pod="kube-system/global-pull-secret-syncer-j26vw" Jul 30 00:18:22.387994 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:22.387924 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qvw7x\"" Jul 30 00:18:22.396038 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:22.396009 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7xxn" Jul 30 00:18:22.397216 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:22.397196 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-f7pzr\"" Jul 30 00:18:22.401688 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:22.401666 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j26vw" Jul 30 00:18:22.405332 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:22.405315 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zxg4x" Jul 30 00:18:22.737312 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:22.737226 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90afa1c1-9879-49ee-9fac-27f2e7e963f3-metrics-certs\") pod \"router-default-877bf98d5-xbkfd\" (UID: \"90afa1c1-9879-49ee-9fac-27f2e7e963f3\") " pod="openshift-ingress/router-default-877bf98d5-xbkfd" Jul 30 00:18:22.737312 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:22.737279 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/866649cb-1758-4709-82af-f07c8abadf4b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-phzr9\" (UID: \"866649cb-1758-4709-82af-f07c8abadf4b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-phzr9" Jul 30 00:18:22.737548 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:22.737391 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90afa1c1-9879-49ee-9fac-27f2e7e963f3-service-ca-bundle\") pod \"router-default-877bf98d5-xbkfd\" (UID: \"90afa1c1-9879-49ee-9fac-27f2e7e963f3\") " pod="openshift-ingress/router-default-877bf98d5-xbkfd" Jul 30 00:18:22.737548 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:22.737429 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f7d33879-e015-4cfe-8d19-11cbc98c1de2-registry-tls\") pod \"image-registry-596d7786c9-nrjxz\" (UID: \"f7d33879-e015-4cfe-8d19-11cbc98c1de2\") " pod="openshift-image-registry/image-registry-596d7786c9-nrjxz" Jul 30 00:18:22.738332 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:22.738308 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90afa1c1-9879-49ee-9fac-27f2e7e963f3-service-ca-bundle\") pod \"router-default-877bf98d5-xbkfd\" (UID: \"90afa1c1-9879-49ee-9fac-27f2e7e963f3\") " pod="openshift-ingress/router-default-877bf98d5-xbkfd" Jul 30 00:18:22.740098 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:22.740072 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f7d33879-e015-4cfe-8d19-11cbc98c1de2-registry-tls\") pod \"image-registry-596d7786c9-nrjxz\" (UID: \"f7d33879-e015-4cfe-8d19-11cbc98c1de2\") " pod="openshift-image-registry/image-registry-596d7786c9-nrjxz" Jul 30 00:18:22.740236 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:22.740154 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90afa1c1-9879-49ee-9fac-27f2e7e963f3-metrics-certs\") pod \"router-default-877bf98d5-xbkfd\" (UID: \"90afa1c1-9879-49ee-9fac-27f2e7e963f3\") " pod="openshift-ingress/router-default-877bf98d5-xbkfd" Jul 30 00:18:22.740236 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:22.740183 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/866649cb-1758-4709-82af-f07c8abadf4b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-phzr9\" (UID: \"866649cb-1758-4709-82af-f07c8abadf4b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-phzr9" Jul 30 00:18:22.838617 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:22.838575 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ffe2ebcf-0fa0-4863-ac2f-568e509d8103-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-ggg5l\" (UID: \"ffe2ebcf-0fa0-4863-ac2f-568e509d8103\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-ggg5l" Jul 30 00:18:22.838839 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:22.838632 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5ca5f4f2-1c36-4eeb-a43c-8533b4d52e84-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-tx22w\" (UID: \"5ca5f4f2-1c36-4eeb-a43c-8533b4d52e84\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-tx22w" Jul 30 00:18:22.838839 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:22.838714 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166-metrics-tls\") pod \"dns-default-6kh2h\" (UID: \"bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166\") " pod="openshift-dns/dns-default-6kh2h" Jul 30 00:18:22.841617 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:22.841563 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166-metrics-tls\") pod \"dns-default-6kh2h\" (UID: \"bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166\") " pod="openshift-dns/dns-default-6kh2h" Jul 30 00:18:22.841617 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:22.841589 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ffe2ebcf-0fa0-4863-ac2f-568e509d8103-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-ggg5l\" (UID: \"ffe2ebcf-0fa0-4863-ac2f-568e509d8103\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-ggg5l" Jul 30 00:18:22.841804 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:22.841631 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5ca5f4f2-1c36-4eeb-a43c-8533b4d52e84-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-tx22w\" (UID: \"5ca5f4f2-1c36-4eeb-a43c-8533b4d52e84\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-tx22w" Jul 30 00:18:22.951177 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:22.942366 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-ndhmg\"" Jul 30 00:18:22.951177 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:22.949834 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-phzr9" Jul 30 00:18:22.953250 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:22.953047 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-s7c9v\"" Jul 30 00:18:22.961056 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:22.960776 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-596d7786c9-nrjxz" Jul 30 00:18:22.963049 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:22.962837 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-nspqp\"" Jul 30 00:18:22.970757 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:22.970598 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-877bf98d5-xbkfd" Jul 30 00:18:22.990570 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:22.990461 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-twxtz\"" Jul 30 00:18:22.999166 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:22.998952 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6kh2h" Jul 30 00:18:23.012586 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:23.012539 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-l669n\"" Jul 30 00:18:23.019649 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:23.019426 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-ggg5l" Jul 30 00:18:23.037030 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:23.036799 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-976dl\"" Jul 30 00:18:23.043877 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:23.043832 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-tx22w" Jul 30 00:18:23.067979 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:23.067468 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zxg4x"] Jul 30 00:18:23.124994 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:23.124781 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-z7xxn"] Jul 30 00:18:23.137224 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:18:23.137168 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3101459e_308a_4623_884f_8f54273f1bfd.slice/crio-ca95049f3a5e2dfa6470ab89da4c8b18fd3b980475f91d7d2f25c1c563ad0d8d WatchSource:0}: Error finding container ca95049f3a5e2dfa6470ab89da4c8b18fd3b980475f91d7d2f25c1c563ad0d8d: Status 404 returned error can't find the container with id ca95049f3a5e2dfa6470ab89da4c8b18fd3b980475f91d7d2f25c1c563ad0d8d Jul 30 00:18:23.224364 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:23.224217 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-877bf98d5-xbkfd"] Jul 30 00:18:23.244344 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:23.243531 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-596d7786c9-nrjxz"] Jul 30 00:18:23.274606 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:23.274537 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6kh2h"] Jul 30 00:18:23.283632 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:18:23.283590 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb9a9f0d_f590_4e7f_91b6_7e3eb5b5a166.slice/crio-29b4fb00ecf192cac90a8d0afb7e1f1acb8bdef323d6d2c0cd36f53a09b315af WatchSource:0}: Error finding container 29b4fb00ecf192cac90a8d0afb7e1f1acb8bdef323d6d2c0cd36f53a09b315af: Status 404 returned error can't find the container with id 29b4fb00ecf192cac90a8d0afb7e1f1acb8bdef323d6d2c0cd36f53a09b315af Jul 30 00:18:23.293200 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:23.293123 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-phzr9"] Jul 30 00:18:23.295116 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:23.295091 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-ggg5l"] Jul 30 00:18:23.296472 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:18:23.296445 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod866649cb_1758_4709_82af_f07c8abadf4b.slice/crio-c1d015a84d5aae345c4a1f9a794402148ed6be756b68c42e784f5909e04e55f8 WatchSource:0}: Error finding container c1d015a84d5aae345c4a1f9a794402148ed6be756b68c42e784f5909e04e55f8: Status 404 returned error can't find the container with id c1d015a84d5aae345c4a1f9a794402148ed6be756b68c42e784f5909e04e55f8 Jul 30 00:18:23.302425 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:23.302384 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-j26vw"] Jul 30 00:18:23.328581 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:23.328560 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-tx22w"] Jul 30 00:18:23.331517 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:18:23.331488 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ca5f4f2_1c36_4eeb_a43c_8533b4d52e84.slice/crio-2e13b8a1a75870ea3559e2416f23b3105ba1c351f29c1b7a19d3b6e04b1321d5 WatchSource:0}: Error finding container 2e13b8a1a75870ea3559e2416f23b3105ba1c351f29c1b7a19d3b6e04b1321d5: Status 404 returned error can't find the container with id 2e13b8a1a75870ea3559e2416f23b3105ba1c351f29c1b7a19d3b6e04b1321d5 Jul 30 00:18:23.471098 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:23.471066 2563 scope.go:117] "RemoveContainer" containerID="0f52c9523949e67b478fa6e8b423f78d6e26d166b63e2f80b94b5b5341740fcc" Jul 30 00:18:23.756534 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:23.756452 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zxg4x" event={"ID":"8567e26f-2898-4d1b-8000-b8bd3326c9ac","Type":"ContainerStarted","Data":"bf28ead2a685ee2c5960f03e2e3cdae5d5e5cc9c092e23238ab761e9304e8738"} Jul 30 00:18:23.762797 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:23.762687 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t7sqw" event={"ID":"78222dee-1367-47b9-a2bc-e97144bbeadc","Type":"ContainerStarted","Data":"227fcc99689c6b31c4090618bc9690343fe19745cce53bf59fddaa528df854c5"} Jul 30 00:18:23.766567 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:23.765752 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-877bf98d5-xbkfd" event={"ID":"90afa1c1-9879-49ee-9fac-27f2e7e963f3","Type":"ContainerStarted","Data":"9ce9ff00c748c5da7934b092c1cc81b752926511c9f2c1035bc000015c7ee3c4"} Jul 30 00:18:23.766567 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:23.765796 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-877bf98d5-xbkfd" event={"ID":"90afa1c1-9879-49ee-9fac-27f2e7e963f3","Type":"ContainerStarted","Data":"c5f98f093a75fa97601349faf5c0fe26c39e985d1e6f7d56091591f7f287a404"} Jul 30 00:18:23.767917 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:23.767839 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6kh2h" event={"ID":"bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166","Type":"ContainerStarted","Data":"29b4fb00ecf192cac90a8d0afb7e1f1acb8bdef323d6d2c0cd36f53a09b315af"} Jul 30 00:18:23.772631 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:23.772606 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-z7xxn" event={"ID":"3101459e-308a-4623-884f-8f54273f1bfd","Type":"ContainerStarted","Data":"81cbd7a1882c3c3f94e7e07b6c4076412868db0c1d0c10911858f8d5ad0bc6e9"} Jul 30 00:18:23.772761 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:23.772647 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-z7xxn" event={"ID":"3101459e-308a-4623-884f-8f54273f1bfd","Type":"ContainerStarted","Data":"ca95049f3a5e2dfa6470ab89da4c8b18fd3b980475f91d7d2f25c1c563ad0d8d"} Jul 30 00:18:23.773022 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:23.772967 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-z7xxn" Jul 30 00:18:23.776781 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:23.775920 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-b92tw_0ef8f120-f712-4ed9-adbd-40f785d4aa9d/console-operator/1.log" Jul 30 00:18:23.776781 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:23.775998 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-b92tw" event={"ID":"0ef8f120-f712-4ed9-adbd-40f785d4aa9d","Type":"ContainerStarted","Data":"53afb939c3e7dc59ec02ef3bd43ca0db305253e80f6764f1e580b28bb080b9d3"} Jul 30 00:18:23.776781 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:23.776716 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-b92tw" Jul 30 00:18:23.779408 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:23.779382 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-596d7786c9-nrjxz" event={"ID":"f7d33879-e015-4cfe-8d19-11cbc98c1de2","Type":"ContainerStarted","Data":"e865380c260eb34e1710b5323c9f3f7d6927d2c268b02b34f513fbcc5fafb066"} Jul 30 00:18:23.779524 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:23.779429 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-596d7786c9-nrjxz" event={"ID":"f7d33879-e015-4cfe-8d19-11cbc98c1de2","Type":"ContainerStarted","Data":"314bb4783fded2dcfff2ebb1ff1a06bcc1afd80f6ae5dbf1beb8955028a07a06"} Jul 30 00:18:23.779963 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:23.779945 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-596d7786c9-nrjxz" Jul 30 00:18:23.782084 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:23.782036 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-ggg5l" event={"ID":"ffe2ebcf-0fa0-4863-ac2f-568e509d8103","Type":"ContainerStarted","Data":"cf19e1b4cc3a35609ec747f80cec9bb4ccb8ff270850c098421f0757b8398779"} Jul 30 00:18:23.786025 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:23.783662 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-phzr9" event={"ID":"866649cb-1758-4709-82af-f07c8abadf4b","Type":"ContainerStarted","Data":"c1d015a84d5aae345c4a1f9a794402148ed6be756b68c42e784f5909e04e55f8"} Jul 30 00:18:23.786025 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:23.785025 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-877bf98d5-xbkfd" podStartSLOduration=58.785008682 podStartE2EDuration="58.785008682s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:17:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:18:23.784066986 +0000 UTC m=+67.842822951" watchObservedRunningTime="2026-07-30 00:18:23.785008682 +0000 UTC m=+67.843764651" Jul 30 00:18:23.788754 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:23.788715 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-j26vw" event={"ID":"f099a95a-e24f-4e05-bba1-bccd334f000c","Type":"ContainerStarted","Data":"fbf6c36e7f877b95c6baa05122591caa0e387bb608e3321b0066b64f92d874bd"} Jul 30 00:18:23.791369 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:23.791347 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-tx22w" event={"ID":"5ca5f4f2-1c36-4eeb-a43c-8533b4d52e84","Type":"ContainerStarted","Data":"2e13b8a1a75870ea3559e2416f23b3105ba1c351f29c1b7a19d3b6e04b1321d5"} Jul 30 00:18:23.800090 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:23.799049 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-z7xxn" podStartSLOduration=67.799032294 podStartE2EDuration="1m7.799032294s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:17:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:18:23.798206948 +0000 UTC m=+67.856962913" watchObservedRunningTime="2026-07-30 00:18:23.799032294 +0000 UTC m=+67.857788263" Jul 30 00:18:23.814217 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:23.813828 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-b92tw" podStartSLOduration=49.3753662 podStartE2EDuration="58.813818s" totalImagesPullingTime="9.4384518s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:17:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:18:23.813674221 +0000 UTC m=+67.872430191" watchObservedRunningTime="2026-07-30 00:18:23.813818 +0000 UTC m=+67.872573970" Jul 30 00:18:23.834786 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:23.834717 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-596d7786c9-nrjxz" podStartSLOduration=66.834698194 podStartE2EDuration="1m6.834698194s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:17:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:18:23.832976292 +0000 UTC m=+67.891732261" watchObservedRunningTime="2026-07-30 00:18:23.834698194 +0000 UTC m=+67.893454165" Jul 30 00:18:23.970987 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:23.970918 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-877bf98d5-xbkfd" Jul 30 00:18:23.974768 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:23.974663 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-877bf98d5-xbkfd" Jul 30 00:18:24.392273 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.392243 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-b92tw" Jul 30 00:18:24.566563 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.566533 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-56466cd794-4l5lg"] Jul 30 00:18:24.574590 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.574468 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-77947dbf5d-c5gzl"] Jul 30 00:18:24.575815 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.575457 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-56466cd794-4l5lg" Jul 30 00:18:24.580385 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.579124 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 30 00:18:24.580385 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.579392 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-kg9nf\"" Jul 30 00:18:24.580385 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.579844 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 30 00:18:24.580385 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.580083 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 30 00:18:24.580385 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.580333 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 30 00:18:24.600235 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.585009 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c56b75766-z8lc8"] Jul 30 00:18:24.600235 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.587040 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77947dbf5d-c5gzl" Jul 30 00:18:24.600235 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.589651 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c56b75766-z8lc8" Jul 30 00:18:24.600235 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.590881 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 30 00:18:24.600235 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.591077 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-56466cd794-4l5lg"] Jul 30 00:18:24.600235 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.592430 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-77947dbf5d-c5gzl"] Jul 30 00:18:24.600235 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.594841 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 30 00:18:24.600235 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.598383 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 30 00:18:24.600235 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.598644 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 30 00:18:24.600235 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.598971 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 30 00:18:24.605639 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.603457 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c56b75766-z8lc8"] Jul 30 00:18:24.639070 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.639029 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:24.693896 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.693820 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-jzwkw"] Jul 30 00:18:24.697350 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.697327 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-jzwkw" Jul 30 00:18:24.700937 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.700913 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 30 00:18:24.701181 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.701163 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-n2pzz\"" Jul 30 00:18:24.704364 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.704321 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 30 00:18:24.730373 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.730135 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-jzwkw"] Jul 30 00:18:24.763975 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.763424 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8b2a01cb-b31d-4a67-91de-c53a3035fca9-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6c56b75766-z8lc8\" (UID: \"8b2a01cb-b31d-4a67-91de-c53a3035fca9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c56b75766-z8lc8" Jul 30 00:18:24.763975 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.763494 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ts2h\" (UniqueName: \"kubernetes.io/projected/8b2a01cb-b31d-4a67-91de-c53a3035fca9-kube-api-access-9ts2h\") pod \"cluster-proxy-proxy-agent-6c56b75766-z8lc8\" (UID: \"8b2a01cb-b31d-4a67-91de-c53a3035fca9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c56b75766-z8lc8" Jul 30 00:18:24.763975 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.763544 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/7aa28d4d-0b2a-4f16-b470-15e3a9216c0d-klusterlet-config\") pod \"klusterlet-addon-workmgr-77947dbf5d-c5gzl\" (UID: \"7aa28d4d-0b2a-4f16-b470-15e3a9216c0d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77947dbf5d-c5gzl" Jul 30 00:18:24.763975 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.763572 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/8b2a01cb-b31d-4a67-91de-c53a3035fca9-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6c56b75766-z8lc8\" (UID: \"8b2a01cb-b31d-4a67-91de-c53a3035fca9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c56b75766-z8lc8" Jul 30 00:18:24.763975 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.763643 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7aa28d4d-0b2a-4f16-b470-15e3a9216c0d-tmp\") pod \"klusterlet-addon-workmgr-77947dbf5d-c5gzl\" (UID: \"7aa28d4d-0b2a-4f16-b470-15e3a9216c0d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77947dbf5d-c5gzl" Jul 30 00:18:24.763975 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.763688 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f3812cb7-d263-4c0f-95ad-b45992af6900-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-56466cd794-4l5lg\" (UID: \"f3812cb7-d263-4c0f-95ad-b45992af6900\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-56466cd794-4l5lg" Jul 30 00:18:24.763975 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.763729 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdssz\" (UniqueName: \"kubernetes.io/projected/f3812cb7-d263-4c0f-95ad-b45992af6900-kube-api-access-cdssz\") pod \"managed-serviceaccount-addon-agent-56466cd794-4l5lg\" (UID: \"f3812cb7-d263-4c0f-95ad-b45992af6900\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-56466cd794-4l5lg" Jul 30 00:18:24.763975 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.763776 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b2a01cb-b31d-4a67-91de-c53a3035fca9-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6c56b75766-z8lc8\" (UID: \"8b2a01cb-b31d-4a67-91de-c53a3035fca9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c56b75766-z8lc8" Jul 30 00:18:24.763975 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.763841 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/8b2a01cb-b31d-4a67-91de-c53a3035fca9-ca\") pod \"cluster-proxy-proxy-agent-6c56b75766-z8lc8\" (UID: \"8b2a01cb-b31d-4a67-91de-c53a3035fca9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c56b75766-z8lc8" Jul 30 00:18:24.763975 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.763865 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/8b2a01cb-b31d-4a67-91de-c53a3035fca9-hub\") pod \"cluster-proxy-proxy-agent-6c56b75766-z8lc8\" (UID: \"8b2a01cb-b31d-4a67-91de-c53a3035fca9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c56b75766-z8lc8" Jul 30 00:18:24.763975 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.763890 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-824hh\" (UniqueName: \"kubernetes.io/projected/7aa28d4d-0b2a-4f16-b470-15e3a9216c0d-kube-api-access-824hh\") pod \"klusterlet-addon-workmgr-77947dbf5d-c5gzl\" (UID: \"7aa28d4d-0b2a-4f16-b470-15e3a9216c0d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77947dbf5d-c5gzl" Jul 30 00:18:24.798200 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.797618 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-877bf98d5-xbkfd" Jul 30 00:18:24.798889 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.798660 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-877bf98d5-xbkfd" Jul 30 00:18:24.859042 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.858976 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:24.866766 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.865267 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b2a01cb-b31d-4a67-91de-c53a3035fca9-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6c56b75766-z8lc8\" (UID: \"8b2a01cb-b31d-4a67-91de-c53a3035fca9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c56b75766-z8lc8" Jul 30 00:18:24.866766 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.865313 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/8b2a01cb-b31d-4a67-91de-c53a3035fca9-ca\") pod \"cluster-proxy-proxy-agent-6c56b75766-z8lc8\" (UID: \"8b2a01cb-b31d-4a67-91de-c53a3035fca9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c56b75766-z8lc8" Jul 30 00:18:24.866766 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.865338 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/8b2a01cb-b31d-4a67-91de-c53a3035fca9-hub\") pod \"cluster-proxy-proxy-agent-6c56b75766-z8lc8\" (UID: \"8b2a01cb-b31d-4a67-91de-c53a3035fca9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c56b75766-z8lc8" Jul 30 00:18:24.866766 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.865362 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-824hh\" (UniqueName: \"kubernetes.io/projected/7aa28d4d-0b2a-4f16-b470-15e3a9216c0d-kube-api-access-824hh\") pod \"klusterlet-addon-workmgr-77947dbf5d-c5gzl\" (UID: \"7aa28d4d-0b2a-4f16-b470-15e3a9216c0d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77947dbf5d-c5gzl" Jul 30 00:18:24.866766 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.865409 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8b2a01cb-b31d-4a67-91de-c53a3035fca9-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6c56b75766-z8lc8\" (UID: \"8b2a01cb-b31d-4a67-91de-c53a3035fca9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c56b75766-z8lc8" Jul 30 00:18:24.866766 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.865469 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ts2h\" (UniqueName: \"kubernetes.io/projected/8b2a01cb-b31d-4a67-91de-c53a3035fca9-kube-api-access-9ts2h\") pod \"cluster-proxy-proxy-agent-6c56b75766-z8lc8\" (UID: \"8b2a01cb-b31d-4a67-91de-c53a3035fca9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c56b75766-z8lc8" Jul 30 00:18:24.866766 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.865513 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/7aa28d4d-0b2a-4f16-b470-15e3a9216c0d-klusterlet-config\") pod \"klusterlet-addon-workmgr-77947dbf5d-c5gzl\" (UID: \"7aa28d4d-0b2a-4f16-b470-15e3a9216c0d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77947dbf5d-c5gzl" Jul 30 00:18:24.866766 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.865542 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/8b2a01cb-b31d-4a67-91de-c53a3035fca9-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6c56b75766-z8lc8\" (UID: \"8b2a01cb-b31d-4a67-91de-c53a3035fca9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c56b75766-z8lc8" Jul 30 00:18:24.866766 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.865578 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jwrp\" (UniqueName: \"kubernetes.io/projected/7ae2552a-b83f-49a5-b779-c80e727f40b4-kube-api-access-7jwrp\") pod \"downloads-7c97dc4555-jzwkw\" (UID: \"7ae2552a-b83f-49a5-b779-c80e727f40b4\") " pod="openshift-console/downloads-7c97dc4555-jzwkw" Jul 30 00:18:24.866766 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.865609 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7aa28d4d-0b2a-4f16-b470-15e3a9216c0d-tmp\") pod \"klusterlet-addon-workmgr-77947dbf5d-c5gzl\" (UID: \"7aa28d4d-0b2a-4f16-b470-15e3a9216c0d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77947dbf5d-c5gzl" Jul 30 00:18:24.866766 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.865637 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f3812cb7-d263-4c0f-95ad-b45992af6900-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-56466cd794-4l5lg\" (UID: \"f3812cb7-d263-4c0f-95ad-b45992af6900\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-56466cd794-4l5lg" Jul 30 00:18:24.866766 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.865674 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdssz\" (UniqueName: \"kubernetes.io/projected/f3812cb7-d263-4c0f-95ad-b45992af6900-kube-api-access-cdssz\") pod \"managed-serviceaccount-addon-agent-56466cd794-4l5lg\" (UID: \"f3812cb7-d263-4c0f-95ad-b45992af6900\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-56466cd794-4l5lg" Jul 30 00:18:24.870818 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.869508 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7aa28d4d-0b2a-4f16-b470-15e3a9216c0d-tmp\") pod \"klusterlet-addon-workmgr-77947dbf5d-c5gzl\" (UID: \"7aa28d4d-0b2a-4f16-b470-15e3a9216c0d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77947dbf5d-c5gzl" Jul 30 00:18:24.871042 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.871002 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b2a01cb-b31d-4a67-91de-c53a3035fca9-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6c56b75766-z8lc8\" (UID: \"8b2a01cb-b31d-4a67-91de-c53a3035fca9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c56b75766-z8lc8" Jul 30 00:18:24.877123 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.877078 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/8b2a01cb-b31d-4a67-91de-c53a3035fca9-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6c56b75766-z8lc8\" (UID: \"8b2a01cb-b31d-4a67-91de-c53a3035fca9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c56b75766-z8lc8" Jul 30 00:18:24.877325 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.877300 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/7aa28d4d-0b2a-4f16-b470-15e3a9216c0d-klusterlet-config\") pod \"klusterlet-addon-workmgr-77947dbf5d-c5gzl\" (UID: \"7aa28d4d-0b2a-4f16-b470-15e3a9216c0d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77947dbf5d-c5gzl" Jul 30 00:18:24.878352 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.878322 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/8b2a01cb-b31d-4a67-91de-c53a3035fca9-hub\") pod \"cluster-proxy-proxy-agent-6c56b75766-z8lc8\" (UID: \"8b2a01cb-b31d-4a67-91de-c53a3035fca9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c56b75766-z8lc8" Jul 30 00:18:24.880893 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.880845 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f3812cb7-d263-4c0f-95ad-b45992af6900-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-56466cd794-4l5lg\" (UID: \"f3812cb7-d263-4c0f-95ad-b45992af6900\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-56466cd794-4l5lg" Jul 30 00:18:24.888149 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.888109 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/8b2a01cb-b31d-4a67-91de-c53a3035fca9-ca\") pod \"cluster-proxy-proxy-agent-6c56b75766-z8lc8\" (UID: \"8b2a01cb-b31d-4a67-91de-c53a3035fca9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c56b75766-z8lc8" Jul 30 00:18:24.888829 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.888789 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8b2a01cb-b31d-4a67-91de-c53a3035fca9-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6c56b75766-z8lc8\" (UID: \"8b2a01cb-b31d-4a67-91de-c53a3035fca9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c56b75766-z8lc8" Jul 30 00:18:24.895797 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.895423 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ts2h\" (UniqueName: \"kubernetes.io/projected/8b2a01cb-b31d-4a67-91de-c53a3035fca9-kube-api-access-9ts2h\") pod \"cluster-proxy-proxy-agent-6c56b75766-z8lc8\" (UID: \"8b2a01cb-b31d-4a67-91de-c53a3035fca9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c56b75766-z8lc8" Jul 30 00:18:24.896244 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.896183 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdssz\" (UniqueName: \"kubernetes.io/projected/f3812cb7-d263-4c0f-95ad-b45992af6900-kube-api-access-cdssz\") pod \"managed-serviceaccount-addon-agent-56466cd794-4l5lg\" (UID: \"f3812cb7-d263-4c0f-95ad-b45992af6900\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-56466cd794-4l5lg" Jul 30 00:18:24.898079 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.898056 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-824hh\" (UniqueName: \"kubernetes.io/projected/7aa28d4d-0b2a-4f16-b470-15e3a9216c0d-kube-api-access-824hh\") pod \"klusterlet-addon-workmgr-77947dbf5d-c5gzl\" (UID: \"7aa28d4d-0b2a-4f16-b470-15e3a9216c0d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77947dbf5d-c5gzl" Jul 30 00:18:24.917690 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.917574 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-56466cd794-4l5lg" Jul 30 00:18:24.934407 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.934382 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77947dbf5d-c5gzl" Jul 30 00:18:24.942890 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.941641 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c56b75766-z8lc8" Jul 30 00:18:24.967843 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.966847 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jwrp\" (UniqueName: \"kubernetes.io/projected/7ae2552a-b83f-49a5-b779-c80e727f40b4-kube-api-access-7jwrp\") pod \"downloads-7c97dc4555-jzwkw\" (UID: \"7ae2552a-b83f-49a5-b779-c80e727f40b4\") " pod="openshift-console/downloads-7c97dc4555-jzwkw" Jul 30 00:18:24.978834 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:24.978790 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jwrp\" (UniqueName: \"kubernetes.io/projected/7ae2552a-b83f-49a5-b779-c80e727f40b4-kube-api-access-7jwrp\") pod \"downloads-7c97dc4555-jzwkw\" (UID: \"7ae2552a-b83f-49a5-b779-c80e727f40b4\") " pod="openshift-console/downloads-7c97dc4555-jzwkw" Jul 30 00:18:25.020715 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:25.020449 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-jzwkw" Jul 30 00:18:25.053106 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:25.053050 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:25.152526 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:25.152440 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:25.249376 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:25.249272 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:25.309350 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:25.309291 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:25.657540 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:25.657471 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:25.940878 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:25.940441 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-56466cd794-4l5lg"] Jul 30 00:18:26.175164 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:18:26.175119 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf3812cb7_d263_4c0f_95ad_b45992af6900.slice/crio-798d43e37bfc69d4aeaf1338b421bb06420344dfde1342218bf27465d2e35531 WatchSource:0}: Error finding container 798d43e37bfc69d4aeaf1338b421bb06420344dfde1342218bf27465d2e35531: Status 404 returned error can't find the container with id 798d43e37bfc69d4aeaf1338b421bb06420344dfde1342218bf27465d2e35531 Jul 30 00:18:26.266637 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:26.266133 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:26.333197 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:26.333172 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-77947dbf5d-c5gzl"] Jul 30 00:18:26.386625 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:26.386586 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:26.490141 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:26.490042 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:26.547416 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:26.547374 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:26.807913 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:26.807877 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-56466cd794-4l5lg" event={"ID":"f3812cb7-d263-4c0f-95ad-b45992af6900","Type":"ContainerStarted","Data":"798d43e37bfc69d4aeaf1338b421bb06420344dfde1342218bf27465d2e35531"} Jul 30 00:18:28.528965 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:28.528931 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:28.598688 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:28.598655 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:28.643350 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:18:28.643319 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7aa28d4d_0b2a_4f16_b470_15e3a9216c0d.slice/crio-e8846c61cac538df5c41472567df11c45bb0faa9adee355fd078cd3668fbfa0c WatchSource:0}: Error finding container e8846c61cac538df5c41472567df11c45bb0faa9adee355fd078cd3668fbfa0c: Status 404 returned error can't find the container with id e8846c61cac538df5c41472567df11c45bb0faa9adee355fd078cd3668fbfa0c Jul 30 00:18:28.762275 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:28.762238 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c56b75766-z8lc8"] Jul 30 00:18:28.817103 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:28.817016 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77947dbf5d-c5gzl" event={"ID":"7aa28d4d-0b2a-4f16-b470-15e3a9216c0d","Type":"ContainerStarted","Data":"e8846c61cac538df5c41472567df11c45bb0faa9adee355fd078cd3668fbfa0c"} Jul 30 00:18:29.622397 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:29.622360 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:29.685652 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:29.685616 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:30.136450 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:30.136396 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:30.194426 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:30.194385 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:30.449405 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:18:30.449314 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8b2a01cb_b31d_4a67_91de_c53a3035fca9.slice/crio-2decd0a78c7f3d1997bf2d13924991b6408e7d3f39ec3c68f4a288982279a5b9 WatchSource:0}: Error finding container 2decd0a78c7f3d1997bf2d13924991b6408e7d3f39ec3c68f4a288982279a5b9: Status 404 returned error can't find the container with id 2decd0a78c7f3d1997bf2d13924991b6408e7d3f39ec3c68f4a288982279a5b9 Jul 30 00:18:30.580833 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:30.580802 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-jzwkw"] Jul 30 00:18:30.827199 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:30.827164 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c56b75766-z8lc8" event={"ID":"8b2a01cb-b31d-4a67-91de-c53a3035fca9","Type":"ContainerStarted","Data":"2decd0a78c7f3d1997bf2d13924991b6408e7d3f39ec3c68f4a288982279a5b9"} Jul 30 00:18:31.265246 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:18:31.265216 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ae2552a_b83f_49a5_b779_c80e727f40b4.slice/crio-3d3d6d1da1f845654ea1c9bbe2cf90027547908d23a01a4fd3c95f339c74559c WatchSource:0}: Error finding container 3d3d6d1da1f845654ea1c9bbe2cf90027547908d23a01a4fd3c95f339c74559c: Status 404 returned error can't find the container with id 3d3d6d1da1f845654ea1c9bbe2cf90027547908d23a01a4fd3c95f339c74559c Jul 30 00:18:31.661611 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:31.661221 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:31.758615 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:31.758543 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:31.835946 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:31.835777 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6kh2h" event={"ID":"bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166","Type":"ContainerStarted","Data":"04350154ad0c9459e051c22776b1432d6b076bac229fc6dd52e73387371d38ed"} Jul 30 00:18:31.839340 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:31.839229 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-ggg5l" event={"ID":"ffe2ebcf-0fa0-4863-ac2f-568e509d8103","Type":"ContainerStarted","Data":"1528378e03aeca86527444ee7a570b269cdb98407e2c7ac8a91cd0461db31064"} Jul 30 00:18:31.842025 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:31.841659 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-phzr9" event={"ID":"866649cb-1758-4709-82af-f07c8abadf4b","Type":"ContainerStarted","Data":"343eb3c9bd11aa0b3a4a0098e16ad3df4267679a1001a87ad621798d712e0195"} Jul 30 00:18:31.844021 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:31.843846 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-j26vw" event={"ID":"f099a95a-e24f-4e05-bba1-bccd334f000c","Type":"ContainerStarted","Data":"c060c433d8b126b3619384e7c34c6cdf6c5c934fb3ac1852d3c0a7f2f2921dbd"} Jul 30 00:18:31.847014 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:31.846923 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-tx22w" event={"ID":"5ca5f4f2-1c36-4eeb-a43c-8533b4d52e84","Type":"ContainerStarted","Data":"3a1f9f4e1c130bec3e3ecdfbd5f127aaff0ac70fc2bf0993f02e29bf5d7dccef"} Jul 30 00:18:31.855092 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:31.854801 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zxg4x" event={"ID":"8567e26f-2898-4d1b-8000-b8bd3326c9ac","Type":"ContainerStarted","Data":"9453d11eaf05dd6126b3da3b83e2add171cae479e474d94c4d7a1154f45d051f"} Jul 30 00:18:31.857823 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:31.857777 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-jzwkw" event={"ID":"7ae2552a-b83f-49a5-b779-c80e727f40b4","Type":"ContainerStarted","Data":"3d3d6d1da1f845654ea1c9bbe2cf90027547908d23a01a4fd3c95f339c74559c"} Jul 30 00:18:31.871977 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:31.871914 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-phzr9" podStartSLOduration=58.708853727 podStartE2EDuration="1m6.871894302s" totalImagesPullingTime="8.163040575s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:17:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:18:31.869819395 +0000 UTC m=+75.928575367" watchObservedRunningTime="2026-07-30 00:18:31.871894302 +0000 UTC m=+75.930650272" Jul 30 00:18:31.895172 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:31.893582 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-j26vw" podStartSLOduration=67.72147392 podStartE2EDuration="1m15.893563741s" totalImagesPullingTime="8.172089821s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 00:17:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:18:31.893031142 +0000 UTC m=+75.951787112" watchObservedRunningTime="2026-07-30 00:18:31.893563741 +0000 UTC m=+75.952319691" Jul 30 00:18:32.054604 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:32.054521 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:32.146723 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:32.146591 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:32.226134 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:32.226079 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:32.672697 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:32.672662 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:32.877626 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:32.877590 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zxg4x" event={"ID":"8567e26f-2898-4d1b-8000-b8bd3326c9ac","Type":"ContainerStarted","Data":"b4302f5f3bfbf5353dc9e194531f60ddb74427c4685d639f00f61933705aee41"} Jul 30 00:18:32.885811 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:32.885775 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t7sqw" event={"ID":"78222dee-1367-47b9-a2bc-e97144bbeadc","Type":"ContainerStarted","Data":"cd53fcfa7576f36133370b4029c07c8f532781b55f4c2079ba133def7c23cc0b"} Jul 30 00:18:32.894283 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:32.894233 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-74c9f884b7-tx22w" podStartSLOduration=40.492832381 podStartE2EDuration="43.89421236s" totalImagesPullingTime="3.401379979s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:17:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:18:31.917221766 +0000 UTC m=+75.975977735" watchObservedRunningTime="2026-07-30 00:18:32.89421236 +0000 UTC m=+76.952968330" Jul 30 00:18:32.894618 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:32.894581 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-zxg4x" podStartSLOduration=73.240106085 podStartE2EDuration="1m16.894572228s" totalImagesPullingTime="3.654466143s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:17:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:18:32.893569798 +0000 UTC m=+76.952325771" watchObservedRunningTime="2026-07-30 00:18:32.894572228 +0000 UTC m=+76.953328197" Jul 30 00:18:32.897321 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:32.897294 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6kh2h" event={"ID":"bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166","Type":"ContainerStarted","Data":"59f11f8440612e1c627c2d742cc4de0413b1de2563c64a3d07e325108e9b2ab5"} Jul 30 00:18:32.898210 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:32.898187 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-6kh2h" Jul 30 00:18:32.906216 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:32.905719 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-ggg5l" event={"ID":"ffe2ebcf-0fa0-4863-ac2f-568e509d8103","Type":"ContainerStarted","Data":"d420d90b5eb8e3f73a3cde8552b07a756bbf1293d3a243724573eeb405cc80cc"} Jul 30 00:18:32.917399 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:32.916120 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-6kh2h" podStartSLOduration=34.740417236 podStartE2EDuration="42.91609937s" totalImagesPullingTime="8.175682134s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:17:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:18:32.913959803 +0000 UTC m=+76.972715774" watchObservedRunningTime="2026-07-30 00:18:32.91609937 +0000 UTC m=+76.974855337" Jul 30 00:18:32.933105 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:32.932882 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-ggg5l" podStartSLOduration=59.804885619 podStartE2EDuration="1m7.93286042s" totalImagesPullingTime="8.127974801s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:17:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:18:32.930882642 +0000 UTC m=+76.989638611" watchObservedRunningTime="2026-07-30 00:18:32.93286042 +0000 UTC m=+76.991616390" Jul 30 00:18:33.018915 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:33.018841 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:33.075788 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:33.075684 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:33.647234 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:33.647177 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:33.700911 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:33.700816 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:35.181721 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:35.181544 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:35.393877 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:35.393661 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:38.291514 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:38.291474 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:40.207318 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:40.207182 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:40.270964 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:40.270924 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:40.327638 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:40.327600 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:41.958204 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:41.958173 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-56466cd794-4l5lg" event={"ID":"f3812cb7-d263-4c0f-95ad-b45992af6900","Type":"ContainerStarted","Data":"f618fce4c6669fe5a9bc09660caff85170066f56d18212c98a3d0c1cac3aff3e"} Jul 30 00:18:41.959623 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:41.959595 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77947dbf5d-c5gzl" event={"ID":"7aa28d4d-0b2a-4f16-b470-15e3a9216c0d","Type":"ContainerStarted","Data":"963037baea1852ecea078e1e515d25d3aaa730f1cd47a9085d834639692f688f"} Jul 30 00:18:41.959769 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:41.959688 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77947dbf5d-c5gzl" Jul 30 00:18:41.961572 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:41.961544 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77947dbf5d-c5gzl" Jul 30 00:18:41.961746 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:41.961711 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t7sqw" event={"ID":"78222dee-1367-47b9-a2bc-e97144bbeadc","Type":"ContainerStarted","Data":"91492ebc15394138c3de9c5fe405c6c30c7c33181036754fbb0c34d0edfb019f"} Jul 30 00:18:41.963566 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:41.963545 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c56b75766-z8lc8" event={"ID":"8b2a01cb-b31d-4a67-91de-c53a3035fca9","Type":"ContainerStarted","Data":"5df750021dad8db917f777f543546e7e88c9f0525cdb8d202824514b16ffd283"} Jul 30 00:18:41.963631 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:41.963578 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c56b75766-z8lc8" event={"ID":"8b2a01cb-b31d-4a67-91de-c53a3035fca9","Type":"ContainerStarted","Data":"291a012f6e5d0e80ef616ddc0a10e3e5f4b45fab2b022a8b4313ee47a56475c8"} Jul 30 00:18:41.963631 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:41.963590 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c56b75766-z8lc8" event={"ID":"8b2a01cb-b31d-4a67-91de-c53a3035fca9","Type":"ContainerStarted","Data":"29870405b0d6a2480fa40ca29fd5b4d3112d9e2dc4f25185ba1c38000054e60a"} Jul 30 00:18:41.975121 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:41.975082 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-56466cd794-4l5lg" podStartSLOduration=3.064043668 podStartE2EDuration="17.975070484s" totalImagesPullingTime="14.911026816s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:18:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:18:41.973900277 +0000 UTC m=+86.032656246" watchObservedRunningTime="2026-07-30 00:18:41.975070484 +0000 UTC m=+86.033826451" Jul 30 00:18:41.995188 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:41.995152 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c56b75766-z8lc8" podStartSLOduration=7.317351243 podStartE2EDuration="17.995136155s" totalImagesPullingTime="10.677784912s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:18:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:18:41.994055954 +0000 UTC m=+86.052811923" watchObservedRunningTime="2026-07-30 00:18:41.995136155 +0000 UTC m=+86.053892123" Jul 30 00:18:42.010100 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:42.010052 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77947dbf5d-c5gzl" podStartSLOduration=5.709538155 podStartE2EDuration="18.010038511s" totalImagesPullingTime="12.300500356s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:18:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:18:42.008347899 +0000 UTC m=+86.067103868" watchObservedRunningTime="2026-07-30 00:18:42.010038511 +0000 UTC m=+86.068794481" Jul 30 00:18:42.026079 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:42.026034 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-t7sqw" podStartSLOduration=18.251870985 podStartE2EDuration="41.026017968s" totalImagesPullingTime="22.774146983s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 00:18:01 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:18:42.025979474 +0000 UTC m=+86.084735444" watchObservedRunningTime="2026-07-30 00:18:42.026017968 +0000 UTC m=+86.084773939" Jul 30 00:18:42.302037 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:42.301994 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:42.966894 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:42.966836 2563 patch_prober.go:28] interesting pod/image-registry-596d7786c9-nrjxz container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 30 00:18:42.967361 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:42.966924 2563 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-596d7786c9-nrjxz" podUID="f7d33879-e015-4cfe-8d19-11cbc98c1de2" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 30 00:18:43.929282 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:43.929250 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-6kh2h" Jul 30 00:18:44.286496 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:44.286448 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:44.390598 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:44.390561 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:44.482719 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:44.482596 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:45.602010 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:45.601974 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:45.807236 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:45.807209 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-596d7786c9-nrjxz" Jul 30 00:18:45.994505 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:45.994388 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:46.044665 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:46.044628 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:47.361801 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:47.361757 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:47.461605 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:47.461562 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:47.605446 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:47.605402 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-6wt75"] Jul 30 00:18:47.613523 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:47.613455 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6wt75" Jul 30 00:18:47.616066 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:47.616026 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 30 00:18:47.616545 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:47.616263 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-b9dj7\"" Jul 30 00:18:47.616545 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:47.616487 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 30 00:18:47.617175 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:47.617052 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 30 00:18:47.617175 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:47.617066 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 30 00:18:47.630919 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:47.630892 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:47.699070 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:47.698941 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/40d7b3cc-417f-4c61-a04d-e9dc9625bcfe-node-exporter-tls\") pod \"node-exporter-6wt75\" (UID: \"40d7b3cc-417f-4c61-a04d-e9dc9625bcfe\") " pod="openshift-monitoring/node-exporter-6wt75" Jul 30 00:18:47.699070 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:47.698988 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qh779\" (UniqueName: \"kubernetes.io/projected/40d7b3cc-417f-4c61-a04d-e9dc9625bcfe-kube-api-access-qh779\") pod \"node-exporter-6wt75\" (UID: \"40d7b3cc-417f-4c61-a04d-e9dc9625bcfe\") " pod="openshift-monitoring/node-exporter-6wt75" Jul 30 00:18:47.699070 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:47.699034 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/40d7b3cc-417f-4c61-a04d-e9dc9625bcfe-node-exporter-accelerators-collector-config\") pod \"node-exporter-6wt75\" (UID: \"40d7b3cc-417f-4c61-a04d-e9dc9625bcfe\") " pod="openshift-monitoring/node-exporter-6wt75" Jul 30 00:18:47.699338 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:47.699120 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/40d7b3cc-417f-4c61-a04d-e9dc9625bcfe-node-exporter-textfile\") pod \"node-exporter-6wt75\" (UID: \"40d7b3cc-417f-4c61-a04d-e9dc9625bcfe\") " pod="openshift-monitoring/node-exporter-6wt75" Jul 30 00:18:47.699338 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:47.699156 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/40d7b3cc-417f-4c61-a04d-e9dc9625bcfe-metrics-client-ca\") pod \"node-exporter-6wt75\" (UID: \"40d7b3cc-417f-4c61-a04d-e9dc9625bcfe\") " pod="openshift-monitoring/node-exporter-6wt75" Jul 30 00:18:47.699338 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:47.699203 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/40d7b3cc-417f-4c61-a04d-e9dc9625bcfe-node-exporter-wtmp\") pod \"node-exporter-6wt75\" (UID: \"40d7b3cc-417f-4c61-a04d-e9dc9625bcfe\") " pod="openshift-monitoring/node-exporter-6wt75" Jul 30 00:18:47.699338 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:47.699247 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/40d7b3cc-417f-4c61-a04d-e9dc9625bcfe-sys\") pod \"node-exporter-6wt75\" (UID: \"40d7b3cc-417f-4c61-a04d-e9dc9625bcfe\") " pod="openshift-monitoring/node-exporter-6wt75" Jul 30 00:18:47.699338 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:47.699269 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/40d7b3cc-417f-4c61-a04d-e9dc9625bcfe-root\") pod \"node-exporter-6wt75\" (UID: \"40d7b3cc-417f-4c61-a04d-e9dc9625bcfe\") " pod="openshift-monitoring/node-exporter-6wt75" Jul 30 00:18:47.699338 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:47.699292 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/40d7b3cc-417f-4c61-a04d-e9dc9625bcfe-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6wt75\" (UID: \"40d7b3cc-417f-4c61-a04d-e9dc9625bcfe\") " pod="openshift-monitoring/node-exporter-6wt75" Jul 30 00:18:47.729565 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:47.729443 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:47.800689 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:47.800644 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/40d7b3cc-417f-4c61-a04d-e9dc9625bcfe-node-exporter-accelerators-collector-config\") pod \"node-exporter-6wt75\" (UID: \"40d7b3cc-417f-4c61-a04d-e9dc9625bcfe\") " pod="openshift-monitoring/node-exporter-6wt75" Jul 30 00:18:47.800869 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:47.800715 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/40d7b3cc-417f-4c61-a04d-e9dc9625bcfe-node-exporter-textfile\") pod \"node-exporter-6wt75\" (UID: \"40d7b3cc-417f-4c61-a04d-e9dc9625bcfe\") " pod="openshift-monitoring/node-exporter-6wt75" Jul 30 00:18:47.800869 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:47.800760 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/40d7b3cc-417f-4c61-a04d-e9dc9625bcfe-metrics-client-ca\") pod \"node-exporter-6wt75\" (UID: \"40d7b3cc-417f-4c61-a04d-e9dc9625bcfe\") " pod="openshift-monitoring/node-exporter-6wt75" Jul 30 00:18:47.800869 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:47.800800 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/40d7b3cc-417f-4c61-a04d-e9dc9625bcfe-node-exporter-wtmp\") pod \"node-exporter-6wt75\" (UID: \"40d7b3cc-417f-4c61-a04d-e9dc9625bcfe\") " pod="openshift-monitoring/node-exporter-6wt75" Jul 30 00:18:47.800869 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:47.800846 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/40d7b3cc-417f-4c61-a04d-e9dc9625bcfe-sys\") pod \"node-exporter-6wt75\" (UID: \"40d7b3cc-417f-4c61-a04d-e9dc9625bcfe\") " pod="openshift-monitoring/node-exporter-6wt75" Jul 30 00:18:47.800869 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:47.800862 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/40d7b3cc-417f-4c61-a04d-e9dc9625bcfe-root\") pod \"node-exporter-6wt75\" (UID: \"40d7b3cc-417f-4c61-a04d-e9dc9625bcfe\") " pod="openshift-monitoring/node-exporter-6wt75" Jul 30 00:18:47.801147 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:47.800876 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/40d7b3cc-417f-4c61-a04d-e9dc9625bcfe-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6wt75\" (UID: \"40d7b3cc-417f-4c61-a04d-e9dc9625bcfe\") " pod="openshift-monitoring/node-exporter-6wt75" Jul 30 00:18:47.801147 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:47.800942 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/40d7b3cc-417f-4c61-a04d-e9dc9625bcfe-node-exporter-tls\") pod \"node-exporter-6wt75\" (UID: \"40d7b3cc-417f-4c61-a04d-e9dc9625bcfe\") " pod="openshift-monitoring/node-exporter-6wt75" Jul 30 00:18:47.801147 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:47.800945 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/40d7b3cc-417f-4c61-a04d-e9dc9625bcfe-sys\") pod \"node-exporter-6wt75\" (UID: \"40d7b3cc-417f-4c61-a04d-e9dc9625bcfe\") " pod="openshift-monitoring/node-exporter-6wt75" Jul 30 00:18:47.801147 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:47.800969 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qh779\" (UniqueName: \"kubernetes.io/projected/40d7b3cc-417f-4c61-a04d-e9dc9625bcfe-kube-api-access-qh779\") pod \"node-exporter-6wt75\" (UID: \"40d7b3cc-417f-4c61-a04d-e9dc9625bcfe\") " pod="openshift-monitoring/node-exporter-6wt75" Jul 30 00:18:47.801147 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:47.800971 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/40d7b3cc-417f-4c61-a04d-e9dc9625bcfe-root\") pod \"node-exporter-6wt75\" (UID: \"40d7b3cc-417f-4c61-a04d-e9dc9625bcfe\") " pod="openshift-monitoring/node-exporter-6wt75" Jul 30 00:18:47.801147 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:47.801092 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/40d7b3cc-417f-4c61-a04d-e9dc9625bcfe-node-exporter-wtmp\") pod \"node-exporter-6wt75\" (UID: \"40d7b3cc-417f-4c61-a04d-e9dc9625bcfe\") " pod="openshift-monitoring/node-exporter-6wt75" Jul 30 00:18:47.801147 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:47.801126 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/40d7b3cc-417f-4c61-a04d-e9dc9625bcfe-node-exporter-textfile\") pod \"node-exporter-6wt75\" (UID: \"40d7b3cc-417f-4c61-a04d-e9dc9625bcfe\") " pod="openshift-monitoring/node-exporter-6wt75" Jul 30 00:18:47.801473 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:18:47.801163 2563 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 30 00:18:47.801473 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:18:47.801225 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40d7b3cc-417f-4c61-a04d-e9dc9625bcfe-node-exporter-tls podName:40d7b3cc-417f-4c61-a04d-e9dc9625bcfe nodeName:}" failed. No retries permitted until 2026-07-30 00:18:48.301206189 +0000 UTC m=+92.359962151 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/40d7b3cc-417f-4c61-a04d-e9dc9625bcfe-node-exporter-tls") pod "node-exporter-6wt75" (UID: "40d7b3cc-417f-4c61-a04d-e9dc9625bcfe") : secret "node-exporter-tls" not found Jul 30 00:18:47.801473 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:47.801343 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/40d7b3cc-417f-4c61-a04d-e9dc9625bcfe-node-exporter-accelerators-collector-config\") pod \"node-exporter-6wt75\" (UID: \"40d7b3cc-417f-4c61-a04d-e9dc9625bcfe\") " pod="openshift-monitoring/node-exporter-6wt75" Jul 30 00:18:47.801473 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:47.801354 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/40d7b3cc-417f-4c61-a04d-e9dc9625bcfe-metrics-client-ca\") pod \"node-exporter-6wt75\" (UID: \"40d7b3cc-417f-4c61-a04d-e9dc9625bcfe\") " pod="openshift-monitoring/node-exporter-6wt75" Jul 30 00:18:47.803777 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:47.803748 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/40d7b3cc-417f-4c61-a04d-e9dc9625bcfe-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6wt75\" (UID: \"40d7b3cc-417f-4c61-a04d-e9dc9625bcfe\") " pod="openshift-monitoring/node-exporter-6wt75" Jul 30 00:18:47.813010 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:47.812987 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qh779\" (UniqueName: \"kubernetes.io/projected/40d7b3cc-417f-4c61-a04d-e9dc9625bcfe-kube-api-access-qh779\") pod \"node-exporter-6wt75\" (UID: \"40d7b3cc-417f-4c61-a04d-e9dc9625bcfe\") " pod="openshift-monitoring/node-exporter-6wt75" Jul 30 00:18:47.844246 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:47.844211 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:47.944869 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:47.944780 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:48.031545 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:48.031496 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:48.305029 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:48.304990 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/40d7b3cc-417f-4c61-a04d-e9dc9625bcfe-node-exporter-tls\") pod \"node-exporter-6wt75\" (UID: \"40d7b3cc-417f-4c61-a04d-e9dc9625bcfe\") " pod="openshift-monitoring/node-exporter-6wt75" Jul 30 00:18:48.307758 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:48.307710 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/40d7b3cc-417f-4c61-a04d-e9dc9625bcfe-node-exporter-tls\") pod \"node-exporter-6wt75\" (UID: \"40d7b3cc-417f-4c61-a04d-e9dc9625bcfe\") " pod="openshift-monitoring/node-exporter-6wt75" Jul 30 00:18:48.515087 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:48.515047 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:48.526713 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:48.526681 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6wt75" Jul 30 00:18:48.622658 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:48.622622 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:48.697460 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:48.697418 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:48.753926 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:48.753888 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:50.520117 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:50.520082 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:50.579145 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:50.579113 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:50.684509 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:50.684475 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:51.379576 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:51.379531 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:51.450587 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:51.450528 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-596d7786c9-nrjxz"] Jul 30 00:18:51.631627 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:51.631515 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:51.731990 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:51.731953 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:52.360248 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:52.360213 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:52.412186 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:52.412140 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:52.565098 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:52.565061 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:52.684535 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:52.684433 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:53.043997 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:53.043955 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:53.096575 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:53.096538 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:53.820025 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:53.819987 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:54.133377 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:54.133290 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:54.166131 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:18:54.166092 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40d7b3cc_417f_4c61_a04d_e9dc9625bcfe.slice/crio-b6555777ba2e8a5fbc54f892e5fbf14b5eef913dc3da6df31c79751c3abe36d0 WatchSource:0}: Error finding container b6555777ba2e8a5fbc54f892e5fbf14b5eef913dc3da6df31c79751c3abe36d0: Status 404 returned error can't find the container with id b6555777ba2e8a5fbc54f892e5fbf14b5eef913dc3da6df31c79751c3abe36d0 Jul 30 00:18:54.188270 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:54.188240 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:54.523398 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:54.523370 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:54.624588 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:54.624551 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:54.799625 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:54.799585 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-z7xxn" Jul 30 00:18:55.027198 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:55.027158 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-jzwkw" event={"ID":"7ae2552a-b83f-49a5-b779-c80e727f40b4","Type":"ContainerStarted","Data":"8e694b6c3c0a1255f18db7fcfe5993bd963e4d10dbffa0ca4b45775fc4e7719d"} Jul 30 00:18:55.027684 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:55.027636 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-jzwkw" Jul 30 00:18:55.029225 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:55.029162 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6wt75" event={"ID":"40d7b3cc-417f-4c61-a04d-e9dc9625bcfe","Type":"ContainerStarted","Data":"b6555777ba2e8a5fbc54f892e5fbf14b5eef913dc3da6df31c79751c3abe36d0"} Jul 30 00:18:55.033473 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:55.033395 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:55.043046 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:55.042838 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-jzwkw" Jul 30 00:18:55.046417 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:55.042709 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-jzwkw" podStartSLOduration=7.829966836 podStartE2EDuration="31.042690041s" totalImagesPullingTime="23.212723205s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:18:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:18:55.042299967 +0000 UTC m=+99.101055936" watchObservedRunningTime="2026-07-30 00:18:55.042690041 +0000 UTC m=+99.101446015" Jul 30 00:18:55.135218 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:55.135135 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:55.606136 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:55.606105 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:55.691190 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:55.691154 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:55.752572 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:55.752537 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:56.035665 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:56.035625 2563 generic.go:356] "Generic (PLEG): container finished" podID="40d7b3cc-417f-4c61-a04d-e9dc9625bcfe" containerID="74d04e86fa1d0de82a0e7cbbaac9454169dc69c359aad59ffddc88a2f41f44b6" exitCode=0 Jul 30 00:18:56.036151 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:56.035723 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6wt75" event={"ID":"40d7b3cc-417f-4c61-a04d-e9dc9625bcfe","Type":"ContainerDied","Data":"74d04e86fa1d0de82a0e7cbbaac9454169dc69c359aad59ffddc88a2f41f44b6"} Jul 30 00:18:57.043008 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:57.042962 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6wt75" event={"ID":"40d7b3cc-417f-4c61-a04d-e9dc9625bcfe","Type":"ContainerStarted","Data":"57facde26683e1a8fe38afe95b7f848ee35b6f474a315e7903c377fc55e74657"} Jul 30 00:18:57.043512 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:57.043035 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6wt75" event={"ID":"40d7b3cc-417f-4c61-a04d-e9dc9625bcfe","Type":"ContainerStarted","Data":"d1c6cd53a874cc2b8e06da99e817dd39bb46d0beca3468f278209497830f5ad8"} Jul 30 00:18:57.067082 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:57.067027 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-6wt75" podStartSLOduration=8.571330268 podStartE2EDuration="10.067011551s" totalImagesPullingTime="1.389621705s" totalInitContainerRuntime="106.059578ms" isStatefulPod=true podCreationTimestamp="2026-07-30 00:18:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:18:57.065049291 +0000 UTC m=+101.123805264" watchObservedRunningTime="2026-07-30 00:18:57.067011551 +0000 UTC m=+101.125767519" Jul 30 00:18:57.629828 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:57.629787 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:57.682872 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:57.682833 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:57.837238 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:57.837179 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:18:57.899421 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:18:57.899342 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:19:02.051233 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:02.051193 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:19:02.852471 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:02.852426 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:19:02.953690 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:02.953650 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:19:03.649051 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:03.649008 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:19:03.749214 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:03.749163 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:19:04.110602 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:04.110551 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:19:04.174481 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:04.174446 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:19:04.707193 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:04.707164 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:19:04.759439 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:04.759396 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:19:04.813181 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:04.813145 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:19:12.105493 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:12.105458 2563 generic.go:356] "Generic (PLEG): container finished" podID="c9caa632-edc8-43e9-ab2a-b29758f8150f" containerID="6e43df617a22688b997aaa58e093dd57ac4d3991b02f4edd8f9dfe74f431e909" exitCode=0 Jul 30 00:19:12.105904 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:12.105534 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-xjl58" event={"ID":"c9caa632-edc8-43e9-ab2a-b29758f8150f","Type":"ContainerDied","Data":"6e43df617a22688b997aaa58e093dd57ac4d3991b02f4edd8f9dfe74f431e909"} Jul 30 00:19:12.105904 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:12.105847 2563 scope.go:117] "RemoveContainer" containerID="6e43df617a22688b997aaa58e093dd57ac4d3991b02f4edd8f9dfe74f431e909" Jul 30 00:19:13.110973 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:13.110936 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-xjl58" event={"ID":"c9caa632-edc8-43e9-ab2a-b29758f8150f","Type":"ContainerStarted","Data":"6a9ff4e7e131faf31f33913c18a00d650c4656dda3de86e8acf927c66546dd4d"} Jul 30 00:19:16.471624 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:16.471589 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-596d7786c9-nrjxz" podUID="f7d33879-e015-4cfe-8d19-11cbc98c1de2" containerName="registry" containerID="cri-o://e865380c260eb34e1710b5323c9f3f7d6927d2c268b02b34f513fbcc5fafb066" gracePeriod=30 Jul 30 00:19:16.756718 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:16.756691 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-596d7786c9-nrjxz" Jul 30 00:19:16.856854 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:16.856818 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zngl5\" (UniqueName: \"kubernetes.io/projected/f7d33879-e015-4cfe-8d19-11cbc98c1de2-kube-api-access-zngl5\") pod \"f7d33879-e015-4cfe-8d19-11cbc98c1de2\" (UID: \"f7d33879-e015-4cfe-8d19-11cbc98c1de2\") " Jul 30 00:19:16.857029 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:16.856864 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f7d33879-e015-4cfe-8d19-11cbc98c1de2-installation-pull-secrets\") pod \"f7d33879-e015-4cfe-8d19-11cbc98c1de2\" (UID: \"f7d33879-e015-4cfe-8d19-11cbc98c1de2\") " Jul 30 00:19:16.857029 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:16.856891 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f7d33879-e015-4cfe-8d19-11cbc98c1de2-registry-tls\") pod \"f7d33879-e015-4cfe-8d19-11cbc98c1de2\" (UID: \"f7d33879-e015-4cfe-8d19-11cbc98c1de2\") " Jul 30 00:19:16.857029 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:16.856916 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f7d33879-e015-4cfe-8d19-11cbc98c1de2-image-registry-private-configuration\") pod \"f7d33879-e015-4cfe-8d19-11cbc98c1de2\" (UID: \"f7d33879-e015-4cfe-8d19-11cbc98c1de2\") " Jul 30 00:19:16.857195 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:16.857075 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f7d33879-e015-4cfe-8d19-11cbc98c1de2-ca-trust-extracted\") pod \"f7d33879-e015-4cfe-8d19-11cbc98c1de2\" (UID: \"f7d33879-e015-4cfe-8d19-11cbc98c1de2\") " Jul 30 00:19:16.857195 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:16.857110 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f7d33879-e015-4cfe-8d19-11cbc98c1de2-bound-sa-token\") pod \"f7d33879-e015-4cfe-8d19-11cbc98c1de2\" (UID: \"f7d33879-e015-4cfe-8d19-11cbc98c1de2\") " Jul 30 00:19:16.857195 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:16.857148 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f7d33879-e015-4cfe-8d19-11cbc98c1de2-registry-certificates\") pod \"f7d33879-e015-4cfe-8d19-11cbc98c1de2\" (UID: \"f7d33879-e015-4cfe-8d19-11cbc98c1de2\") " Jul 30 00:19:16.857195 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:16.857173 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f7d33879-e015-4cfe-8d19-11cbc98c1de2-trusted-ca\") pod \"f7d33879-e015-4cfe-8d19-11cbc98c1de2\" (UID: \"f7d33879-e015-4cfe-8d19-11cbc98c1de2\") " Jul 30 00:19:16.857612 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:16.857579 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7d33879-e015-4cfe-8d19-11cbc98c1de2-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "f7d33879-e015-4cfe-8d19-11cbc98c1de2" (UID: "f7d33879-e015-4cfe-8d19-11cbc98c1de2"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 00:19:16.857906 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:16.857883 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7d33879-e015-4cfe-8d19-11cbc98c1de2-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "f7d33879-e015-4cfe-8d19-11cbc98c1de2" (UID: "f7d33879-e015-4cfe-8d19-11cbc98c1de2"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 00:19:16.859968 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:16.859926 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7d33879-e015-4cfe-8d19-11cbc98c1de2-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "f7d33879-e015-4cfe-8d19-11cbc98c1de2" (UID: "f7d33879-e015-4cfe-8d19-11cbc98c1de2"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 00:19:16.859968 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:16.859921 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7d33879-e015-4cfe-8d19-11cbc98c1de2-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "f7d33879-e015-4cfe-8d19-11cbc98c1de2" (UID: "f7d33879-e015-4cfe-8d19-11cbc98c1de2"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 00:19:16.860106 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:16.859992 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7d33879-e015-4cfe-8d19-11cbc98c1de2-kube-api-access-zngl5" (OuterVolumeSpecName: "kube-api-access-zngl5") pod "f7d33879-e015-4cfe-8d19-11cbc98c1de2" (UID: "f7d33879-e015-4cfe-8d19-11cbc98c1de2"). InnerVolumeSpecName "kube-api-access-zngl5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 00:19:16.860106 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:16.860026 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7d33879-e015-4cfe-8d19-11cbc98c1de2-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "f7d33879-e015-4cfe-8d19-11cbc98c1de2" (UID: "f7d33879-e015-4cfe-8d19-11cbc98c1de2"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 00:19:16.860106 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:16.860069 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7d33879-e015-4cfe-8d19-11cbc98c1de2-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "f7d33879-e015-4cfe-8d19-11cbc98c1de2" (UID: "f7d33879-e015-4cfe-8d19-11cbc98c1de2"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 00:19:16.868306 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:16.868281 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7d33879-e015-4cfe-8d19-11cbc98c1de2-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "f7d33879-e015-4cfe-8d19-11cbc98c1de2" (UID: "f7d33879-e015-4cfe-8d19-11cbc98c1de2"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:19:16.958718 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:16.958687 2563 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f7d33879-e015-4cfe-8d19-11cbc98c1de2-installation-pull-secrets\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:19:16.958718 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:16.958716 2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f7d33879-e015-4cfe-8d19-11cbc98c1de2-registry-tls\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:19:16.958912 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:16.958728 2563 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f7d33879-e015-4cfe-8d19-11cbc98c1de2-image-registry-private-configuration\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:19:16.958912 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:16.958760 2563 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f7d33879-e015-4cfe-8d19-11cbc98c1de2-ca-trust-extracted\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:19:16.958912 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:16.958769 2563 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f7d33879-e015-4cfe-8d19-11cbc98c1de2-bound-sa-token\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:19:16.958912 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:16.958777 2563 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f7d33879-e015-4cfe-8d19-11cbc98c1de2-registry-certificates\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:19:16.958912 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:16.958784 2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f7d33879-e015-4cfe-8d19-11cbc98c1de2-trusted-ca\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:19:16.958912 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:16.958792 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zngl5\" (UniqueName: \"kubernetes.io/projected/f7d33879-e015-4cfe-8d19-11cbc98c1de2-kube-api-access-zngl5\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:19:17.126599 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:17.126561 2563 generic.go:356] "Generic (PLEG): container finished" podID="f7d33879-e015-4cfe-8d19-11cbc98c1de2" containerID="e865380c260eb34e1710b5323c9f3f7d6927d2c268b02b34f513fbcc5fafb066" exitCode=0 Jul 30 00:19:17.126783 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:17.126606 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-596d7786c9-nrjxz" Jul 30 00:19:17.126783 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:17.126644 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-596d7786c9-nrjxz" event={"ID":"f7d33879-e015-4cfe-8d19-11cbc98c1de2","Type":"ContainerDied","Data":"e865380c260eb34e1710b5323c9f3f7d6927d2c268b02b34f513fbcc5fafb066"} Jul 30 00:19:17.126783 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:17.126715 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-596d7786c9-nrjxz" event={"ID":"f7d33879-e015-4cfe-8d19-11cbc98c1de2","Type":"ContainerDied","Data":"314bb4783fded2dcfff2ebb1ff1a06bcc1afd80f6ae5dbf1beb8955028a07a06"} Jul 30 00:19:17.126783 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:17.126758 2563 scope.go:117] "RemoveContainer" containerID="e865380c260eb34e1710b5323c9f3f7d6927d2c268b02b34f513fbcc5fafb066" Jul 30 00:19:17.135708 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:17.135691 2563 scope.go:117] "RemoveContainer" containerID="e865380c260eb34e1710b5323c9f3f7d6927d2c268b02b34f513fbcc5fafb066" Jul 30 00:19:17.135989 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:19:17.135967 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e865380c260eb34e1710b5323c9f3f7d6927d2c268b02b34f513fbcc5fafb066\": container with ID starting with e865380c260eb34e1710b5323c9f3f7d6927d2c268b02b34f513fbcc5fafb066 not found: ID does not exist" containerID="e865380c260eb34e1710b5323c9f3f7d6927d2c268b02b34f513fbcc5fafb066" Jul 30 00:19:17.136030 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:17.135996 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e865380c260eb34e1710b5323c9f3f7d6927d2c268b02b34f513fbcc5fafb066"} err="failed to get container status \"e865380c260eb34e1710b5323c9f3f7d6927d2c268b02b34f513fbcc5fafb066\": rpc error: code = NotFound desc = could not find container \"e865380c260eb34e1710b5323c9f3f7d6927d2c268b02b34f513fbcc5fafb066\": container with ID starting with e865380c260eb34e1710b5323c9f3f7d6927d2c268b02b34f513fbcc5fafb066 not found: ID does not exist" Jul 30 00:19:17.148012 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:17.147991 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-596d7786c9-nrjxz"] Jul 30 00:19:17.149399 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:17.149381 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-596d7786c9-nrjxz"] Jul 30 00:19:18.475265 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:18.475229 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7d33879-e015-4cfe-8d19-11cbc98c1de2" path="/var/lib/kubelet/pods/f7d33879-e015-4cfe-8d19-11cbc98c1de2/volumes" Jul 30 00:19:18.646716 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:18.646677 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:19:18.696861 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:18.696814 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:19:27.168039 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:27.168002 2563 generic.go:356] "Generic (PLEG): container finished" podID="7d591175-5d66-48f9-be3e-c89d48534295" containerID="6fee375cdddfb6067431338f61470964f87dbd15d41c3fcb599e5d6520ccc54b" exitCode=0 Jul 30 00:19:27.168482 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:27.168053 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c9vdq" event={"ID":"7d591175-5d66-48f9-be3e-c89d48534295","Type":"ContainerDied","Data":"6fee375cdddfb6067431338f61470964f87dbd15d41c3fcb599e5d6520ccc54b"} Jul 30 00:19:27.168482 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:27.168383 2563 scope.go:117] "RemoveContainer" containerID="6fee375cdddfb6067431338f61470964f87dbd15d41c3fcb599e5d6520ccc54b" Jul 30 00:19:28.173653 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:28.173621 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c9vdq" event={"ID":"7d591175-5d66-48f9-be3e-c89d48534295","Type":"ContainerStarted","Data":"c0c1182d86adb5fc0fd85bd801c793cea938cfc9b52c3962326ba7151e9669eb"} Jul 30 00:19:32.190581 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:32.190546 2563 generic.go:356] "Generic (PLEG): container finished" podID="3fa91894-839d-4035-8b46-a787fe957864" containerID="7f6e0b98d84ae97afadf008e3b6734b2c3e9f8f08c1a6a2d2cad89c727b69f8b" exitCode=0 Jul 30 00:19:32.191102 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:32.190619 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-ddm85" event={"ID":"3fa91894-839d-4035-8b46-a787fe957864","Type":"ContainerDied","Data":"7f6e0b98d84ae97afadf008e3b6734b2c3e9f8f08c1a6a2d2cad89c727b69f8b"} Jul 30 00:19:32.191102 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:32.191000 2563 scope.go:117] "RemoveContainer" containerID="7f6e0b98d84ae97afadf008e3b6734b2c3e9f8f08c1a6a2d2cad89c727b69f8b" Jul 30 00:19:33.196012 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:33.195978 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-ddm85" event={"ID":"3fa91894-839d-4035-8b46-a787fe957864","Type":"ContainerStarted","Data":"d6aab2d8b9bc4353343c9e7ecbe8f31009846bde1134a4c58c386412c360f40b"} Jul 30 00:19:36.190229 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:36.190191 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:19:36.384899 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:36.384861 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:19:46.757983 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:46.757946 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:19:46.808064 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:46.808028 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:19:46.858887 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:46.858852 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:19:46.912406 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:46.912364 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:19:47.464952 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:47.464918 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:19:47.516178 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:47.516146 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:19:57.905312 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:57.905270 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:19:58.004632 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:58.004591 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:19:58.053148 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:19:58.053112 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:20:20.871828 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:20:20.871788 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:20:42.346449 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:20:42.346407 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:20:42.397933 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:20:42.397894 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:20:51.435962 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:20:51.435913 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:20:51.490312 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:20:51.490276 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:20:58.908668 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:20:58.908627 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:20:58.962307 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:20:58.962270 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:20:59.055048 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:20:59.055001 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:20:59.654450 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:20:59.654409 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:20:59.709553 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:20:59.709518 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:21:02.779446 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:21:02.779407 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:21:02.880709 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:21:02.880682 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:21:03.283877 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:21:03.283844 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:21:03.351175 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:21:03.351141 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:21:07.691170 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:21:07.691130 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:21:07.743393 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:21:07.743366 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:21:10.141177 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:21:10.141147 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:21:10.241864 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:21:10.241826 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:21:10.291903 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:21:10.291848 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:22:16.339238 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:22:16.339209 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-b92tw_0ef8f120-f712-4ed9-adbd-40f785d4aa9d/console-operator/1.log" Jul 30 00:22:16.348442 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:22:16.348419 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-b92tw_0ef8f120-f712-4ed9-adbd-40f785d4aa9d/console-operator/1.log" Jul 30 00:22:16.349487 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:22:16.349465 2563 kubelet.go:1648] "Image garbage collection succeeded" Jul 30 00:22:18.355342 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:22:18.355301 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:22:19.746247 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:22:19.746211 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:22:23.025585 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:22:23.025555 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:22:23.125999 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:22:23.125967 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:22:54.746500 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:22:54.746459 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:22:55.649230 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:22:55.649195 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:22:58.571246 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:22:58.571213 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:22:58.669392 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:22:58.669358 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:22:59.728941 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:22:59.728907 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:22:59.832367 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:22:59.832334 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:23:00.792194 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:00.792154 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:23:00.892924 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:00.892886 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:23:02.814319 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:02.814282 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:23:03.195755 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:03.195640 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:23:05.358616 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:05.358584 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:23:05.444034 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:05.443976 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:23:09.634055 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:09.633979 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:23:09.734593 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:09.734559 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:23:11.416835 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:11.416804 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:23:12.634209 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:12.634173 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:23:12.735359 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:12.735328 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:23:20.498873 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:20.498840 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:23:20.598970 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:20.598929 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:23:22.172094 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:22.172053 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:23:23.322423 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:23.322379 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:23:23.376109 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:23.376068 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:23:36.477022 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:36.476966 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:23:36.580957 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:36.580924 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:23:37.995253 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:37.995211 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:23:38.091098 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:38.091059 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:23:38.614630 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:38.614581 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:23:38.832295 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:38.832264 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:23:38.934294 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:38.934200 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:23:38.960862 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:38.960832 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5b74bc5486-frgtd"] Jul 30 00:23:38.961154 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:38.961142 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7d33879-e015-4cfe-8d19-11cbc98c1de2" containerName="registry" Jul 30 00:23:38.961201 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:38.961156 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7d33879-e015-4cfe-8d19-11cbc98c1de2" containerName="registry" Jul 30 00:23:38.961235 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:38.961217 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7d33879-e015-4cfe-8d19-11cbc98c1de2" containerName="registry" Jul 30 00:23:38.963082 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:38.963068 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5b74bc5486-frgtd" Jul 30 00:23:38.967896 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:38.967875 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 30 00:23:38.968051 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:38.968033 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 30 00:23:38.968376 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:38.968360 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 30 00:23:38.968473 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:38.968419 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-kwj6t\"" Jul 30 00:23:38.968554 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:38.968537 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 30 00:23:38.998544 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:38.998510 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5b74bc5486-frgtd"] Jul 30 00:23:39.125777 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:39.125713 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ce5296aa-53c2-4afb-ab15-02535e8eecd5-manifests\") pod \"opendatahub-operator-controller-manager-5b74bc5486-frgtd\" (UID: \"ce5296aa-53c2-4afb-ab15-02535e8eecd5\") " pod="opendatahub/opendatahub-operator-controller-manager-5b74bc5486-frgtd" Jul 30 00:23:39.125955 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:39.125811 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ce5296aa-53c2-4afb-ab15-02535e8eecd5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5b74bc5486-frgtd\" (UID: \"ce5296aa-53c2-4afb-ab15-02535e8eecd5\") " pod="opendatahub/opendatahub-operator-controller-manager-5b74bc5486-frgtd" Jul 30 00:23:39.125955 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:39.125835 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ce5296aa-53c2-4afb-ab15-02535e8eecd5-webhook-cert\") pod \"opendatahub-operator-controller-manager-5b74bc5486-frgtd\" (UID: \"ce5296aa-53c2-4afb-ab15-02535e8eecd5\") " pod="opendatahub/opendatahub-operator-controller-manager-5b74bc5486-frgtd" Jul 30 00:23:39.125955 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:39.125857 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xt589\" (UniqueName: \"kubernetes.io/projected/ce5296aa-53c2-4afb-ab15-02535e8eecd5-kube-api-access-xt589\") pod \"opendatahub-operator-controller-manager-5b74bc5486-frgtd\" (UID: \"ce5296aa-53c2-4afb-ab15-02535e8eecd5\") " pod="opendatahub/opendatahub-operator-controller-manager-5b74bc5486-frgtd" Jul 30 00:23:39.125955 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:39.125887 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ce5296aa-53c2-4afb-ab15-02535e8eecd5-tmp\") pod \"opendatahub-operator-controller-manager-5b74bc5486-frgtd\" (UID: \"ce5296aa-53c2-4afb-ab15-02535e8eecd5\") " pod="opendatahub/opendatahub-operator-controller-manager-5b74bc5486-frgtd" Jul 30 00:23:39.153855 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:39.153821 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:23:39.226946 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:39.226840 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ce5296aa-53c2-4afb-ab15-02535e8eecd5-webhook-cert\") pod \"opendatahub-operator-controller-manager-5b74bc5486-frgtd\" (UID: \"ce5296aa-53c2-4afb-ab15-02535e8eecd5\") " pod="opendatahub/opendatahub-operator-controller-manager-5b74bc5486-frgtd" Jul 30 00:23:39.226946 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:39.226902 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xt589\" (UniqueName: \"kubernetes.io/projected/ce5296aa-53c2-4afb-ab15-02535e8eecd5-kube-api-access-xt589\") pod \"opendatahub-operator-controller-manager-5b74bc5486-frgtd\" (UID: \"ce5296aa-53c2-4afb-ab15-02535e8eecd5\") " pod="opendatahub/opendatahub-operator-controller-manager-5b74bc5486-frgtd" Jul 30 00:23:39.226946 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:39.226926 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ce5296aa-53c2-4afb-ab15-02535e8eecd5-tmp\") pod \"opendatahub-operator-controller-manager-5b74bc5486-frgtd\" (UID: \"ce5296aa-53c2-4afb-ab15-02535e8eecd5\") " pod="opendatahub/opendatahub-operator-controller-manager-5b74bc5486-frgtd" Jul 30 00:23:39.227185 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:39.226984 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ce5296aa-53c2-4afb-ab15-02535e8eecd5-manifests\") pod \"opendatahub-operator-controller-manager-5b74bc5486-frgtd\" (UID: \"ce5296aa-53c2-4afb-ab15-02535e8eecd5\") " pod="opendatahub/opendatahub-operator-controller-manager-5b74bc5486-frgtd" Jul 30 00:23:39.227185 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:39.227024 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ce5296aa-53c2-4afb-ab15-02535e8eecd5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5b74bc5486-frgtd\" (UID: \"ce5296aa-53c2-4afb-ab15-02535e8eecd5\") " pod="opendatahub/opendatahub-operator-controller-manager-5b74bc5486-frgtd" Jul 30 00:23:39.227421 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:39.227401 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ce5296aa-53c2-4afb-ab15-02535e8eecd5-manifests\") pod \"opendatahub-operator-controller-manager-5b74bc5486-frgtd\" (UID: \"ce5296aa-53c2-4afb-ab15-02535e8eecd5\") " pod="opendatahub/opendatahub-operator-controller-manager-5b74bc5486-frgtd" Jul 30 00:23:39.229462 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:39.229430 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ce5296aa-53c2-4afb-ab15-02535e8eecd5-tmp\") pod \"opendatahub-operator-controller-manager-5b74bc5486-frgtd\" (UID: \"ce5296aa-53c2-4afb-ab15-02535e8eecd5\") " pod="opendatahub/opendatahub-operator-controller-manager-5b74bc5486-frgtd" Jul 30 00:23:39.229584 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:39.229438 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ce5296aa-53c2-4afb-ab15-02535e8eecd5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5b74bc5486-frgtd\" (UID: \"ce5296aa-53c2-4afb-ab15-02535e8eecd5\") " pod="opendatahub/opendatahub-operator-controller-manager-5b74bc5486-frgtd" Jul 30 00:23:39.229584 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:39.229570 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ce5296aa-53c2-4afb-ab15-02535e8eecd5-webhook-cert\") pod \"opendatahub-operator-controller-manager-5b74bc5486-frgtd\" (UID: \"ce5296aa-53c2-4afb-ab15-02535e8eecd5\") " pod="opendatahub/opendatahub-operator-controller-manager-5b74bc5486-frgtd" Jul 30 00:23:39.235701 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:39.235680 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xt589\" (UniqueName: \"kubernetes.io/projected/ce5296aa-53c2-4afb-ab15-02535e8eecd5-kube-api-access-xt589\") pod \"opendatahub-operator-controller-manager-5b74bc5486-frgtd\" (UID: \"ce5296aa-53c2-4afb-ab15-02535e8eecd5\") " pod="opendatahub/opendatahub-operator-controller-manager-5b74bc5486-frgtd" Jul 30 00:23:39.258997 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:39.258961 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:23:39.272942 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:39.272910 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5b74bc5486-frgtd" Jul 30 00:23:39.354936 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:39.354860 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:23:39.409650 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:39.409616 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5b74bc5486-frgtd"] Jul 30 00:23:39.412365 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:39.412332 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:23:39.413804 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:23:39.413778 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce5296aa_53c2_4afb_ab15_02535e8eecd5.slice/crio-2acc599436391630692380f518457ce9a05935916474fe4e1384728c500f5066 WatchSource:0}: Error finding container 2acc599436391630692380f518457ce9a05935916474fe4e1384728c500f5066: Status 404 returned error can't find the container with id 2acc599436391630692380f518457ce9a05935916474fe4e1384728c500f5066 Jul 30 00:23:39.415536 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:39.415518 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 30 00:23:39.477155 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:39.477074 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:23:39.578348 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:39.578309 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:23:40.143797 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:40.143754 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5b74bc5486-frgtd" event={"ID":"ce5296aa-53c2-4afb-ab15-02535e8eecd5","Type":"ContainerStarted","Data":"2acc599436391630692380f518457ce9a05935916474fe4e1384728c500f5066"} Jul 30 00:23:43.160998 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:43.160954 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5b74bc5486-frgtd" event={"ID":"ce5296aa-53c2-4afb-ab15-02535e8eecd5","Type":"ContainerStarted","Data":"51927f5edd6b2c032c6efbe2c3b85b759f64a997d11c6fd599a244136ffd7806"} Jul 30 00:23:44.166936 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:44.166900 2563 generic.go:356] "Generic (PLEG): container finished" podID="ce5296aa-53c2-4afb-ab15-02535e8eecd5" containerID="51927f5edd6b2c032c6efbe2c3b85b759f64a997d11c6fd599a244136ffd7806" exitCode=0 Jul 30 00:23:44.167376 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:44.166965 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5b74bc5486-frgtd" event={"ID":"ce5296aa-53c2-4afb-ab15-02535e8eecd5","Type":"ContainerDied","Data":"51927f5edd6b2c032c6efbe2c3b85b759f64a997d11c6fd599a244136ffd7806"} Jul 30 00:23:45.172921 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:45.172885 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5b74bc5486-frgtd" event={"ID":"ce5296aa-53c2-4afb-ab15-02535e8eecd5","Type":"ContainerStarted","Data":"d62fafcc160177d919d23f1b985f5c215e71e519480ffa150f03877e220b93db"} Jul 30 00:23:45.173304 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:45.172962 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5b74bc5486-frgtd" Jul 30 00:23:45.194382 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:45.194331 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5b74bc5486-frgtd" podStartSLOduration=2.888451371 podStartE2EDuration="7.194318262s" totalImagesPullingTime="3.298435956s" totalInitContainerRuntime="1.007430935s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:23:38 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:23:45.191782447 +0000 UTC m=+389.250538414" watchObservedRunningTime="2026-07-30 00:23:45.194318262 +0000 UTC m=+389.253074230" Jul 30 00:23:56.180046 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:56.180000 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5b74bc5486-frgtd" Jul 30 00:23:58.400773 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:58.400723 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:23:58.509838 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:58.509808 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:23:59.137774 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:59.137726 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-xrx9m"] Jul 30 00:23:59.147500 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:59.147476 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-xrx9m" Jul 30 00:23:59.150061 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:59.150024 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 30 00:23:59.150061 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:59.150037 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-tvtl8\"" Jul 30 00:23:59.150913 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:59.150885 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:23:59.154968 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:59.154944 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-xrx9m"] Jul 30 00:23:59.172006 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:59.171982 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-94d649dc8-9v89r"] Jul 30 00:23:59.173249 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:59.173222 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2e7054c1-45e5-443c-8564-731e433d469d-cert\") pod \"odh-model-controller-7998c67dc4-xrx9m\" (UID: \"2e7054c1-45e5-443c-8564-731e433d469d\") " pod="opendatahub/odh-model-controller-7998c67dc4-xrx9m" Jul 30 00:23:59.173376 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:59.173275 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jv4ss\" (UniqueName: \"kubernetes.io/projected/2e7054c1-45e5-443c-8564-731e433d469d-kube-api-access-jv4ss\") pod \"odh-model-controller-7998c67dc4-xrx9m\" (UID: \"2e7054c1-45e5-443c-8564-731e433d469d\") " pod="opendatahub/odh-model-controller-7998c67dc4-xrx9m" Jul 30 00:23:59.174552 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:59.174535 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-94d649dc8-9v89r" Jul 30 00:23:59.177146 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:59.177128 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 30 00:23:59.177462 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:59.177448 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 30 00:23:59.177627 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:59.177502 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-wvrtc\"" Jul 30 00:23:59.184230 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:59.184209 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-94d649dc8-9v89r"] Jul 30 00:23:59.237413 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:59.237386 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:23:59.273616 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:59.273585 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jv4ss\" (UniqueName: \"kubernetes.io/projected/2e7054c1-45e5-443c-8564-731e433d469d-kube-api-access-jv4ss\") pod \"odh-model-controller-7998c67dc4-xrx9m\" (UID: \"2e7054c1-45e5-443c-8564-731e433d469d\") " pod="opendatahub/odh-model-controller-7998c67dc4-xrx9m" Jul 30 00:23:59.273763 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:59.273655 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/41a9676d-a4c8-4390-8de0-a7e6eae32c57-tmp\") pod \"kube-auth-proxy-94d649dc8-9v89r\" (UID: \"41a9676d-a4c8-4390-8de0-a7e6eae32c57\") " pod="openshift-ingress/kube-auth-proxy-94d649dc8-9v89r" Jul 30 00:23:59.273763 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:59.273702 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41a9676d-a4c8-4390-8de0-a7e6eae32c57-tls-certs\") pod \"kube-auth-proxy-94d649dc8-9v89r\" (UID: \"41a9676d-a4c8-4390-8de0-a7e6eae32c57\") " pod="openshift-ingress/kube-auth-proxy-94d649dc8-9v89r" Jul 30 00:23:59.273763 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:59.273753 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmggf\" (UniqueName: \"kubernetes.io/projected/41a9676d-a4c8-4390-8de0-a7e6eae32c57-kube-api-access-jmggf\") pod \"kube-auth-proxy-94d649dc8-9v89r\" (UID: \"41a9676d-a4c8-4390-8de0-a7e6eae32c57\") " pod="openshift-ingress/kube-auth-proxy-94d649dc8-9v89r" Jul 30 00:23:59.273876 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:59.273784 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2e7054c1-45e5-443c-8564-731e433d469d-cert\") pod \"odh-model-controller-7998c67dc4-xrx9m\" (UID: \"2e7054c1-45e5-443c-8564-731e433d469d\") " pod="opendatahub/odh-model-controller-7998c67dc4-xrx9m" Jul 30 00:23:59.273910 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:23:59.273894 2563 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 30 00:23:59.273971 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:23:59.273961 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e7054c1-45e5-443c-8564-731e433d469d-cert podName:2e7054c1-45e5-443c-8564-731e433d469d nodeName:}" failed. No retries permitted until 2026-07-30 00:23:59.773940801 +0000 UTC m=+403.832696751 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/2e7054c1-45e5-443c-8564-731e433d469d-cert") pod "odh-model-controller-7998c67dc4-xrx9m" (UID: "2e7054c1-45e5-443c-8564-731e433d469d") : secret "odh-model-controller-webhook-cert" not found Jul 30 00:23:59.281828 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:59.281805 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jv4ss\" (UniqueName: \"kubernetes.io/projected/2e7054c1-45e5-443c-8564-731e433d469d-kube-api-access-jv4ss\") pod \"odh-model-controller-7998c67dc4-xrx9m\" (UID: \"2e7054c1-45e5-443c-8564-731e433d469d\") " pod="opendatahub/odh-model-controller-7998c67dc4-xrx9m" Jul 30 00:23:59.337097 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:59.337071 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:23:59.374694 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:59.374670 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/41a9676d-a4c8-4390-8de0-a7e6eae32c57-tmp\") pod \"kube-auth-proxy-94d649dc8-9v89r\" (UID: \"41a9676d-a4c8-4390-8de0-a7e6eae32c57\") " pod="openshift-ingress/kube-auth-proxy-94d649dc8-9v89r" Jul 30 00:23:59.374811 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:59.374714 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41a9676d-a4c8-4390-8de0-a7e6eae32c57-tls-certs\") pod \"kube-auth-proxy-94d649dc8-9v89r\" (UID: \"41a9676d-a4c8-4390-8de0-a7e6eae32c57\") " pod="openshift-ingress/kube-auth-proxy-94d649dc8-9v89r" Jul 30 00:23:59.374811 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:59.374749 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmggf\" (UniqueName: \"kubernetes.io/projected/41a9676d-a4c8-4390-8de0-a7e6eae32c57-kube-api-access-jmggf\") pod \"kube-auth-proxy-94d649dc8-9v89r\" (UID: \"41a9676d-a4c8-4390-8de0-a7e6eae32c57\") " pod="openshift-ingress/kube-auth-proxy-94d649dc8-9v89r" Jul 30 00:23:59.376805 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:59.376784 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/41a9676d-a4c8-4390-8de0-a7e6eae32c57-tmp\") pod \"kube-auth-proxy-94d649dc8-9v89r\" (UID: \"41a9676d-a4c8-4390-8de0-a7e6eae32c57\") " pod="openshift-ingress/kube-auth-proxy-94d649dc8-9v89r" Jul 30 00:23:59.377507 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:59.377490 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41a9676d-a4c8-4390-8de0-a7e6eae32c57-tls-certs\") pod \"kube-auth-proxy-94d649dc8-9v89r\" (UID: \"41a9676d-a4c8-4390-8de0-a7e6eae32c57\") " pod="openshift-ingress/kube-auth-proxy-94d649dc8-9v89r" Jul 30 00:23:59.385634 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:59.385605 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmggf\" (UniqueName: \"kubernetes.io/projected/41a9676d-a4c8-4390-8de0-a7e6eae32c57-kube-api-access-jmggf\") pod \"kube-auth-proxy-94d649dc8-9v89r\" (UID: \"41a9676d-a4c8-4390-8de0-a7e6eae32c57\") " pod="openshift-ingress/kube-auth-proxy-94d649dc8-9v89r" Jul 30 00:23:59.437680 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:59.437618 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:23:59.488472 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:59.488451 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-94d649dc8-9v89r" Jul 30 00:23:59.541305 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:59.541269 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:23:59.608979 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:59.608949 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-ktnsg"] Jul 30 00:23:59.615441 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:59.615405 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:23:59.615699 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:59.615681 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-ktnsg" Jul 30 00:23:59.620149 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:59.620116 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 30 00:23:59.620268 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:59.620116 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-lrrk5\"" Jul 30 00:23:59.621471 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:59.621447 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-94d649dc8-9v89r"] Jul 30 00:23:59.622825 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:23:59.622800 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41a9676d_a4c8_4390_8de0_a7e6eae32c57.slice/crio-72ab3b465d7e8c9bb71f2f592c47cfb777a2102018764586289b06615a3cad6c WatchSource:0}: Error finding container 72ab3b465d7e8c9bb71f2f592c47cfb777a2102018764586289b06615a3cad6c: Status 404 returned error can't find the container with id 72ab3b465d7e8c9bb71f2f592c47cfb777a2102018764586289b06615a3cad6c Jul 30 00:23:59.628382 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:59.628311 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-ktnsg"] Jul 30 00:23:59.677544 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:59.677508 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84vbd\" (UniqueName: \"kubernetes.io/projected/ac97530b-0479-4196-967b-a1affa4fd298-kube-api-access-84vbd\") pod \"model-serving-api-54d89bd5fc-ktnsg\" (UID: \"ac97530b-0479-4196-967b-a1affa4fd298\") " pod="opendatahub/model-serving-api-54d89bd5fc-ktnsg" Jul 30 00:23:59.677690 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:59.677562 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac97530b-0479-4196-967b-a1affa4fd298-tls-certs\") pod \"model-serving-api-54d89bd5fc-ktnsg\" (UID: \"ac97530b-0479-4196-967b-a1affa4fd298\") " pod="opendatahub/model-serving-api-54d89bd5fc-ktnsg" Jul 30 00:23:59.738633 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:59.738555 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:23:59.778670 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:59.778642 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84vbd\" (UniqueName: \"kubernetes.io/projected/ac97530b-0479-4196-967b-a1affa4fd298-kube-api-access-84vbd\") pod \"model-serving-api-54d89bd5fc-ktnsg\" (UID: \"ac97530b-0479-4196-967b-a1affa4fd298\") " pod="opendatahub/model-serving-api-54d89bd5fc-ktnsg" Jul 30 00:23:59.778810 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:59.778686 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2e7054c1-45e5-443c-8564-731e433d469d-cert\") pod \"odh-model-controller-7998c67dc4-xrx9m\" (UID: \"2e7054c1-45e5-443c-8564-731e433d469d\") " pod="opendatahub/odh-model-controller-7998c67dc4-xrx9m" Jul 30 00:23:59.778810 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:59.778713 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac97530b-0479-4196-967b-a1affa4fd298-tls-certs\") pod \"model-serving-api-54d89bd5fc-ktnsg\" (UID: \"ac97530b-0479-4196-967b-a1affa4fd298\") " pod="opendatahub/model-serving-api-54d89bd5fc-ktnsg" Jul 30 00:23:59.781227 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:59.781208 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2e7054c1-45e5-443c-8564-731e433d469d-cert\") pod \"odh-model-controller-7998c67dc4-xrx9m\" (UID: \"2e7054c1-45e5-443c-8564-731e433d469d\") " pod="opendatahub/odh-model-controller-7998c67dc4-xrx9m" Jul 30 00:23:59.781322 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:59.781255 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac97530b-0479-4196-967b-a1affa4fd298-tls-certs\") pod \"model-serving-api-54d89bd5fc-ktnsg\" (UID: \"ac97530b-0479-4196-967b-a1affa4fd298\") " pod="opendatahub/model-serving-api-54d89bd5fc-ktnsg" Jul 30 00:23:59.786133 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:59.786106 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84vbd\" (UniqueName: \"kubernetes.io/projected/ac97530b-0479-4196-967b-a1affa4fd298-kube-api-access-84vbd\") pod \"model-serving-api-54d89bd5fc-ktnsg\" (UID: \"ac97530b-0479-4196-967b-a1affa4fd298\") " pod="opendatahub/model-serving-api-54d89bd5fc-ktnsg" Jul 30 00:23:59.930794 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:23:59.930755 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-ktnsg" Jul 30 00:24:00.059464 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:00.059410 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-xrx9m" Jul 30 00:24:00.061474 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:00.061451 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-ktnsg"] Jul 30 00:24:00.063399 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:24:00.063374 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac97530b_0479_4196_967b_a1affa4fd298.slice/crio-6f8be28fe02336f8121f3f51144e613b77418555a0d78dcdfdf1cf677b2b310a WatchSource:0}: Error finding container 6f8be28fe02336f8121f3f51144e613b77418555a0d78dcdfdf1cf677b2b310a: Status 404 returned error can't find the container with id 6f8be28fe02336f8121f3f51144e613b77418555a0d78dcdfdf1cf677b2b310a Jul 30 00:24:00.196282 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:00.196256 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-xrx9m"] Jul 30 00:24:00.198303 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:24:00.198270 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e7054c1_45e5_443c_8564_731e433d469d.slice/crio-9a5ac91e74a00ac0269a284ced56db096007b8d71bda2a9b357e40a4563061fd WatchSource:0}: Error finding container 9a5ac91e74a00ac0269a284ced56db096007b8d71bda2a9b357e40a4563061fd: Status 404 returned error can't find the container with id 9a5ac91e74a00ac0269a284ced56db096007b8d71bda2a9b357e40a4563061fd Jul 30 00:24:00.236642 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:00.236618 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-xrx9m" event={"ID":"2e7054c1-45e5-443c-8564-731e433d469d","Type":"ContainerStarted","Data":"9a5ac91e74a00ac0269a284ced56db096007b8d71bda2a9b357e40a4563061fd"} Jul 30 00:24:00.237673 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:00.237650 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-ktnsg" event={"ID":"ac97530b-0479-4196-967b-a1affa4fd298","Type":"ContainerStarted","Data":"6f8be28fe02336f8121f3f51144e613b77418555a0d78dcdfdf1cf677b2b310a"} Jul 30 00:24:00.238663 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:00.238637 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-94d649dc8-9v89r" event={"ID":"41a9676d-a4c8-4390-8de0-a7e6eae32c57","Type":"ContainerStarted","Data":"72ab3b465d7e8c9bb71f2f592c47cfb777a2102018764586289b06615a3cad6c"} Jul 30 00:24:00.493228 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:00.493072 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:24:00.493822 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:00.493799 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6w2cb"] Jul 30 00:24:00.496254 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:00.496218 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6w2cb" Jul 30 00:24:00.498716 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:00.498692 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 30 00:24:00.498836 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:00.498745 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 30 00:24:00.498836 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:00.498692 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 30 00:24:00.498836 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:00.498697 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-x2thh\"" Jul 30 00:24:00.498836 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:00.498694 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 30 00:24:00.507576 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:00.507547 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6w2cb"] Jul 30 00:24:00.564427 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:00.564389 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:24:00.586621 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:00.586586 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7ed721de-9dd1-48d0-86ab-5822cb6c6c4b-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-6w2cb\" (UID: \"7ed721de-9dd1-48d0-86ab-5822cb6c6c4b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6w2cb" Jul 30 00:24:00.586777 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:00.586629 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7ed721de-9dd1-48d0-86ab-5822cb6c6c4b-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-6w2cb\" (UID: \"7ed721de-9dd1-48d0-86ab-5822cb6c6c4b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6w2cb" Jul 30 00:24:00.586777 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:00.586655 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rmr5\" (UniqueName: \"kubernetes.io/projected/7ed721de-9dd1-48d0-86ab-5822cb6c6c4b-kube-api-access-6rmr5\") pod \"istiod-openshift-gateway-7b7995d5cd-6w2cb\" (UID: \"7ed721de-9dd1-48d0-86ab-5822cb6c6c4b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6w2cb" Jul 30 00:24:00.586777 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:00.586712 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ed721de-9dd1-48d0-86ab-5822cb6c6c4b-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-6w2cb\" (UID: \"7ed721de-9dd1-48d0-86ab-5822cb6c6c4b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6w2cb" Jul 30 00:24:00.586777 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:00.586730 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7ed721de-9dd1-48d0-86ab-5822cb6c6c4b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-6w2cb\" (UID: \"7ed721de-9dd1-48d0-86ab-5822cb6c6c4b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6w2cb" Jul 30 00:24:00.586965 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:00.586803 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7ed721de-9dd1-48d0-86ab-5822cb6c6c4b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-6w2cb\" (UID: \"7ed721de-9dd1-48d0-86ab-5822cb6c6c4b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6w2cb" Jul 30 00:24:00.586965 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:00.586849 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ed721de-9dd1-48d0-86ab-5822cb6c6c4b-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-6w2cb\" (UID: \"7ed721de-9dd1-48d0-86ab-5822cb6c6c4b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6w2cb" Jul 30 00:24:00.688283 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:00.688240 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7ed721de-9dd1-48d0-86ab-5822cb6c6c4b-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-6w2cb\" (UID: \"7ed721de-9dd1-48d0-86ab-5822cb6c6c4b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6w2cb" Jul 30 00:24:00.688457 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:00.688303 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rmr5\" (UniqueName: \"kubernetes.io/projected/7ed721de-9dd1-48d0-86ab-5822cb6c6c4b-kube-api-access-6rmr5\") pod \"istiod-openshift-gateway-7b7995d5cd-6w2cb\" (UID: \"7ed721de-9dd1-48d0-86ab-5822cb6c6c4b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6w2cb" Jul 30 00:24:00.688457 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:00.688355 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ed721de-9dd1-48d0-86ab-5822cb6c6c4b-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-6w2cb\" (UID: \"7ed721de-9dd1-48d0-86ab-5822cb6c6c4b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6w2cb" Jul 30 00:24:00.688457 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:00.688386 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7ed721de-9dd1-48d0-86ab-5822cb6c6c4b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-6w2cb\" (UID: \"7ed721de-9dd1-48d0-86ab-5822cb6c6c4b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6w2cb" Jul 30 00:24:00.688457 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:00.688447 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7ed721de-9dd1-48d0-86ab-5822cb6c6c4b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-6w2cb\" (UID: \"7ed721de-9dd1-48d0-86ab-5822cb6c6c4b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6w2cb" Jul 30 00:24:00.688674 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:00.688485 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ed721de-9dd1-48d0-86ab-5822cb6c6c4b-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-6w2cb\" (UID: \"7ed721de-9dd1-48d0-86ab-5822cb6c6c4b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6w2cb" Jul 30 00:24:00.688674 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:00.688522 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7ed721de-9dd1-48d0-86ab-5822cb6c6c4b-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-6w2cb\" (UID: \"7ed721de-9dd1-48d0-86ab-5822cb6c6c4b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6w2cb" Jul 30 00:24:00.689278 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:00.689251 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7ed721de-9dd1-48d0-86ab-5822cb6c6c4b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-6w2cb\" (UID: \"7ed721de-9dd1-48d0-86ab-5822cb6c6c4b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6w2cb" Jul 30 00:24:00.691448 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:00.691345 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7ed721de-9dd1-48d0-86ab-5822cb6c6c4b-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-6w2cb\" (UID: \"7ed721de-9dd1-48d0-86ab-5822cb6c6c4b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6w2cb" Jul 30 00:24:00.691566 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:00.691470 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7ed721de-9dd1-48d0-86ab-5822cb6c6c4b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-6w2cb\" (UID: \"7ed721de-9dd1-48d0-86ab-5822cb6c6c4b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6w2cb" Jul 30 00:24:00.691566 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:00.691518 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ed721de-9dd1-48d0-86ab-5822cb6c6c4b-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-6w2cb\" (UID: \"7ed721de-9dd1-48d0-86ab-5822cb6c6c4b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6w2cb" Jul 30 00:24:00.692363 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:00.692323 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7ed721de-9dd1-48d0-86ab-5822cb6c6c4b-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-6w2cb\" (UID: \"7ed721de-9dd1-48d0-86ab-5822cb6c6c4b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6w2cb" Jul 30 00:24:00.697332 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:00.697288 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rmr5\" (UniqueName: \"kubernetes.io/projected/7ed721de-9dd1-48d0-86ab-5822cb6c6c4b-kube-api-access-6rmr5\") pod \"istiod-openshift-gateway-7b7995d5cd-6w2cb\" (UID: \"7ed721de-9dd1-48d0-86ab-5822cb6c6c4b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6w2cb" Jul 30 00:24:00.697771 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:00.697725 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ed721de-9dd1-48d0-86ab-5822cb6c6c4b-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-6w2cb\" (UID: \"7ed721de-9dd1-48d0-86ab-5822cb6c6c4b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6w2cb" Jul 30 00:24:00.808845 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:00.808690 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6w2cb" Jul 30 00:24:00.987844 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:00.987774 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6w2cb"] Jul 30 00:24:00.993081 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:24:00.992967 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ed721de_9dd1_48d0_86ab_5822cb6c6c4b.slice/crio-4e686a2337e56b8a8ff92be1ff1bf7d844807e473b54410af5e34807a5f28f57 WatchSource:0}: Error finding container 4e686a2337e56b8a8ff92be1ff1bf7d844807e473b54410af5e34807a5f28f57: Status 404 returned error can't find the container with id 4e686a2337e56b8a8ff92be1ff1bf7d844807e473b54410af5e34807a5f28f57 Jul 30 00:24:01.248468 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:01.248388 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6w2cb" event={"ID":"7ed721de-9dd1-48d0-86ab-5822cb6c6c4b","Type":"ContainerStarted","Data":"4e686a2337e56b8a8ff92be1ff1bf7d844807e473b54410af5e34807a5f28f57"} Jul 30 00:24:01.332330 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:01.332282 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:24:02.256545 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:02.256495 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-ktnsg" event={"ID":"ac97530b-0479-4196-967b-a1affa4fd298","Type":"ContainerStarted","Data":"1878445719e8add04c2752c3226e934a75a41e63d7a757e64afa8f4b500115fb"} Jul 30 00:24:02.257038 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:02.256577 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-ktnsg" Jul 30 00:24:02.273627 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:02.273537 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-ktnsg" podStartSLOduration=1.5202308580000001 podStartE2EDuration="3.273514725s" totalImagesPullingTime="1.753283867s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:23:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:24:02.27286665 +0000 UTC m=+406.331622631" watchObservedRunningTime="2026-07-30 00:24:02.273514725 +0000 UTC m=+406.332270696" Jul 30 00:24:04.271676 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:04.271621 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-xrx9m" event={"ID":"2e7054c1-45e5-443c-8564-731e433d469d","Type":"ContainerStarted","Data":"b9faa7a4122a022e24083ee35034dd07e25c1609c46bd5d9b021dfe26a5c5388"} Jul 30 00:24:04.272216 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:04.271843 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-xrx9m" Jul 30 00:24:04.274941 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:04.274865 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-94d649dc8-9v89r" event={"ID":"41a9676d-a4c8-4390-8de0-a7e6eae32c57","Type":"ContainerStarted","Data":"9af9e5381c7ea3144a26415fb048dcd3b4bcb6bfbdcab305fadc9b950357e5cb"} Jul 30 00:24:04.305664 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:04.305597 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-xrx9m" podStartSLOduration=1.582706521 podStartE2EDuration="5.305581855s" totalImagesPullingTime="3.722875334s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:23:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:24:04.304840808 +0000 UTC m=+408.363596777" watchObservedRunningTime="2026-07-30 00:24:04.305581855 +0000 UTC m=+408.364337880" Jul 30 00:24:04.365311 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:04.361405 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-94d649dc8-9v89r" podStartSLOduration=1.062230356 podStartE2EDuration="5.361384583s" totalImagesPullingTime="4.299154227s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:23:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:24:04.36101894 +0000 UTC m=+408.419774898" watchObservedRunningTime="2026-07-30 00:24:04.361384583 +0000 UTC m=+408.420140552" Jul 30 00:24:05.243867 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:05.243824 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:24:05.244009 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:05.243915 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:24:06.286435 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:06.286395 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6w2cb" event={"ID":"7ed721de-9dd1-48d0-86ab-5822cb6c6c4b","Type":"ContainerStarted","Data":"eadcd51d6c25a0670a66ed2cbeddfe353db293896e249f1ae93aa72bff2da631"} Jul 30 00:24:06.286897 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:06.286553 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6w2cb" Jul 30 00:24:06.288363 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:06.288334 2563 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-6w2cb container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 30 00:24:06.288477 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:06.288389 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6w2cb" podUID="7ed721de-9dd1-48d0-86ab-5822cb6c6c4b" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 30 00:24:06.310040 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:06.309989 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6w2cb" podStartSLOduration=2.061414782 podStartE2EDuration="6.309977015s" totalImagesPullingTime="4.248562233s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:24:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:24:06.30883926 +0000 UTC m=+410.367595238" watchObservedRunningTime="2026-07-30 00:24:06.309977015 +0000 UTC m=+410.368732983" Jul 30 00:24:06.580310 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:06.580197 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:24:06.681213 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:06.681179 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:24:07.226146 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:07.226101 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h"] Jul 30 00:24:07.229682 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:07.229659 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h" Jul 30 00:24:07.232481 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:07.232455 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-7pgj7\"" Jul 30 00:24:07.240722 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:07.240696 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h"] Jul 30 00:24:07.247392 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:07.247365 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9cd41c0-eeee-4253-b07b-3e93b15a60fd-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h\" (UID: \"d9cd41c0-eeee-4253-b07b-3e93b15a60fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h" Jul 30 00:24:07.247549 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:07.247397 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9cd41c0-eeee-4253-b07b-3e93b15a60fd-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h\" (UID: \"d9cd41c0-eeee-4253-b07b-3e93b15a60fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h" Jul 30 00:24:07.247549 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:07.247428 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9cd41c0-eeee-4253-b07b-3e93b15a60fd-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h\" (UID: \"d9cd41c0-eeee-4253-b07b-3e93b15a60fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h" Jul 30 00:24:07.247549 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:07.247509 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9cd41c0-eeee-4253-b07b-3e93b15a60fd-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h\" (UID: \"d9cd41c0-eeee-4253-b07b-3e93b15a60fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h" Jul 30 00:24:07.247549 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:07.247547 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9cd41c0-eeee-4253-b07b-3e93b15a60fd-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h\" (UID: \"d9cd41c0-eeee-4253-b07b-3e93b15a60fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h" Jul 30 00:24:07.247812 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:07.247568 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9cd41c0-eeee-4253-b07b-3e93b15a60fd-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h\" (UID: \"d9cd41c0-eeee-4253-b07b-3e93b15a60fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h" Jul 30 00:24:07.247812 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:07.247621 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9cd41c0-eeee-4253-b07b-3e93b15a60fd-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h\" (UID: \"d9cd41c0-eeee-4253-b07b-3e93b15a60fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h" Jul 30 00:24:07.247812 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:07.247650 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9cd41c0-eeee-4253-b07b-3e93b15a60fd-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h\" (UID: \"d9cd41c0-eeee-4253-b07b-3e93b15a60fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h" Jul 30 00:24:07.247812 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:07.247687 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tblp2\" (UniqueName: \"kubernetes.io/projected/d9cd41c0-eeee-4253-b07b-3e93b15a60fd-kube-api-access-tblp2\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h\" (UID: \"d9cd41c0-eeee-4253-b07b-3e93b15a60fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h" Jul 30 00:24:07.277328 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:07.277288 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:24:07.291179 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:07.291159 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6w2cb" Jul 30 00:24:07.348422 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:07.348379 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9cd41c0-eeee-4253-b07b-3e93b15a60fd-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h\" (UID: \"d9cd41c0-eeee-4253-b07b-3e93b15a60fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h" Jul 30 00:24:07.348597 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:07.348448 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9cd41c0-eeee-4253-b07b-3e93b15a60fd-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h\" (UID: \"d9cd41c0-eeee-4253-b07b-3e93b15a60fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h" Jul 30 00:24:07.348597 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:07.348486 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9cd41c0-eeee-4253-b07b-3e93b15a60fd-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h\" (UID: \"d9cd41c0-eeee-4253-b07b-3e93b15a60fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h" Jul 30 00:24:07.348597 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:07.348553 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9cd41c0-eeee-4253-b07b-3e93b15a60fd-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h\" (UID: \"d9cd41c0-eeee-4253-b07b-3e93b15a60fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h" Jul 30 00:24:07.348597 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:07.348579 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9cd41c0-eeee-4253-b07b-3e93b15a60fd-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h\" (UID: \"d9cd41c0-eeee-4253-b07b-3e93b15a60fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h" Jul 30 00:24:07.348816 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:07.348606 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9cd41c0-eeee-4253-b07b-3e93b15a60fd-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h\" (UID: \"d9cd41c0-eeee-4253-b07b-3e93b15a60fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h" Jul 30 00:24:07.348816 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:07.348676 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9cd41c0-eeee-4253-b07b-3e93b15a60fd-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h\" (UID: \"d9cd41c0-eeee-4253-b07b-3e93b15a60fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h" Jul 30 00:24:07.348816 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:07.348704 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9cd41c0-eeee-4253-b07b-3e93b15a60fd-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h\" (UID: \"d9cd41c0-eeee-4253-b07b-3e93b15a60fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h" Jul 30 00:24:07.348816 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:07.348770 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tblp2\" (UniqueName: \"kubernetes.io/projected/d9cd41c0-eeee-4253-b07b-3e93b15a60fd-kube-api-access-tblp2\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h\" (UID: \"d9cd41c0-eeee-4253-b07b-3e93b15a60fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h" Jul 30 00:24:07.349450 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:07.349420 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9cd41c0-eeee-4253-b07b-3e93b15a60fd-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h\" (UID: \"d9cd41c0-eeee-4253-b07b-3e93b15a60fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h" Jul 30 00:24:07.350220 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:07.350183 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9cd41c0-eeee-4253-b07b-3e93b15a60fd-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h\" (UID: \"d9cd41c0-eeee-4253-b07b-3e93b15a60fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h" Jul 30 00:24:07.350428 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:07.350407 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9cd41c0-eeee-4253-b07b-3e93b15a60fd-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h\" (UID: \"d9cd41c0-eeee-4253-b07b-3e93b15a60fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h" Jul 30 00:24:07.352758 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:07.351721 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9cd41c0-eeee-4253-b07b-3e93b15a60fd-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h\" (UID: \"d9cd41c0-eeee-4253-b07b-3e93b15a60fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h" Jul 30 00:24:07.355765 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:07.352928 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9cd41c0-eeee-4253-b07b-3e93b15a60fd-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h\" (UID: \"d9cd41c0-eeee-4253-b07b-3e93b15a60fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h" Jul 30 00:24:07.364041 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:07.362296 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9cd41c0-eeee-4253-b07b-3e93b15a60fd-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h\" (UID: \"d9cd41c0-eeee-4253-b07b-3e93b15a60fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h" Jul 30 00:24:07.364041 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:07.362545 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9cd41c0-eeee-4253-b07b-3e93b15a60fd-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h\" (UID: \"d9cd41c0-eeee-4253-b07b-3e93b15a60fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h" Jul 30 00:24:07.372026 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:07.371995 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9cd41c0-eeee-4253-b07b-3e93b15a60fd-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h\" (UID: \"d9cd41c0-eeee-4253-b07b-3e93b15a60fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h" Jul 30 00:24:07.372155 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:07.372038 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tblp2\" (UniqueName: \"kubernetes.io/projected/d9cd41c0-eeee-4253-b07b-3e93b15a60fd-kube-api-access-tblp2\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h\" (UID: \"d9cd41c0-eeee-4253-b07b-3e93b15a60fd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h" Jul 30 00:24:07.519712 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:07.519677 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:24:07.543488 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:07.543463 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h" Jul 30 00:24:07.691039 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:07.690998 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h"] Jul 30 00:24:07.694469 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:24:07.694442 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9cd41c0_eeee_4253_b07b_3e93b15a60fd.slice/crio-81737b08b6b98efc95b3d2cf5a0f0ba61e7dc4d4c18c1e716571401b199dc107 WatchSource:0}: Error finding container 81737b08b6b98efc95b3d2cf5a0f0ba61e7dc4d4c18c1e716571401b199dc107: Status 404 returned error can't find the container with id 81737b08b6b98efc95b3d2cf5a0f0ba61e7dc4d4c18c1e716571401b199dc107 Jul 30 00:24:08.296233 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:08.296197 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h" event={"ID":"d9cd41c0-eeee-4253-b07b-3e93b15a60fd","Type":"ContainerStarted","Data":"81737b08b6b98efc95b3d2cf5a0f0ba61e7dc4d4c18c1e716571401b199dc107"} Jul 30 00:24:09.296697 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:09.296657 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:24:09.488219 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:09.488185 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:24:09.496973 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:09.496894 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-jbf4x"] Jul 30 00:24:09.500619 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:09.500597 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jbf4x" Jul 30 00:24:09.503726 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:09.503701 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 30 00:24:09.503862 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:09.503720 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-8d54l\"" Jul 30 00:24:09.517246 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:09.517203 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-jbf4x"] Jul 30 00:24:09.565207 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:09.565115 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:24:09.568178 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:09.568150 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrmtv\" (UniqueName: \"kubernetes.io/projected/9ed1c3a7-6249-4633-b09a-f38dfb35647f-kube-api-access-wrmtv\") pod \"llmisvc-controller-manager-777c6bd7d5-jbf4x\" (UID: \"9ed1c3a7-6249-4633-b09a-f38dfb35647f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jbf4x" Jul 30 00:24:09.568314 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:09.568197 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9ed1c3a7-6249-4633-b09a-f38dfb35647f-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-jbf4x\" (UID: \"9ed1c3a7-6249-4633-b09a-f38dfb35647f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jbf4x" Jul 30 00:24:09.669566 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:09.669139 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrmtv\" (UniqueName: \"kubernetes.io/projected/9ed1c3a7-6249-4633-b09a-f38dfb35647f-kube-api-access-wrmtv\") pod \"llmisvc-controller-manager-777c6bd7d5-jbf4x\" (UID: \"9ed1c3a7-6249-4633-b09a-f38dfb35647f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jbf4x" Jul 30 00:24:09.669566 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:09.669203 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9ed1c3a7-6249-4633-b09a-f38dfb35647f-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-jbf4x\" (UID: \"9ed1c3a7-6249-4633-b09a-f38dfb35647f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jbf4x" Jul 30 00:24:09.672909 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:09.672879 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9ed1c3a7-6249-4633-b09a-f38dfb35647f-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-jbf4x\" (UID: \"9ed1c3a7-6249-4633-b09a-f38dfb35647f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jbf4x" Jul 30 00:24:09.677009 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:09.676911 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:24:09.680548 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:09.680498 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrmtv\" (UniqueName: \"kubernetes.io/projected/9ed1c3a7-6249-4633-b09a-f38dfb35647f-kube-api-access-wrmtv\") pod \"llmisvc-controller-manager-777c6bd7d5-jbf4x\" (UID: \"9ed1c3a7-6249-4633-b09a-f38dfb35647f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jbf4x" Jul 30 00:24:09.801205 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:09.801172 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:24:09.818084 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:09.818013 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jbf4x" Jul 30 00:24:09.930401 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:09.930339 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:24:09.969623 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:09.969595 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-jbf4x"] Jul 30 00:24:09.972719 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:24:09.972685 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod9ed1c3a7_6249_4633_b09a_f38dfb35647f.slice/crio-279ed9db60595f37a987c680cee68c7b573ad0102c658c2b35bd3a601c638203 WatchSource:0}: Error finding container 279ed9db60595f37a987c680cee68c7b573ad0102c658c2b35bd3a601c638203: Status 404 returned error can't find the container with id 279ed9db60595f37a987c680cee68c7b573ad0102c658c2b35bd3a601c638203 Jul 30 00:24:10.306025 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:10.305991 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jbf4x" event={"ID":"9ed1c3a7-6249-4633-b09a-f38dfb35647f","Type":"ContainerStarted","Data":"279ed9db60595f37a987c680cee68c7b573ad0102c658c2b35bd3a601c638203"} Jul 30 00:24:11.465269 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:11.465238 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:24:11.465539 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:11.465307 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:24:11.465539 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:11.465338 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:24:11.812065 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:11.812030 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6w2cb"] Jul 30 00:24:11.812278 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:11.812231 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6w2cb" podUID="7ed721de-9dd1-48d0-86ab-5822cb6c6c4b" containerName="discovery" containerID="cri-o://eadcd51d6c25a0670a66ed2cbeddfe353db293896e249f1ae93aa72bff2da631" gracePeriod=30 Jul 30 00:24:12.068556 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:12.068501 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6w2cb" Jul 30 00:24:12.190650 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:12.190601 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7ed721de-9dd1-48d0-86ab-5822cb6c6c4b-istio-kubeconfig\") pod \"7ed721de-9dd1-48d0-86ab-5822cb6c6c4b\" (UID: \"7ed721de-9dd1-48d0-86ab-5822cb6c6c4b\") " Jul 30 00:24:12.190865 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:12.190673 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7ed721de-9dd1-48d0-86ab-5822cb6c6c4b-istio-csr-dns-cert\") pod \"7ed721de-9dd1-48d0-86ab-5822cb6c6c4b\" (UID: \"7ed721de-9dd1-48d0-86ab-5822cb6c6c4b\") " Jul 30 00:24:12.190865 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:12.190695 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7ed721de-9dd1-48d0-86ab-5822cb6c6c4b-cacerts\") pod \"7ed721de-9dd1-48d0-86ab-5822cb6c6c4b\" (UID: \"7ed721de-9dd1-48d0-86ab-5822cb6c6c4b\") " Jul 30 00:24:12.190865 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:12.190730 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ed721de-9dd1-48d0-86ab-5822cb6c6c4b-istio-token\") pod \"7ed721de-9dd1-48d0-86ab-5822cb6c6c4b\" (UID: \"7ed721de-9dd1-48d0-86ab-5822cb6c6c4b\") " Jul 30 00:24:12.191118 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:12.190873 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6rmr5\" (UniqueName: \"kubernetes.io/projected/7ed721de-9dd1-48d0-86ab-5822cb6c6c4b-kube-api-access-6rmr5\") pod \"7ed721de-9dd1-48d0-86ab-5822cb6c6c4b\" (UID: \"7ed721de-9dd1-48d0-86ab-5822cb6c6c4b\") " Jul 30 00:24:12.191118 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:12.190923 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ed721de-9dd1-48d0-86ab-5822cb6c6c4b-local-certs\") pod \"7ed721de-9dd1-48d0-86ab-5822cb6c6c4b\" (UID: \"7ed721de-9dd1-48d0-86ab-5822cb6c6c4b\") " Jul 30 00:24:12.191118 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:12.190954 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7ed721de-9dd1-48d0-86ab-5822cb6c6c4b-istio-csr-ca-configmap\") pod \"7ed721de-9dd1-48d0-86ab-5822cb6c6c4b\" (UID: \"7ed721de-9dd1-48d0-86ab-5822cb6c6c4b\") " Jul 30 00:24:12.191600 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:12.191567 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ed721de-9dd1-48d0-86ab-5822cb6c6c4b-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "7ed721de-9dd1-48d0-86ab-5822cb6c6c4b" (UID: "7ed721de-9dd1-48d0-86ab-5822cb6c6c4b"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 00:24:12.193155 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:12.193105 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ed721de-9dd1-48d0-86ab-5822cb6c6c4b-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "7ed721de-9dd1-48d0-86ab-5822cb6c6c4b" (UID: "7ed721de-9dd1-48d0-86ab-5822cb6c6c4b"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 00:24:12.193262 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:12.193192 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ed721de-9dd1-48d0-86ab-5822cb6c6c4b-local-certs" (OuterVolumeSpecName: "local-certs") pod "7ed721de-9dd1-48d0-86ab-5822cb6c6c4b" (UID: "7ed721de-9dd1-48d0-86ab-5822cb6c6c4b"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:24:12.193327 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:12.193278 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ed721de-9dd1-48d0-86ab-5822cb6c6c4b-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "7ed721de-9dd1-48d0-86ab-5822cb6c6c4b" (UID: "7ed721de-9dd1-48d0-86ab-5822cb6c6c4b"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 00:24:12.193385 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:12.193330 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ed721de-9dd1-48d0-86ab-5822cb6c6c4b-istio-token" (OuterVolumeSpecName: "istio-token") pod "7ed721de-9dd1-48d0-86ab-5822cb6c6c4b" (UID: "7ed721de-9dd1-48d0-86ab-5822cb6c6c4b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 00:24:12.193385 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:12.193358 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ed721de-9dd1-48d0-86ab-5822cb6c6c4b-cacerts" (OuterVolumeSpecName: "cacerts") pod "7ed721de-9dd1-48d0-86ab-5822cb6c6c4b" (UID: "7ed721de-9dd1-48d0-86ab-5822cb6c6c4b"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 00:24:12.193482 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:12.193462 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ed721de-9dd1-48d0-86ab-5822cb6c6c4b-kube-api-access-6rmr5" (OuterVolumeSpecName: "kube-api-access-6rmr5") pod "7ed721de-9dd1-48d0-86ab-5822cb6c6c4b" (UID: "7ed721de-9dd1-48d0-86ab-5822cb6c6c4b"). InnerVolumeSpecName "kube-api-access-6rmr5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 00:24:12.292088 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:12.292049 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6rmr5\" (UniqueName: \"kubernetes.io/projected/7ed721de-9dd1-48d0-86ab-5822cb6c6c4b-kube-api-access-6rmr5\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:24:12.292088 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:12.292082 2563 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ed721de-9dd1-48d0-86ab-5822cb6c6c4b-local-certs\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:24:12.292088 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:12.292095 2563 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7ed721de-9dd1-48d0-86ab-5822cb6c6c4b-istio-csr-ca-configmap\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:24:12.292301 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:12.292107 2563 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7ed721de-9dd1-48d0-86ab-5822cb6c6c4b-istio-kubeconfig\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:24:12.292301 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:12.292119 2563 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7ed721de-9dd1-48d0-86ab-5822cb6c6c4b-istio-csr-dns-cert\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:24:12.292301 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:12.292131 2563 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7ed721de-9dd1-48d0-86ab-5822cb6c6c4b-cacerts\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:24:12.292301 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:12.292142 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ed721de-9dd1-48d0-86ab-5822cb6c6c4b-istio-token\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:24:12.317103 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:12.317075 2563 generic.go:356] "Generic (PLEG): container finished" podID="7ed721de-9dd1-48d0-86ab-5822cb6c6c4b" containerID="eadcd51d6c25a0670a66ed2cbeddfe353db293896e249f1ae93aa72bff2da631" exitCode=0 Jul 30 00:24:12.317249 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:12.317122 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6w2cb" Jul 30 00:24:12.317249 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:12.317145 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6w2cb" event={"ID":"7ed721de-9dd1-48d0-86ab-5822cb6c6c4b","Type":"ContainerDied","Data":"eadcd51d6c25a0670a66ed2cbeddfe353db293896e249f1ae93aa72bff2da631"} Jul 30 00:24:12.317249 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:12.317187 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6w2cb" event={"ID":"7ed721de-9dd1-48d0-86ab-5822cb6c6c4b","Type":"ContainerDied","Data":"4e686a2337e56b8a8ff92be1ff1bf7d844807e473b54410af5e34807a5f28f57"} Jul 30 00:24:12.317249 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:12.317215 2563 scope.go:117] "RemoveContainer" containerID="eadcd51d6c25a0670a66ed2cbeddfe353db293896e249f1ae93aa72bff2da631" Jul 30 00:24:12.318851 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:12.318788 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h" event={"ID":"d9cd41c0-eeee-4253-b07b-3e93b15a60fd","Type":"ContainerStarted","Data":"b3b4529735f6bf5516f0d41e3a9e5b31a86974902d6e33fa91cfa3ce0fdec874"} Jul 30 00:24:12.326490 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:12.326473 2563 scope.go:117] "RemoveContainer" containerID="eadcd51d6c25a0670a66ed2cbeddfe353db293896e249f1ae93aa72bff2da631" Jul 30 00:24:12.326721 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:24:12.326705 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eadcd51d6c25a0670a66ed2cbeddfe353db293896e249f1ae93aa72bff2da631\": container with ID starting with eadcd51d6c25a0670a66ed2cbeddfe353db293896e249f1ae93aa72bff2da631 not found: ID does not exist" containerID="eadcd51d6c25a0670a66ed2cbeddfe353db293896e249f1ae93aa72bff2da631" Jul 30 00:24:12.326806 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:12.326755 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eadcd51d6c25a0670a66ed2cbeddfe353db293896e249f1ae93aa72bff2da631"} err="failed to get container status \"eadcd51d6c25a0670a66ed2cbeddfe353db293896e249f1ae93aa72bff2da631\": rpc error: code = NotFound desc = could not find container \"eadcd51d6c25a0670a66ed2cbeddfe353db293896e249f1ae93aa72bff2da631\": container with ID starting with eadcd51d6c25a0670a66ed2cbeddfe353db293896e249f1ae93aa72bff2da631 not found: ID does not exist" Jul 30 00:24:12.346960 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:12.346914 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h" podStartSLOduration=1.5784479980000001 podStartE2EDuration="5.346900496s" totalImagesPullingTime="3.768452498s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:24:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:24:12.346478548 +0000 UTC m=+416.405234517" watchObservedRunningTime="2026-07-30 00:24:12.346900496 +0000 UTC m=+416.405656465" Jul 30 00:24:12.361994 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:12.361968 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6w2cb"] Jul 30 00:24:12.365915 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:12.365884 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6w2cb"] Jul 30 00:24:12.479311 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:12.479281 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ed721de-9dd1-48d0-86ab-5822cb6c6c4b" path="/var/lib/kubelet/pods/7ed721de-9dd1-48d0-86ab-5822cb6c6c4b/volumes" Jul 30 00:24:12.544259 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:12.544229 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h" Jul 30 00:24:12.549303 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:12.549280 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h" Jul 30 00:24:13.268409 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:13.268379 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-ktnsg" Jul 30 00:24:13.325439 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:13.325405 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h" Jul 30 00:24:13.326466 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:13.326440 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h" Jul 30 00:24:15.282062 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:15.282035 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-xrx9m" Jul 30 00:24:15.337757 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:15.337707 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jbf4x" event={"ID":"9ed1c3a7-6249-4633-b09a-f38dfb35647f","Type":"ContainerStarted","Data":"ae2a5c090c514a546621df780646667a79940e9fda7811a2dd03820cc37ee672"} Jul 30 00:24:15.337978 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:15.337961 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jbf4x" Jul 30 00:24:15.366338 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:15.366285 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jbf4x" podStartSLOduration=1.66671404 podStartE2EDuration="6.366271256s" totalImagesPullingTime="4.699557216s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:24:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:24:15.365022244 +0000 UTC m=+419.423778226" watchObservedRunningTime="2026-07-30 00:24:15.366271256 +0000 UTC m=+419.425027224" Jul 30 00:24:29.964605 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:29.964567 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:24:30.024792 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:30.024759 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:24:34.919601 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:34.919572 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:24:34.975816 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:34.975775 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:24:46.344804 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:46.344706 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jbf4x" Jul 30 00:24:59.852637 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:59.852596 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:24:59.908904 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:59.908872 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7nvxk"] Jul 30 00:24:59.909204 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:59.909192 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ed721de-9dd1-48d0-86ab-5822cb6c6c4b" containerName="discovery" Jul 30 00:24:59.909250 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:59.909206 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ed721de-9dd1-48d0-86ab-5822cb6c6c4b" containerName="discovery" Jul 30 00:24:59.909284 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:59.909269 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ed721de-9dd1-48d0-86ab-5822cb6c6c4b" containerName="discovery" Jul 30 00:24:59.911303 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:59.911283 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7nvxk" Jul 30 00:24:59.913883 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:59.913862 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 30 00:24:59.914702 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:59.914682 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 30 00:24:59.914797 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:59.914682 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-q85dr\"" Jul 30 00:24:59.925508 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:59.925487 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7nvxk"] Jul 30 00:24:59.939284 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:59.939257 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:24:59.994335 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:59.994302 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvms5\" (UniqueName: \"kubernetes.io/projected/fdcb9a46-b068-4d9a-a6c0-633b0ae5a1dc-kube-api-access-qvms5\") pod \"kuadrant-operator-catalog-7nvxk\" (UID: \"fdcb9a46-b068-4d9a-a6c0-633b0ae5a1dc\") " pod="kuadrant-system/kuadrant-operator-catalog-7nvxk" Jul 30 00:24:59.995139 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:24:59.995102 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:25:00.095447 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:00.095414 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvms5\" (UniqueName: \"kubernetes.io/projected/fdcb9a46-b068-4d9a-a6c0-633b0ae5a1dc-kube-api-access-qvms5\") pod \"kuadrant-operator-catalog-7nvxk\" (UID: \"fdcb9a46-b068-4d9a-a6c0-633b0ae5a1dc\") " pod="kuadrant-system/kuadrant-operator-catalog-7nvxk" Jul 30 00:25:00.115166 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:00.115094 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvms5\" (UniqueName: \"kubernetes.io/projected/fdcb9a46-b068-4d9a-a6c0-633b0ae5a1dc-kube-api-access-qvms5\") pod \"kuadrant-operator-catalog-7nvxk\" (UID: \"fdcb9a46-b068-4d9a-a6c0-633b0ae5a1dc\") " pod="kuadrant-system/kuadrant-operator-catalog-7nvxk" Jul 30 00:25:00.221805 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:00.221764 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7nvxk" Jul 30 00:25:00.277402 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:00.277373 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7nvxk"] Jul 30 00:25:00.351814 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:00.351792 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7nvxk"] Jul 30 00:25:00.353623 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:25:00.353597 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdcb9a46_b068_4d9a_a6c0_633b0ae5a1dc.slice/crio-ff122fb1f7e052cfac159de5cadbbd0a31fc140d17ba1adb0b24c06043d5afc3 WatchSource:0}: Error finding container ff122fb1f7e052cfac159de5cadbbd0a31fc140d17ba1adb0b24c06043d5afc3: Status 404 returned error can't find the container with id ff122fb1f7e052cfac159de5cadbbd0a31fc140d17ba1adb0b24c06043d5afc3 Jul 30 00:25:00.485831 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:00.485763 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xjfmq"] Jul 30 00:25:00.488524 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:00.488507 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-xjfmq" Jul 30 00:25:00.496947 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:00.496919 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xjfmq"] Jul 30 00:25:00.497644 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:00.497612 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dnzl9\" (UniqueName: \"kubernetes.io/projected/5f54c640-8113-4b8b-831e-57adaed12fa1-kube-api-access-dnzl9\") pod \"kuadrant-operator-catalog-xjfmq\" (UID: \"5f54c640-8113-4b8b-831e-57adaed12fa1\") " pod="kuadrant-system/kuadrant-operator-catalog-xjfmq" Jul 30 00:25:00.542809 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:00.542778 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7nvxk" event={"ID":"fdcb9a46-b068-4d9a-a6c0-633b0ae5a1dc","Type":"ContainerStarted","Data":"ff122fb1f7e052cfac159de5cadbbd0a31fc140d17ba1adb0b24c06043d5afc3"} Jul 30 00:25:00.598328 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:00.598294 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnzl9\" (UniqueName: \"kubernetes.io/projected/5f54c640-8113-4b8b-831e-57adaed12fa1-kube-api-access-dnzl9\") pod \"kuadrant-operator-catalog-xjfmq\" (UID: \"5f54c640-8113-4b8b-831e-57adaed12fa1\") " pod="kuadrant-system/kuadrant-operator-catalog-xjfmq" Jul 30 00:25:00.606177 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:00.606155 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnzl9\" (UniqueName: \"kubernetes.io/projected/5f54c640-8113-4b8b-831e-57adaed12fa1-kube-api-access-dnzl9\") pod \"kuadrant-operator-catalog-xjfmq\" (UID: \"5f54c640-8113-4b8b-831e-57adaed12fa1\") " pod="kuadrant-system/kuadrant-operator-catalog-xjfmq" Jul 30 00:25:00.798457 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:00.798420 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-xjfmq" Jul 30 00:25:00.922820 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:00.922794 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xjfmq"] Jul 30 00:25:00.924674 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:25:00.924649 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f54c640_8113_4b8b_831e_57adaed12fa1.slice/crio-dc47e895bad2c06950401d023e4f44519f2d294ef0c77805cc9ba0ac13a21b8a WatchSource:0}: Error finding container dc47e895bad2c06950401d023e4f44519f2d294ef0c77805cc9ba0ac13a21b8a: Status 404 returned error can't find the container with id dc47e895bad2c06950401d023e4f44519f2d294ef0c77805cc9ba0ac13a21b8a Jul 30 00:25:01.551097 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:01.551054 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-xjfmq" event={"ID":"5f54c640-8113-4b8b-831e-57adaed12fa1","Type":"ContainerStarted","Data":"dc47e895bad2c06950401d023e4f44519f2d294ef0c77805cc9ba0ac13a21b8a"} Jul 30 00:25:03.563414 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:03.563380 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-xjfmq" event={"ID":"5f54c640-8113-4b8b-831e-57adaed12fa1","Type":"ContainerStarted","Data":"47ad1030fd2ace46343a6eb9dd0ce1f1094f64f5f04216ee7ed04b41c46455ed"} Jul 30 00:25:03.564785 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:03.564761 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7nvxk" event={"ID":"fdcb9a46-b068-4d9a-a6c0-633b0ae5a1dc","Type":"ContainerStarted","Data":"c2814935c9f9a7aff34969912b8019d6344ead4ae495a15d90f2ec753607dbba"} Jul 30 00:25:03.564891 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:03.564817 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-7nvxk" podUID="fdcb9a46-b068-4d9a-a6c0-633b0ae5a1dc" containerName="registry-server" containerID="cri-o://c2814935c9f9a7aff34969912b8019d6344ead4ae495a15d90f2ec753607dbba" gracePeriod=2 Jul 30 00:25:03.578146 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:03.578106 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-xjfmq" podStartSLOduration=1.7413484700000001 podStartE2EDuration="3.578092342s" totalImagesPullingTime="1.836743872s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:25:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:25:03.577151691 +0000 UTC m=+467.635907660" watchObservedRunningTime="2026-07-30 00:25:03.578092342 +0000 UTC m=+467.636848310" Jul 30 00:25:03.591438 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:03.591398 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-7nvxk" podStartSLOduration=2.272725839 podStartE2EDuration="4.591383736s" totalImagesPullingTime="2.318657897s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:24:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:25:03.590190633 +0000 UTC m=+467.648946603" watchObservedRunningTime="2026-07-30 00:25:03.591383736 +0000 UTC m=+467.650139704" Jul 30 00:25:03.808886 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:03.808858 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7nvxk" Jul 30 00:25:03.834461 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:03.834398 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qvms5\" (UniqueName: \"kubernetes.io/projected/fdcb9a46-b068-4d9a-a6c0-633b0ae5a1dc-kube-api-access-qvms5\") pod \"fdcb9a46-b068-4d9a-a6c0-633b0ae5a1dc\" (UID: \"fdcb9a46-b068-4d9a-a6c0-633b0ae5a1dc\") " Jul 30 00:25:03.836492 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:03.836466 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdcb9a46-b068-4d9a-a6c0-633b0ae5a1dc-kube-api-access-qvms5" (OuterVolumeSpecName: "kube-api-access-qvms5") pod "fdcb9a46-b068-4d9a-a6c0-633b0ae5a1dc" (UID: "fdcb9a46-b068-4d9a-a6c0-633b0ae5a1dc"). InnerVolumeSpecName "kube-api-access-qvms5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 00:25:03.934944 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:03.934915 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qvms5\" (UniqueName: \"kubernetes.io/projected/fdcb9a46-b068-4d9a-a6c0-633b0ae5a1dc-kube-api-access-qvms5\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:25:04.570521 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:04.570481 2563 generic.go:356] "Generic (PLEG): container finished" podID="fdcb9a46-b068-4d9a-a6c0-633b0ae5a1dc" containerID="c2814935c9f9a7aff34969912b8019d6344ead4ae495a15d90f2ec753607dbba" exitCode=0 Jul 30 00:25:04.570521 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:04.570523 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7nvxk" Jul 30 00:25:04.570994 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:04.570558 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7nvxk" event={"ID":"fdcb9a46-b068-4d9a-a6c0-633b0ae5a1dc","Type":"ContainerDied","Data":"c2814935c9f9a7aff34969912b8019d6344ead4ae495a15d90f2ec753607dbba"} Jul 30 00:25:04.570994 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:04.570605 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7nvxk" event={"ID":"fdcb9a46-b068-4d9a-a6c0-633b0ae5a1dc","Type":"ContainerDied","Data":"ff122fb1f7e052cfac159de5cadbbd0a31fc140d17ba1adb0b24c06043d5afc3"} Jul 30 00:25:04.570994 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:04.570628 2563 scope.go:117] "RemoveContainer" containerID="c2814935c9f9a7aff34969912b8019d6344ead4ae495a15d90f2ec753607dbba" Jul 30 00:25:04.579784 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:04.579770 2563 scope.go:117] "RemoveContainer" containerID="c2814935c9f9a7aff34969912b8019d6344ead4ae495a15d90f2ec753607dbba" Jul 30 00:25:04.580105 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:25:04.580088 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2814935c9f9a7aff34969912b8019d6344ead4ae495a15d90f2ec753607dbba\": container with ID starting with c2814935c9f9a7aff34969912b8019d6344ead4ae495a15d90f2ec753607dbba not found: ID does not exist" containerID="c2814935c9f9a7aff34969912b8019d6344ead4ae495a15d90f2ec753607dbba" Jul 30 00:25:04.580173 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:04.580113 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2814935c9f9a7aff34969912b8019d6344ead4ae495a15d90f2ec753607dbba"} err="failed to get container status \"c2814935c9f9a7aff34969912b8019d6344ead4ae495a15d90f2ec753607dbba\": rpc error: code = NotFound desc = could not find container \"c2814935c9f9a7aff34969912b8019d6344ead4ae495a15d90f2ec753607dbba\": container with ID starting with c2814935c9f9a7aff34969912b8019d6344ead4ae495a15d90f2ec753607dbba not found: ID does not exist" Jul 30 00:25:04.586261 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:04.586237 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7nvxk"] Jul 30 00:25:04.591694 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:04.591672 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7nvxk"] Jul 30 00:25:06.475181 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:06.475148 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fdcb9a46-b068-4d9a-a6c0-633b0ae5a1dc" path="/var/lib/kubelet/pods/fdcb9a46-b068-4d9a-a6c0-633b0ae5a1dc/volumes" Jul 30 00:25:10.799011 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:10.798982 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-xjfmq" Jul 30 00:25:10.799400 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:10.799032 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-xjfmq" Jul 30 00:25:10.820132 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:10.820108 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-xjfmq" Jul 30 00:25:11.623031 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:11.623004 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-xjfmq" Jul 30 00:25:25.580247 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:25.580202 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:25:27.924427 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:27.924397 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:25:28.035425 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:28.035388 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:25:29.105238 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:29.105199 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:25:31.729554 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:31.729517 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-d9rft"] Jul 30 00:25:31.730073 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:31.730052 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdcb9a46-b068-4d9a-a6c0-633b0ae5a1dc" containerName="registry-server" Jul 30 00:25:31.730153 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:31.730077 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdcb9a46-b068-4d9a-a6c0-633b0ae5a1dc" containerName="registry-server" Jul 30 00:25:31.730205 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:31.730174 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="fdcb9a46-b068-4d9a-a6c0-633b0ae5a1dc" containerName="registry-server" Jul 30 00:25:31.734353 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:31.734332 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-d9rft" Jul 30 00:25:31.736926 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:31.736905 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 30 00:25:31.736926 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:31.736915 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-hzlh6\"" Jul 30 00:25:31.746010 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:31.745988 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-d9rft"] Jul 30 00:25:31.866262 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:31.866226 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pp6v9\" (UniqueName: \"kubernetes.io/projected/3d79d9f4-f2c8-461b-8f97-fdbe48891d63-kube-api-access-pp6v9\") pod \"dns-operator-controller-manager-864dd78c4c-d9rft\" (UID: \"3d79d9f4-f2c8-461b-8f97-fdbe48891d63\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-d9rft" Jul 30 00:25:31.966858 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:31.966828 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pp6v9\" (UniqueName: \"kubernetes.io/projected/3d79d9f4-f2c8-461b-8f97-fdbe48891d63-kube-api-access-pp6v9\") pod \"dns-operator-controller-manager-864dd78c4c-d9rft\" (UID: \"3d79d9f4-f2c8-461b-8f97-fdbe48891d63\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-d9rft" Jul 30 00:25:31.976835 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:31.976803 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pp6v9\" (UniqueName: \"kubernetes.io/projected/3d79d9f4-f2c8-461b-8f97-fdbe48891d63-kube-api-access-pp6v9\") pod \"dns-operator-controller-manager-864dd78c4c-d9rft\" (UID: \"3d79d9f4-f2c8-461b-8f97-fdbe48891d63\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-d9rft" Jul 30 00:25:32.044049 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:32.044015 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-d9rft" Jul 30 00:25:32.176107 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:32.176037 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-d9rft"] Jul 30 00:25:32.178484 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:25:32.178449 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d79d9f4_f2c8_461b_8f97_fdbe48891d63.slice/crio-3bc07a88f83735d574962f6161cfc4c324b4eadc27f7b3baa25a83956f28e9e2 WatchSource:0}: Error finding container 3bc07a88f83735d574962f6161cfc4c324b4eadc27f7b3baa25a83956f28e9e2: Status 404 returned error can't find the container with id 3bc07a88f83735d574962f6161cfc4c324b4eadc27f7b3baa25a83956f28e9e2 Jul 30 00:25:32.697251 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:32.697221 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-d9rft" event={"ID":"3d79d9f4-f2c8-461b-8f97-fdbe48891d63","Type":"ContainerStarted","Data":"3bc07a88f83735d574962f6161cfc4c324b4eadc27f7b3baa25a83956f28e9e2"} Jul 30 00:25:33.294531 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:33.294397 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:25:34.711860 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:34.711825 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-d9rft" event={"ID":"3d79d9f4-f2c8-461b-8f97-fdbe48891d63","Type":"ContainerStarted","Data":"a2be9b3928b9c90963950e317f81ff5c613e395c1cf5e96b6a6e56dcf994cd8b"} Jul 30 00:25:34.712249 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:34.711943 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-d9rft" Jul 30 00:25:34.728703 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:34.728650 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-d9rft" podStartSLOduration=2.004006483 podStartE2EDuration="3.728635355s" totalImagesPullingTime="1.724628872s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:25:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:25:34.727094327 +0000 UTC m=+498.785850295" watchObservedRunningTime="2026-07-30 00:25:34.728635355 +0000 UTC m=+498.787391323" Jul 30 00:25:35.142304 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:35.138608 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l5gwr"] Jul 30 00:25:35.143308 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:35.143286 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l5gwr" Jul 30 00:25:35.145854 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:35.145832 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-q4h4q\"" Jul 30 00:25:35.153695 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:35.153673 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l5gwr"] Jul 30 00:25:35.299408 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:35.299366 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rhzs\" (UniqueName: \"kubernetes.io/projected/18759d20-e435-4a8e-8e80-d1fe4b49c3ca-kube-api-access-6rhzs\") pod \"limitador-operator-controller-manager-859d7b78bd-l5gwr\" (UID: \"18759d20-e435-4a8e-8e80-d1fe4b49c3ca\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l5gwr" Jul 30 00:25:35.400152 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:35.400071 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rhzs\" (UniqueName: \"kubernetes.io/projected/18759d20-e435-4a8e-8e80-d1fe4b49c3ca-kube-api-access-6rhzs\") pod \"limitador-operator-controller-manager-859d7b78bd-l5gwr\" (UID: \"18759d20-e435-4a8e-8e80-d1fe4b49c3ca\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l5gwr" Jul 30 00:25:35.414265 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:35.414225 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rhzs\" (UniqueName: \"kubernetes.io/projected/18759d20-e435-4a8e-8e80-d1fe4b49c3ca-kube-api-access-6rhzs\") pod \"limitador-operator-controller-manager-859d7b78bd-l5gwr\" (UID: \"18759d20-e435-4a8e-8e80-d1fe4b49c3ca\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l5gwr" Jul 30 00:25:35.453723 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:35.453691 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l5gwr" Jul 30 00:25:35.582192 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:35.582169 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l5gwr"] Jul 30 00:25:35.584040 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:25:35.584008 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18759d20_e435_4a8e_8e80_d1fe4b49c3ca.slice/crio-a7487b65399923d0c595290834201d15420f978f80953ca15c96413adb9bdd91 WatchSource:0}: Error finding container a7487b65399923d0c595290834201d15420f978f80953ca15c96413adb9bdd91: Status 404 returned error can't find the container with id a7487b65399923d0c595290834201d15420f978f80953ca15c96413adb9bdd91 Jul 30 00:25:35.717917 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:35.717826 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l5gwr" event={"ID":"18759d20-e435-4a8e-8e80-d1fe4b49c3ca","Type":"ContainerStarted","Data":"a7487b65399923d0c595290834201d15420f978f80953ca15c96413adb9bdd91"} Jul 30 00:25:35.732811 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:35.732779 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:25:35.901882 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:35.901844 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:25:36.506569 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:36.506505 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:25:36.607680 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:36.607645 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:25:36.706595 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:36.706555 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:25:37.133893 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:37.133865 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:25:37.729682 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:37.729644 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l5gwr" event={"ID":"18759d20-e435-4a8e-8e80-d1fe4b49c3ca","Type":"ContainerStarted","Data":"322067ee07857bff8d4be5b1d7eb4c5bc0dbb1c834aa4893c04f47028a176179"} Jul 30 00:25:37.729904 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:37.729712 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l5gwr" Jul 30 00:25:37.747592 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:37.747543 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l5gwr" podStartSLOduration=1.2194985680000001 podStartE2EDuration="2.747526993s" totalImagesPullingTime="1.528028425s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:25:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:25:37.746727595 +0000 UTC m=+501.805483563" watchObservedRunningTime="2026-07-30 00:25:37.747526993 +0000 UTC m=+501.806282961" Jul 30 00:25:39.886569 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:39.886529 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7px28"] Jul 30 00:25:39.889850 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:39.889831 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7px28" Jul 30 00:25:39.892425 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:39.892401 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-hmwnt\"" Jul 30 00:25:39.902345 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:39.902322 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7px28"] Jul 30 00:25:40.041059 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:40.041022 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/700f1769-b978-4f9b-862e-22e8f20d4c86-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7px28\" (UID: \"700f1769-b978-4f9b-862e-22e8f20d4c86\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7px28" Jul 30 00:25:40.041238 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:40.041154 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxk5n\" (UniqueName: \"kubernetes.io/projected/700f1769-b978-4f9b-862e-22e8f20d4c86-kube-api-access-mxk5n\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7px28\" (UID: \"700f1769-b978-4f9b-862e-22e8f20d4c86\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7px28" Jul 30 00:25:40.142176 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:40.142080 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxk5n\" (UniqueName: \"kubernetes.io/projected/700f1769-b978-4f9b-862e-22e8f20d4c86-kube-api-access-mxk5n\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7px28\" (UID: \"700f1769-b978-4f9b-862e-22e8f20d4c86\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7px28" Jul 30 00:25:40.142176 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:40.142133 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/700f1769-b978-4f9b-862e-22e8f20d4c86-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7px28\" (UID: \"700f1769-b978-4f9b-862e-22e8f20d4c86\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7px28" Jul 30 00:25:40.142543 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:40.142519 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/700f1769-b978-4f9b-862e-22e8f20d4c86-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7px28\" (UID: \"700f1769-b978-4f9b-862e-22e8f20d4c86\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7px28" Jul 30 00:25:40.151151 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:40.151127 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxk5n\" (UniqueName: \"kubernetes.io/projected/700f1769-b978-4f9b-862e-22e8f20d4c86-kube-api-access-mxk5n\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7px28\" (UID: \"700f1769-b978-4f9b-862e-22e8f20d4c86\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7px28" Jul 30 00:25:40.199024 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:40.198991 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7px28" Jul 30 00:25:40.290450 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:40.290390 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:25:40.348200 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:40.348175 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7px28"] Jul 30 00:25:40.350419 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:25:40.350395 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod700f1769_b978_4f9b_862e_22e8f20d4c86.slice/crio-e56ec0a32ca0d88750ea2cd3493abe66c215a1b2c17de8b946b2eef8b0ac5509 WatchSource:0}: Error finding container e56ec0a32ca0d88750ea2cd3493abe66c215a1b2c17de8b946b2eef8b0ac5509: Status 404 returned error can't find the container with id e56ec0a32ca0d88750ea2cd3493abe66c215a1b2c17de8b946b2eef8b0ac5509 Jul 30 00:25:40.726729 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:40.726671 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:25:40.744655 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:40.744625 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7px28" event={"ID":"700f1769-b978-4f9b-862e-22e8f20d4c86","Type":"ContainerStarted","Data":"e56ec0a32ca0d88750ea2cd3493abe66c215a1b2c17de8b946b2eef8b0ac5509"} Jul 30 00:25:41.005992 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:41.005957 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:25:43.339608 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:43.339571 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-z7sxq"] Jul 30 00:25:43.343397 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:43.343371 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-z7sxq" Jul 30 00:25:43.346313 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:43.346286 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-t9dgg\"" Jul 30 00:25:43.359506 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:43.359477 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-z7sxq"] Jul 30 00:25:43.471558 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:43.471525 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjd5m\" (UniqueName: \"kubernetes.io/projected/eab785d4-4f65-4010-928f-1ff5e99e3ff1-kube-api-access-mjd5m\") pod \"authorino-operator-75cc8d5cb7-z7sxq\" (UID: \"eab785d4-4f65-4010-928f-1ff5e99e3ff1\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-z7sxq" Jul 30 00:25:43.572823 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:43.572784 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjd5m\" (UniqueName: \"kubernetes.io/projected/eab785d4-4f65-4010-928f-1ff5e99e3ff1-kube-api-access-mjd5m\") pod \"authorino-operator-75cc8d5cb7-z7sxq\" (UID: \"eab785d4-4f65-4010-928f-1ff5e99e3ff1\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-z7sxq" Jul 30 00:25:43.585394 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:43.585362 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjd5m\" (UniqueName: \"kubernetes.io/projected/eab785d4-4f65-4010-928f-1ff5e99e3ff1-kube-api-access-mjd5m\") pod \"authorino-operator-75cc8d5cb7-z7sxq\" (UID: \"eab785d4-4f65-4010-928f-1ff5e99e3ff1\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-z7sxq" Jul 30 00:25:43.656638 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:43.656568 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-z7sxq" Jul 30 00:25:43.801511 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:43.801484 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-z7sxq"] Jul 30 00:25:43.804255 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:25:43.804226 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeab785d4_4f65_4010_928f_1ff5e99e3ff1.slice/crio-46b5b1bfcc7188ba97f00928d59e8528b35ba04b83f26641b3d081d09c63464c WatchSource:0}: Error finding container 46b5b1bfcc7188ba97f00928d59e8528b35ba04b83f26641b3d081d09c63464c: Status 404 returned error can't find the container with id 46b5b1bfcc7188ba97f00928d59e8528b35ba04b83f26641b3d081d09c63464c Jul 30 00:25:44.769194 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:44.769132 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-z7sxq" event={"ID":"eab785d4-4f65-4010-928f-1ff5e99e3ff1","Type":"ContainerStarted","Data":"46b5b1bfcc7188ba97f00928d59e8528b35ba04b83f26641b3d081d09c63464c"} Jul 30 00:25:45.720848 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:45.720818 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-d9rft" Jul 30 00:25:46.067184 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:46.067144 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:25:46.158684 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:46.158648 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:25:47.787058 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:47.787017 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7px28" event={"ID":"700f1769-b978-4f9b-862e-22e8f20d4c86","Type":"ContainerStarted","Data":"ef3617f735eef64e95e48a589a568ba9668c746194dd119ab15d95e1b69d4abd"} Jul 30 00:25:47.787700 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:47.787125 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7px28" Jul 30 00:25:47.788629 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:47.788606 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-z7sxq" event={"ID":"eab785d4-4f65-4010-928f-1ff5e99e3ff1","Type":"ContainerStarted","Data":"f624e218ae9d7c640e977a205284f2a8cd536d107d8dba45d6ec115399476de2"} Jul 30 00:25:47.788725 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:47.788702 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-z7sxq" Jul 30 00:25:47.804874 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:47.804837 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7px28" podStartSLOduration=2.38771107 podStartE2EDuration="8.804824572s" totalImagesPullingTime="6.417113502s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:25:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:25:47.803690239 +0000 UTC m=+511.862446207" watchObservedRunningTime="2026-07-30 00:25:47.804824572 +0000 UTC m=+511.863580539" Jul 30 00:25:47.819574 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:47.819535 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-z7sxq" podStartSLOduration=1.8137370339999999 podStartE2EDuration="4.819522296s" totalImagesPullingTime="3.005785262s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:25:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:25:47.818903335 +0000 UTC m=+511.877659305" watchObservedRunningTime="2026-07-30 00:25:47.819522296 +0000 UTC m=+511.878278662" Jul 30 00:25:48.736544 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:48.736510 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l5gwr" Jul 30 00:25:48.907298 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:48.907268 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:25:49.004322 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:49.004288 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:25:49.633148 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:49.633110 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:25:49.888819 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:49.888699 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:25:52.302131 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:52.302075 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:25:52.360061 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:52.360026 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:25:58.796641 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:58.796609 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7px28" Jul 30 00:25:58.797057 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:25:58.796686 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-z7sxq" Jul 30 00:26:00.544398 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:00.544361 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7px28"] Jul 30 00:26:00.544983 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:00.544606 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7px28" podUID="700f1769-b978-4f9b-862e-22e8f20d4c86" containerName="manager" containerID="cri-o://ef3617f735eef64e95e48a589a568ba9668c746194dd119ab15d95e1b69d4abd" gracePeriod=2 Jul 30 00:26:00.560997 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:00.560966 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7px28"] Jul 30 00:26:00.568129 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:26:00.568082 2563 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"700f1769-b978-4f9b-862e-22e8f20d4c86\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-7px28\": pods \"kuadrant-operator-controller-manager-8684c9ff49-7px28\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7px28" Jul 30 00:26:00.574392 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:00.574366 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nqdrd"] Jul 30 00:26:00.574789 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:00.574773 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="700f1769-b978-4f9b-862e-22e8f20d4c86" containerName="manager" Jul 30 00:26:00.574789 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:00.574789 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="700f1769-b978-4f9b-862e-22e8f20d4c86" containerName="manager" Jul 30 00:26:00.574991 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:00.574865 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="700f1769-b978-4f9b-862e-22e8f20d4c86" containerName="manager" Jul 30 00:26:00.578215 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:00.578194 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l5gwr"] Jul 30 00:26:00.578311 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:00.578299 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nqdrd" Jul 30 00:26:00.578494 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:00.578464 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l5gwr" podUID="18759d20-e435-4a8e-8e80-d1fe4b49c3ca" containerName="manager" containerID="cri-o://322067ee07857bff8d4be5b1d7eb4c5bc0dbb1c834aa4893c04f47028a176179" gracePeriod=2 Jul 30 00:26:00.582909 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:00.582885 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l5gwr"] Jul 30 00:26:00.593138 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:00.593106 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nqdrd"] Jul 30 00:26:00.607601 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:00.607571 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cxzlt"] Jul 30 00:26:00.608039 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:00.608020 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18759d20-e435-4a8e-8e80-d1fe4b49c3ca" containerName="manager" Jul 30 00:26:00.608039 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:00.608040 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="18759d20-e435-4a8e-8e80-d1fe4b49c3ca" containerName="manager" Jul 30 00:26:00.608188 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:00.608132 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="18759d20-e435-4a8e-8e80-d1fe4b49c3ca" containerName="manager" Jul 30 00:26:00.609097 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:00.609074 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b39ed866-3890-4a9d-82b5-148dbf8e95b0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-nqdrd\" (UID: \"b39ed866-3890-4a9d-82b5-148dbf8e95b0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nqdrd" Jul 30 00:26:00.609186 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:00.609162 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2c46\" (UniqueName: \"kubernetes.io/projected/b39ed866-3890-4a9d-82b5-148dbf8e95b0-kube-api-access-g2c46\") pod \"kuadrant-operator-controller-manager-8684c9ff49-nqdrd\" (UID: \"b39ed866-3890-4a9d-82b5-148dbf8e95b0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nqdrd" Jul 30 00:26:00.611263 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:00.611245 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cxzlt" Jul 30 00:26:00.615153 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:26:00.615127 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-7px28\" is forbidden: User \"system:node:ip-10-0-133-10.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-10.ec2.internal' and this object" podUID="700f1769-b978-4f9b-862e-22e8f20d4c86" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7px28" Jul 30 00:26:00.617253 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:26:00.617224 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-l5gwr\" is forbidden: User \"system:node:ip-10-0-133-10.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-10.ec2.internal' and this object" podUID="18759d20-e435-4a8e-8e80-d1fe4b49c3ca" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l5gwr" Jul 30 00:26:00.619245 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:26:00.619220 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-7px28\" is forbidden: User \"system:node:ip-10-0-133-10.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-10.ec2.internal' and this object" podUID="700f1769-b978-4f9b-862e-22e8f20d4c86" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7px28" Jul 30 00:26:00.622358 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:00.622332 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cxzlt"] Jul 30 00:26:00.709578 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:00.709548 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2c46\" (UniqueName: \"kubernetes.io/projected/b39ed866-3890-4a9d-82b5-148dbf8e95b0-kube-api-access-g2c46\") pod \"kuadrant-operator-controller-manager-8684c9ff49-nqdrd\" (UID: \"b39ed866-3890-4a9d-82b5-148dbf8e95b0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nqdrd" Jul 30 00:26:00.709690 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:00.709608 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b39ed866-3890-4a9d-82b5-148dbf8e95b0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-nqdrd\" (UID: \"b39ed866-3890-4a9d-82b5-148dbf8e95b0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nqdrd" Jul 30 00:26:00.709690 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:00.709669 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6c6x9\" (UniqueName: \"kubernetes.io/projected/a38e3d81-da42-4aca-b7cd-7e4a3f27d6c9-kube-api-access-6c6x9\") pod \"limitador-operator-controller-manager-859d7b78bd-cxzlt\" (UID: \"a38e3d81-da42-4aca-b7cd-7e4a3f27d6c9\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cxzlt" Jul 30 00:26:00.709977 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:00.709959 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b39ed866-3890-4a9d-82b5-148dbf8e95b0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-nqdrd\" (UID: \"b39ed866-3890-4a9d-82b5-148dbf8e95b0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nqdrd" Jul 30 00:26:00.722351 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:00.722304 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2c46\" (UniqueName: \"kubernetes.io/projected/b39ed866-3890-4a9d-82b5-148dbf8e95b0-kube-api-access-g2c46\") pod \"kuadrant-operator-controller-manager-8684c9ff49-nqdrd\" (UID: \"b39ed866-3890-4a9d-82b5-148dbf8e95b0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nqdrd" Jul 30 00:26:00.739063 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:00.739020 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:26:00.810677 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:00.810651 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6c6x9\" (UniqueName: \"kubernetes.io/projected/a38e3d81-da42-4aca-b7cd-7e4a3f27d6c9-kube-api-access-6c6x9\") pod \"limitador-operator-controller-manager-859d7b78bd-cxzlt\" (UID: \"a38e3d81-da42-4aca-b7cd-7e4a3f27d6c9\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cxzlt" Jul 30 00:26:00.821962 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:00.821937 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6c6x9\" (UniqueName: \"kubernetes.io/projected/a38e3d81-da42-4aca-b7cd-7e4a3f27d6c9-kube-api-access-6c6x9\") pod \"limitador-operator-controller-manager-859d7b78bd-cxzlt\" (UID: \"a38e3d81-da42-4aca-b7cd-7e4a3f27d6c9\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cxzlt" Jul 30 00:26:00.830625 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:00.830593 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7px28" Jul 30 00:26:00.832565 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:00.832543 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l5gwr" Jul 30 00:26:00.832894 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:26:00.832870 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-l5gwr\" is forbidden: User \"system:node:ip-10-0-133-10.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-10.ec2.internal' and this object" podUID="18759d20-e435-4a8e-8e80-d1fe4b49c3ca" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l5gwr" Jul 30 00:26:00.834856 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:26:00.834831 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-7px28\" is forbidden: User \"system:node:ip-10-0-133-10.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-10.ec2.internal' and this object" podUID="700f1769-b978-4f9b-862e-22e8f20d4c86" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7px28" Jul 30 00:26:00.835413 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:00.835383 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:26:00.836829 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:26:00.836808 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-l5gwr\" is forbidden: User \"system:node:ip-10-0-133-10.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-10.ec2.internal' and this object" podUID="18759d20-e435-4a8e-8e80-d1fe4b49c3ca" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l5gwr" Jul 30 00:26:00.838621 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:26:00.838601 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-7px28\" is forbidden: User \"system:node:ip-10-0-133-10.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-10.ec2.internal' and this object" podUID="700f1769-b978-4f9b-862e-22e8f20d4c86" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7px28" Jul 30 00:26:00.855661 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:00.855636 2563 generic.go:356] "Generic (PLEG): container finished" podID="700f1769-b978-4f9b-862e-22e8f20d4c86" containerID="ef3617f735eef64e95e48a589a568ba9668c746194dd119ab15d95e1b69d4abd" exitCode=0 Jul 30 00:26:00.855778 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:00.855680 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7px28" Jul 30 00:26:00.855778 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:00.855686 2563 scope.go:117] "RemoveContainer" containerID="ef3617f735eef64e95e48a589a568ba9668c746194dd119ab15d95e1b69d4abd" Jul 30 00:26:00.857080 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:00.857059 2563 generic.go:356] "Generic (PLEG): container finished" podID="18759d20-e435-4a8e-8e80-d1fe4b49c3ca" containerID="322067ee07857bff8d4be5b1d7eb4c5bc0dbb1c834aa4893c04f47028a176179" exitCode=0 Jul 30 00:26:00.857168 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:00.857099 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l5gwr" Jul 30 00:26:00.858028 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:26:00.858001 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-l5gwr\" is forbidden: User \"system:node:ip-10-0-133-10.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-10.ec2.internal' and this object" podUID="18759d20-e435-4a8e-8e80-d1fe4b49c3ca" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l5gwr" Jul 30 00:26:00.859925 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:26:00.859904 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-7px28\" is forbidden: User \"system:node:ip-10-0-133-10.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-10.ec2.internal' and this object" podUID="700f1769-b978-4f9b-862e-22e8f20d4c86" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7px28" Jul 30 00:26:00.861887 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:26:00.861864 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-l5gwr\" is forbidden: User \"system:node:ip-10-0-133-10.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-10.ec2.internal' and this object" podUID="18759d20-e435-4a8e-8e80-d1fe4b49c3ca" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l5gwr" Jul 30 00:26:00.863692 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:26:00.863667 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-7px28\" is forbidden: User \"system:node:ip-10-0-133-10.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-10.ec2.internal' and this object" podUID="700f1769-b978-4f9b-862e-22e8f20d4c86" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7px28" Jul 30 00:26:00.865276 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:00.865261 2563 scope.go:117] "RemoveContainer" containerID="ef3617f735eef64e95e48a589a568ba9668c746194dd119ab15d95e1b69d4abd" Jul 30 00:26:00.865534 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:26:00.865517 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef3617f735eef64e95e48a589a568ba9668c746194dd119ab15d95e1b69d4abd\": container with ID starting with ef3617f735eef64e95e48a589a568ba9668c746194dd119ab15d95e1b69d4abd not found: ID does not exist" containerID="ef3617f735eef64e95e48a589a568ba9668c746194dd119ab15d95e1b69d4abd" Jul 30 00:26:00.865582 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:00.865548 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef3617f735eef64e95e48a589a568ba9668c746194dd119ab15d95e1b69d4abd"} err="failed to get container status \"ef3617f735eef64e95e48a589a568ba9668c746194dd119ab15d95e1b69d4abd\": rpc error: code = NotFound desc = could not find container \"ef3617f735eef64e95e48a589a568ba9668c746194dd119ab15d95e1b69d4abd\": container with ID starting with ef3617f735eef64e95e48a589a568ba9668c746194dd119ab15d95e1b69d4abd not found: ID does not exist" Jul 30 00:26:00.865582 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:00.865564 2563 scope.go:117] "RemoveContainer" containerID="322067ee07857bff8d4be5b1d7eb4c5bc0dbb1c834aa4893c04f47028a176179" Jul 30 00:26:00.872957 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:00.872926 2563 scope.go:117] "RemoveContainer" containerID="322067ee07857bff8d4be5b1d7eb4c5bc0dbb1c834aa4893c04f47028a176179" Jul 30 00:26:00.873196 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:26:00.873177 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"322067ee07857bff8d4be5b1d7eb4c5bc0dbb1c834aa4893c04f47028a176179\": container with ID starting with 322067ee07857bff8d4be5b1d7eb4c5bc0dbb1c834aa4893c04f47028a176179 not found: ID does not exist" containerID="322067ee07857bff8d4be5b1d7eb4c5bc0dbb1c834aa4893c04f47028a176179" Jul 30 00:26:00.873243 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:00.873201 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"322067ee07857bff8d4be5b1d7eb4c5bc0dbb1c834aa4893c04f47028a176179"} err="failed to get container status \"322067ee07857bff8d4be5b1d7eb4c5bc0dbb1c834aa4893c04f47028a176179\": rpc error: code = NotFound desc = could not find container \"322067ee07857bff8d4be5b1d7eb4c5bc0dbb1c834aa4893c04f47028a176179\": container with ID starting with 322067ee07857bff8d4be5b1d7eb4c5bc0dbb1c834aa4893c04f47028a176179 not found: ID does not exist" Jul 30 00:26:00.911719 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:00.911689 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/700f1769-b978-4f9b-862e-22e8f20d4c86-extensions-socket-volume\") pod \"700f1769-b978-4f9b-862e-22e8f20d4c86\" (UID: \"700f1769-b978-4f9b-862e-22e8f20d4c86\") " Jul 30 00:26:00.911883 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:00.911797 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6rhzs\" (UniqueName: \"kubernetes.io/projected/18759d20-e435-4a8e-8e80-d1fe4b49c3ca-kube-api-access-6rhzs\") pod \"18759d20-e435-4a8e-8e80-d1fe4b49c3ca\" (UID: \"18759d20-e435-4a8e-8e80-d1fe4b49c3ca\") " Jul 30 00:26:00.911883 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:00.911862 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mxk5n\" (UniqueName: \"kubernetes.io/projected/700f1769-b978-4f9b-862e-22e8f20d4c86-kube-api-access-mxk5n\") pod \"700f1769-b978-4f9b-862e-22e8f20d4c86\" (UID: \"700f1769-b978-4f9b-862e-22e8f20d4c86\") " Jul 30 00:26:00.912144 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:00.912119 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/700f1769-b978-4f9b-862e-22e8f20d4c86-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "700f1769-b978-4f9b-862e-22e8f20d4c86" (UID: "700f1769-b978-4f9b-862e-22e8f20d4c86"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:26:00.913807 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:00.913784 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18759d20-e435-4a8e-8e80-d1fe4b49c3ca-kube-api-access-6rhzs" (OuterVolumeSpecName: "kube-api-access-6rhzs") pod "18759d20-e435-4a8e-8e80-d1fe4b49c3ca" (UID: "18759d20-e435-4a8e-8e80-d1fe4b49c3ca"). InnerVolumeSpecName "kube-api-access-6rhzs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 00:26:00.913807 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:00.913791 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/700f1769-b978-4f9b-862e-22e8f20d4c86-kube-api-access-mxk5n" (OuterVolumeSpecName: "kube-api-access-mxk5n") pod "700f1769-b978-4f9b-862e-22e8f20d4c86" (UID: "700f1769-b978-4f9b-862e-22e8f20d4c86"). InnerVolumeSpecName "kube-api-access-mxk5n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 00:26:01.003503 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:01.003467 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nqdrd" Jul 30 00:26:01.008722 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:01.008698 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cxzlt" Jul 30 00:26:01.012678 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:01.012658 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6rhzs\" (UniqueName: \"kubernetes.io/projected/18759d20-e435-4a8e-8e80-d1fe4b49c3ca-kube-api-access-6rhzs\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:26:01.012771 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:01.012680 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mxk5n\" (UniqueName: \"kubernetes.io/projected/700f1769-b978-4f9b-862e-22e8f20d4c86-kube-api-access-mxk5n\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:26:01.012771 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:01.012691 2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/700f1769-b978-4f9b-862e-22e8f20d4c86-extensions-socket-volume\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:26:01.150470 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:01.150443 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nqdrd"] Jul 30 00:26:01.153389 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:26:01.153359 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb39ed866_3890_4a9d_82b5_148dbf8e95b0.slice/crio-cc73aa9b651b4d6775e23fb1ad5222fa22af1b3b9d20b2fbe40c0c25b954d116 WatchSource:0}: Error finding container cc73aa9b651b4d6775e23fb1ad5222fa22af1b3b9d20b2fbe40c0c25b954d116: Status 404 returned error can't find the container with id cc73aa9b651b4d6775e23fb1ad5222fa22af1b3b9d20b2fbe40c0c25b954d116 Jul 30 00:26:01.169541 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:01.169519 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cxzlt"] Jul 30 00:26:01.170061 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:26:01.170034 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-l5gwr\" is forbidden: User \"system:node:ip-10-0-133-10.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-10.ec2.internal' and this object" podUID="18759d20-e435-4a8e-8e80-d1fe4b49c3ca" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l5gwr" Jul 30 00:26:01.171961 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:26:01.171932 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-7px28\" is forbidden: User \"system:node:ip-10-0-133-10.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-10.ec2.internal' and this object" podUID="700f1769-b978-4f9b-862e-22e8f20d4c86" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7px28" Jul 30 00:26:01.172095 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:26:01.172057 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda38e3d81_da42_4aca_b7cd_7e4a3f27d6c9.slice/crio-d6260b0bba31157d2236cfa7f2ec58437ccc4354e3725ca0679ad6920d88c0a2 WatchSource:0}: Error finding container d6260b0bba31157d2236cfa7f2ec58437ccc4354e3725ca0679ad6920d88c0a2: Status 404 returned error can't find the container with id d6260b0bba31157d2236cfa7f2ec58437ccc4354e3725ca0679ad6920d88c0a2 Jul 30 00:26:01.173853 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:26:01.173826 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-l5gwr\" is forbidden: User \"system:node:ip-10-0-133-10.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-10.ec2.internal' and this object" podUID="18759d20-e435-4a8e-8e80-d1fe4b49c3ca" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l5gwr" Jul 30 00:26:01.175750 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:26:01.175701 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-7px28\" is forbidden: User \"system:node:ip-10-0-133-10.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-10.ec2.internal' and this object" podUID="700f1769-b978-4f9b-862e-22e8f20d4c86" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7px28" Jul 30 00:26:01.865674 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:01.865586 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cxzlt" event={"ID":"a38e3d81-da42-4aca-b7cd-7e4a3f27d6c9","Type":"ContainerStarted","Data":"81768c76e3bccb4c9c8e6d8c502bc8e3d73315f5a44d27575130717b7beae6ee"} Jul 30 00:26:01.865674 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:01.865636 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cxzlt" event={"ID":"a38e3d81-da42-4aca-b7cd-7e4a3f27d6c9","Type":"ContainerStarted","Data":"d6260b0bba31157d2236cfa7f2ec58437ccc4354e3725ca0679ad6920d88c0a2"} Jul 30 00:26:01.866134 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:01.865688 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cxzlt" Jul 30 00:26:01.867265 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:01.867240 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nqdrd" event={"ID":"b39ed866-3890-4a9d-82b5-148dbf8e95b0","Type":"ContainerStarted","Data":"9321a152d92a8a49e0d9d0fe658a1cdec9824d386ed9908aa2b29f8475d1d725"} Jul 30 00:26:01.867378 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:01.867284 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nqdrd" event={"ID":"b39ed866-3890-4a9d-82b5-148dbf8e95b0","Type":"ContainerStarted","Data":"cc73aa9b651b4d6775e23fb1ad5222fa22af1b3b9d20b2fbe40c0c25b954d116"} Jul 30 00:26:01.867378 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:01.867313 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nqdrd" Jul 30 00:26:01.883106 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:01.883068 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cxzlt" podStartSLOduration=1.883055477 podStartE2EDuration="1.883055477s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:26:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:26:01.881541994 +0000 UTC m=+525.940297966" watchObservedRunningTime="2026-07-30 00:26:01.883055477 +0000 UTC m=+525.941811515" Jul 30 00:26:01.883551 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:26:01.883529 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-l5gwr\" is forbidden: User \"system:node:ip-10-0-133-10.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-10.ec2.internal' and this object" podUID="18759d20-e435-4a8e-8e80-d1fe4b49c3ca" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l5gwr" Jul 30 00:26:01.885282 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:26:01.885257 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-7px28\" is forbidden: User \"system:node:ip-10-0-133-10.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-10.ec2.internal' and this object" podUID="700f1769-b978-4f9b-862e-22e8f20d4c86" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7px28" Jul 30 00:26:01.907044 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:26:01.907015 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-l5gwr\" is forbidden: User \"system:node:ip-10-0-133-10.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-10.ec2.internal' and this object" podUID="18759d20-e435-4a8e-8e80-d1fe4b49c3ca" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l5gwr" Jul 30 00:26:01.907555 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:01.907520 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nqdrd" podStartSLOduration=1.907507501 podStartE2EDuration="1.907507501s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:26:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:26:01.905136153 +0000 UTC m=+525.963892123" watchObservedRunningTime="2026-07-30 00:26:01.907507501 +0000 UTC m=+525.966263467" Jul 30 00:26:01.908795 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:26:01.908774 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-7px28\" is forbidden: User \"system:node:ip-10-0-133-10.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-10.ec2.internal' and this object" podUID="700f1769-b978-4f9b-862e-22e8f20d4c86" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7px28" Jul 30 00:26:02.481639 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:02.481605 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18759d20-e435-4a8e-8e80-d1fe4b49c3ca" path="/var/lib/kubelet/pods/18759d20-e435-4a8e-8e80-d1fe4b49c3ca/volumes" Jul 30 00:26:02.481992 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:02.481978 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="700f1769-b978-4f9b-862e-22e8f20d4c86" path="/var/lib/kubelet/pods/700f1769-b978-4f9b-862e-22e8f20d4c86/volumes" Jul 30 00:26:03.230510 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:03.230477 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:26:03.334859 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:03.334828 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:26:06.059983 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:06.059946 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:26:06.163099 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:06.163068 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:26:09.692495 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:09.692404 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:26:09.792976 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:09.792932 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:26:12.874523 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:12.874489 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nqdrd" Jul 30 00:26:12.874904 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:12.874565 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cxzlt" Jul 30 00:26:18.512676 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:18.512634 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:26:18.609920 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:18.609892 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:26:21.126373 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:21.126333 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:26:21.228260 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:21.228216 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:26:24.322520 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:24.322471 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:26:24.429129 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:24.429099 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:26:24.579618 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:24.579526 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:26:27.715332 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:27.715273 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:26:27.816996 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:27.816951 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:26:29.603375 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:29.603345 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nqdrd"] Jul 30 00:26:29.603763 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:29.603537 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nqdrd" podUID="b39ed866-3890-4a9d-82b5-148dbf8e95b0" containerName="manager" containerID="cri-o://9321a152d92a8a49e0d9d0fe658a1cdec9824d386ed9908aa2b29f8475d1d725" gracePeriod=10 Jul 30 00:26:29.861302 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:29.861240 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nqdrd" Jul 30 00:26:29.898990 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:29.898956 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tmqb4"] Jul 30 00:26:29.899346 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:29.899331 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b39ed866-3890-4a9d-82b5-148dbf8e95b0" containerName="manager" Jul 30 00:26:29.899346 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:29.899345 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b39ed866-3890-4a9d-82b5-148dbf8e95b0" containerName="manager" Jul 30 00:26:29.899485 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:29.899406 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b39ed866-3890-4a9d-82b5-148dbf8e95b0" containerName="manager" Jul 30 00:26:29.901561 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:29.901539 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tmqb4" Jul 30 00:26:29.915369 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:29.915340 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tmqb4"] Jul 30 00:26:29.962766 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:29.962716 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g2c46\" (UniqueName: \"kubernetes.io/projected/b39ed866-3890-4a9d-82b5-148dbf8e95b0-kube-api-access-g2c46\") pod \"b39ed866-3890-4a9d-82b5-148dbf8e95b0\" (UID: \"b39ed866-3890-4a9d-82b5-148dbf8e95b0\") " Jul 30 00:26:29.962931 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:29.962849 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b39ed866-3890-4a9d-82b5-148dbf8e95b0-extensions-socket-volume\") pod \"b39ed866-3890-4a9d-82b5-148dbf8e95b0\" (UID: \"b39ed866-3890-4a9d-82b5-148dbf8e95b0\") " Jul 30 00:26:29.963236 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:29.963200 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b39ed866-3890-4a9d-82b5-148dbf8e95b0-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b39ed866-3890-4a9d-82b5-148dbf8e95b0" (UID: "b39ed866-3890-4a9d-82b5-148dbf8e95b0"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:26:29.964725 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:29.964705 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b39ed866-3890-4a9d-82b5-148dbf8e95b0-kube-api-access-g2c46" (OuterVolumeSpecName: "kube-api-access-g2c46") pod "b39ed866-3890-4a9d-82b5-148dbf8e95b0" (UID: "b39ed866-3890-4a9d-82b5-148dbf8e95b0"). InnerVolumeSpecName "kube-api-access-g2c46". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 00:26:30.002837 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:30.002785 2563 generic.go:356] "Generic (PLEG): container finished" podID="b39ed866-3890-4a9d-82b5-148dbf8e95b0" containerID="9321a152d92a8a49e0d9d0fe658a1cdec9824d386ed9908aa2b29f8475d1d725" exitCode=0 Jul 30 00:26:30.002996 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:30.002841 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nqdrd" Jul 30 00:26:30.002996 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:30.002864 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nqdrd" event={"ID":"b39ed866-3890-4a9d-82b5-148dbf8e95b0","Type":"ContainerDied","Data":"9321a152d92a8a49e0d9d0fe658a1cdec9824d386ed9908aa2b29f8475d1d725"} Jul 30 00:26:30.002996 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:30.002921 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nqdrd" event={"ID":"b39ed866-3890-4a9d-82b5-148dbf8e95b0","Type":"ContainerDied","Data":"cc73aa9b651b4d6775e23fb1ad5222fa22af1b3b9d20b2fbe40c0c25b954d116"} Jul 30 00:26:30.002996 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:30.002953 2563 scope.go:117] "RemoveContainer" containerID="9321a152d92a8a49e0d9d0fe658a1cdec9824d386ed9908aa2b29f8475d1d725" Jul 30 00:26:30.012304 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:30.012276 2563 scope.go:117] "RemoveContainer" containerID="9321a152d92a8a49e0d9d0fe658a1cdec9824d386ed9908aa2b29f8475d1d725" Jul 30 00:26:30.012579 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:26:30.012559 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9321a152d92a8a49e0d9d0fe658a1cdec9824d386ed9908aa2b29f8475d1d725\": container with ID starting with 9321a152d92a8a49e0d9d0fe658a1cdec9824d386ed9908aa2b29f8475d1d725 not found: ID does not exist" containerID="9321a152d92a8a49e0d9d0fe658a1cdec9824d386ed9908aa2b29f8475d1d725" Jul 30 00:26:30.012632 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:30.012585 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9321a152d92a8a49e0d9d0fe658a1cdec9824d386ed9908aa2b29f8475d1d725"} err="failed to get container status \"9321a152d92a8a49e0d9d0fe658a1cdec9824d386ed9908aa2b29f8475d1d725\": rpc error: code = NotFound desc = could not find container \"9321a152d92a8a49e0d9d0fe658a1cdec9824d386ed9908aa2b29f8475d1d725\": container with ID starting with 9321a152d92a8a49e0d9d0fe658a1cdec9824d386ed9908aa2b29f8475d1d725 not found: ID does not exist" Jul 30 00:26:30.027352 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:30.027328 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nqdrd"] Jul 30 00:26:30.030599 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:30.030575 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nqdrd"] Jul 30 00:26:30.064315 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:30.064286 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/92d878fa-dd6e-4025-abc3-35f3cbf1d85b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-tmqb4\" (UID: \"92d878fa-dd6e-4025-abc3-35f3cbf1d85b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tmqb4" Jul 30 00:26:30.064423 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:30.064344 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-db4rq\" (UniqueName: \"kubernetes.io/projected/92d878fa-dd6e-4025-abc3-35f3cbf1d85b-kube-api-access-db4rq\") pod \"kuadrant-operator-controller-manager-86878c769f-tmqb4\" (UID: \"92d878fa-dd6e-4025-abc3-35f3cbf1d85b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tmqb4" Jul 30 00:26:30.064462 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:30.064428 2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b39ed866-3890-4a9d-82b5-148dbf8e95b0-extensions-socket-volume\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:26:30.064462 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:30.064447 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g2c46\" (UniqueName: \"kubernetes.io/projected/b39ed866-3890-4a9d-82b5-148dbf8e95b0-kube-api-access-g2c46\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:26:30.165320 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:30.165225 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/92d878fa-dd6e-4025-abc3-35f3cbf1d85b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-tmqb4\" (UID: \"92d878fa-dd6e-4025-abc3-35f3cbf1d85b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tmqb4" Jul 30 00:26:30.165320 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:30.165309 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-db4rq\" (UniqueName: \"kubernetes.io/projected/92d878fa-dd6e-4025-abc3-35f3cbf1d85b-kube-api-access-db4rq\") pod \"kuadrant-operator-controller-manager-86878c769f-tmqb4\" (UID: \"92d878fa-dd6e-4025-abc3-35f3cbf1d85b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tmqb4" Jul 30 00:26:30.165616 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:30.165594 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/92d878fa-dd6e-4025-abc3-35f3cbf1d85b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-tmqb4\" (UID: \"92d878fa-dd6e-4025-abc3-35f3cbf1d85b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tmqb4" Jul 30 00:26:30.173137 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:30.173111 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-db4rq\" (UniqueName: \"kubernetes.io/projected/92d878fa-dd6e-4025-abc3-35f3cbf1d85b-kube-api-access-db4rq\") pod \"kuadrant-operator-controller-manager-86878c769f-tmqb4\" (UID: \"92d878fa-dd6e-4025-abc3-35f3cbf1d85b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tmqb4" Jul 30 00:26:30.213001 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:30.212976 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tmqb4" Jul 30 00:26:30.342410 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:30.342385 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tmqb4"] Jul 30 00:26:30.344021 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:26:30.343986 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92d878fa_dd6e_4025_abc3_35f3cbf1d85b.slice/crio-945b656648e41ba7c752a566f8683f5607e22ef521992ff4262fca0952d1876f WatchSource:0}: Error finding container 945b656648e41ba7c752a566f8683f5607e22ef521992ff4262fca0952d1876f: Status 404 returned error can't find the container with id 945b656648e41ba7c752a566f8683f5607e22ef521992ff4262fca0952d1876f Jul 30 00:26:30.475976 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:30.475944 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b39ed866-3890-4a9d-82b5-148dbf8e95b0" path="/var/lib/kubelet/pods/b39ed866-3890-4a9d-82b5-148dbf8e95b0/volumes" Jul 30 00:26:31.009119 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:31.009081 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tmqb4" event={"ID":"92d878fa-dd6e-4025-abc3-35f3cbf1d85b","Type":"ContainerStarted","Data":"6fea561e469d1bbb559718ae6b2cdf0d2f293bcdccbe9542e801cb699ade03d5"} Jul 30 00:26:31.009545 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:31.009141 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tmqb4" event={"ID":"92d878fa-dd6e-4025-abc3-35f3cbf1d85b","Type":"ContainerStarted","Data":"945b656648e41ba7c752a566f8683f5607e22ef521992ff4262fca0952d1876f"} Jul 30 00:26:31.009545 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:31.009166 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tmqb4" Jul 30 00:26:31.029848 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:31.029789 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tmqb4" podStartSLOduration=2.029773902 podStartE2EDuration="2.029773902s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:26:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:26:31.028924459 +0000 UTC m=+555.087680427" watchObservedRunningTime="2026-07-30 00:26:31.029773902 +0000 UTC m=+555.088529866" Jul 30 00:26:32.558904 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:32.558868 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:26:36.310353 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:36.310318 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:26:36.410017 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:36.409983 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:26:42.018240 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:42.018202 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tmqb4" Jul 30 00:26:45.912773 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:45.912719 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gf4jt"] Jul 30 00:26:45.916133 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:45.916116 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gf4jt" Jul 30 00:26:45.918998 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:45.918975 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-vprfv\"" Jul 30 00:26:45.937792 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:45.937754 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gf4jt"] Jul 30 00:26:45.983029 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:45.982989 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:26:45.993196 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:45.993164 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70445eac-de72-40b2-ad0e-92667eb8856d-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-gf4jt\" (UID: \"70445eac-de72-40b2-ad0e-92667eb8856d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gf4jt" Jul 30 00:26:45.993333 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:45.993207 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70445eac-de72-40b2-ad0e-92667eb8856d-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-gf4jt\" (UID: \"70445eac-de72-40b2-ad0e-92667eb8856d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gf4jt" Jul 30 00:26:45.993333 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:45.993237 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70445eac-de72-40b2-ad0e-92667eb8856d-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-gf4jt\" (UID: \"70445eac-de72-40b2-ad0e-92667eb8856d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gf4jt" Jul 30 00:26:45.993333 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:45.993282 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70445eac-de72-40b2-ad0e-92667eb8856d-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-gf4jt\" (UID: \"70445eac-de72-40b2-ad0e-92667eb8856d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gf4jt" Jul 30 00:26:45.993518 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:45.993332 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70445eac-de72-40b2-ad0e-92667eb8856d-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-gf4jt\" (UID: \"70445eac-de72-40b2-ad0e-92667eb8856d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gf4jt" Jul 30 00:26:45.993518 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:45.993361 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70445eac-de72-40b2-ad0e-92667eb8856d-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-gf4jt\" (UID: \"70445eac-de72-40b2-ad0e-92667eb8856d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gf4jt" Jul 30 00:26:45.993518 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:45.993392 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70445eac-de72-40b2-ad0e-92667eb8856d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-gf4jt\" (UID: \"70445eac-de72-40b2-ad0e-92667eb8856d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gf4jt" Jul 30 00:26:45.993518 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:45.993410 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fnhs\" (UniqueName: \"kubernetes.io/projected/70445eac-de72-40b2-ad0e-92667eb8856d-kube-api-access-7fnhs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-gf4jt\" (UID: \"70445eac-de72-40b2-ad0e-92667eb8856d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gf4jt" Jul 30 00:26:45.993518 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:45.993434 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70445eac-de72-40b2-ad0e-92667eb8856d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-gf4jt\" (UID: \"70445eac-de72-40b2-ad0e-92667eb8856d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gf4jt" Jul 30 00:26:46.093923 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:46.093895 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70445eac-de72-40b2-ad0e-92667eb8856d-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-gf4jt\" (UID: \"70445eac-de72-40b2-ad0e-92667eb8856d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gf4jt" Jul 30 00:26:46.094092 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:46.093932 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70445eac-de72-40b2-ad0e-92667eb8856d-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-gf4jt\" (UID: \"70445eac-de72-40b2-ad0e-92667eb8856d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gf4jt" Jul 30 00:26:46.094092 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:46.093970 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70445eac-de72-40b2-ad0e-92667eb8856d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-gf4jt\" (UID: \"70445eac-de72-40b2-ad0e-92667eb8856d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gf4jt" Jul 30 00:26:46.094092 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:46.093998 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fnhs\" (UniqueName: \"kubernetes.io/projected/70445eac-de72-40b2-ad0e-92667eb8856d-kube-api-access-7fnhs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-gf4jt\" (UID: \"70445eac-de72-40b2-ad0e-92667eb8856d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gf4jt" Jul 30 00:26:46.094092 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:46.094038 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70445eac-de72-40b2-ad0e-92667eb8856d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-gf4jt\" (UID: \"70445eac-de72-40b2-ad0e-92667eb8856d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gf4jt" Jul 30 00:26:46.094308 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:46.094096 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70445eac-de72-40b2-ad0e-92667eb8856d-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-gf4jt\" (UID: \"70445eac-de72-40b2-ad0e-92667eb8856d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gf4jt" Jul 30 00:26:46.094308 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:46.094126 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70445eac-de72-40b2-ad0e-92667eb8856d-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-gf4jt\" (UID: \"70445eac-de72-40b2-ad0e-92667eb8856d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gf4jt" Jul 30 00:26:46.094426 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:46.094341 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70445eac-de72-40b2-ad0e-92667eb8856d-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-gf4jt\" (UID: \"70445eac-de72-40b2-ad0e-92667eb8856d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gf4jt" Jul 30 00:26:46.094426 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:46.094404 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70445eac-de72-40b2-ad0e-92667eb8856d-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-gf4jt\" (UID: \"70445eac-de72-40b2-ad0e-92667eb8856d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gf4jt" Jul 30 00:26:46.094529 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:46.094450 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70445eac-de72-40b2-ad0e-92667eb8856d-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-gf4jt\" (UID: \"70445eac-de72-40b2-ad0e-92667eb8856d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gf4jt" Jul 30 00:26:46.094581 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:46.094526 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70445eac-de72-40b2-ad0e-92667eb8856d-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-gf4jt\" (UID: \"70445eac-de72-40b2-ad0e-92667eb8856d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gf4jt" Jul 30 00:26:46.094708 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:46.094686 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70445eac-de72-40b2-ad0e-92667eb8856d-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-gf4jt\" (UID: \"70445eac-de72-40b2-ad0e-92667eb8856d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gf4jt" Jul 30 00:26:46.094801 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:46.094711 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70445eac-de72-40b2-ad0e-92667eb8856d-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-gf4jt\" (UID: \"70445eac-de72-40b2-ad0e-92667eb8856d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gf4jt" Jul 30 00:26:46.094859 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:46.094807 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70445eac-de72-40b2-ad0e-92667eb8856d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-gf4jt\" (UID: \"70445eac-de72-40b2-ad0e-92667eb8856d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gf4jt" Jul 30 00:26:46.096445 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:46.096421 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70445eac-de72-40b2-ad0e-92667eb8856d-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-gf4jt\" (UID: \"70445eac-de72-40b2-ad0e-92667eb8856d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gf4jt" Jul 30 00:26:46.096721 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:46.096700 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70445eac-de72-40b2-ad0e-92667eb8856d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-gf4jt\" (UID: \"70445eac-de72-40b2-ad0e-92667eb8856d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gf4jt" Jul 30 00:26:46.101754 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:46.101713 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70445eac-de72-40b2-ad0e-92667eb8856d-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-gf4jt\" (UID: \"70445eac-de72-40b2-ad0e-92667eb8856d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gf4jt" Jul 30 00:26:46.101893 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:46.101876 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fnhs\" (UniqueName: \"kubernetes.io/projected/70445eac-de72-40b2-ad0e-92667eb8856d-kube-api-access-7fnhs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-gf4jt\" (UID: \"70445eac-de72-40b2-ad0e-92667eb8856d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gf4jt" Jul 30 00:26:46.226679 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:46.226602 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gf4jt" Jul 30 00:26:46.249490 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:46.249455 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:26:46.362896 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:46.362870 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gf4jt"] Jul 30 00:26:46.364930 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:26:46.364903 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70445eac_de72_40b2_ad0e_92667eb8856d.slice/crio-c9ee1f2f138dd1e1bdade5ae9e588ae62d0ab573969ec112d8460be7c4724548 WatchSource:0}: Error finding container c9ee1f2f138dd1e1bdade5ae9e588ae62d0ab573969ec112d8460be7c4724548: Status 404 returned error can't find the container with id c9ee1f2f138dd1e1bdade5ae9e588ae62d0ab573969ec112d8460be7c4724548 Jul 30 00:26:46.366909 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:46.366872 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:26:46.366990 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:46.366946 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:26:46.366990 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:46.366978 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:26:47.095021 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:47.094988 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gf4jt" event={"ID":"70445eac-de72-40b2-ad0e-92667eb8856d","Type":"ContainerStarted","Data":"b192cb8ec311b25d511861d87c96d73144698fa10e79ebabdc0eb4a29400f9cb"} Jul 30 00:26:47.095390 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:47.095036 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gf4jt" event={"ID":"70445eac-de72-40b2-ad0e-92667eb8856d","Type":"ContainerStarted","Data":"c9ee1f2f138dd1e1bdade5ae9e588ae62d0ab573969ec112d8460be7c4724548"} Jul 30 00:26:47.118857 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:47.118797 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gf4jt" podStartSLOduration=2.118774373 podStartE2EDuration="2.118774373s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:26:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:26:47.117323722 +0000 UTC m=+571.176079706" watchObservedRunningTime="2026-07-30 00:26:47.118774373 +0000 UTC m=+571.177530344" Jul 30 00:26:47.227333 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:47.227299 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gf4jt" Jul 30 00:26:47.235543 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:47.235514 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gf4jt" Jul 30 00:26:48.099929 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:48.099892 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gf4jt" Jul 30 00:26:48.100992 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:48.100969 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gf4jt" Jul 30 00:26:48.239161 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:48.239114 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:26:48.348942 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:48.348902 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:26:48.407109 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:48.407017 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:26:49.063617 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:49.063579 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:26:49.128952 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:49.128916 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:26:49.211382 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:49.211347 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-zk9zx"] Jul 30 00:26:49.215083 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:49.215064 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-zk9zx" Jul 30 00:26:49.217576 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:49.217545 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-cjb69\"" Jul 30 00:26:49.219422 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:49.219393 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-zk9zx"] Jul 30 00:26:49.219595 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:49.219574 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rrlg\" (UniqueName: \"kubernetes.io/projected/5b184273-dd22-4a59-a680-9bddcaf7ea6a-kube-api-access-8rrlg\") pod \"authorino-75d556bd77-zk9zx\" (UID: \"5b184273-dd22-4a59-a680-9bddcaf7ea6a\") " pod="kuadrant-system/authorino-75d556bd77-zk9zx" Jul 30 00:26:49.320691 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:49.320618 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rrlg\" (UniqueName: \"kubernetes.io/projected/5b184273-dd22-4a59-a680-9bddcaf7ea6a-kube-api-access-8rrlg\") pod \"authorino-75d556bd77-zk9zx\" (UID: \"5b184273-dd22-4a59-a680-9bddcaf7ea6a\") " pod="kuadrant-system/authorino-75d556bd77-zk9zx" Jul 30 00:26:49.328623 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:49.328594 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rrlg\" (UniqueName: \"kubernetes.io/projected/5b184273-dd22-4a59-a680-9bddcaf7ea6a-kube-api-access-8rrlg\") pod \"authorino-75d556bd77-zk9zx\" (UID: \"5b184273-dd22-4a59-a680-9bddcaf7ea6a\") " pod="kuadrant-system/authorino-75d556bd77-zk9zx" Jul 30 00:26:49.526680 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:49.526649 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-zk9zx" Jul 30 00:26:49.659205 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:49.659180 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-zk9zx"] Jul 30 00:26:49.661671 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:26:49.661636 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5b184273_dd22_4a59_a680_9bddcaf7ea6a.slice/crio-007a671fa88e9ba8795bac07e5da1d729799563afa8653bb1c16c8a946d3d3e6 WatchSource:0}: Error finding container 007a671fa88e9ba8795bac07e5da1d729799563afa8653bb1c16c8a946d3d3e6: Status 404 returned error can't find the container with id 007a671fa88e9ba8795bac07e5da1d729799563afa8653bb1c16c8a946d3d3e6 Jul 30 00:26:50.110589 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:50.110542 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-zk9zx" event={"ID":"5b184273-dd22-4a59-a680-9bddcaf7ea6a","Type":"ContainerStarted","Data":"007a671fa88e9ba8795bac07e5da1d729799563afa8653bb1c16c8a946d3d3e6"} Jul 30 00:26:53.128110 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:53.128072 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-zk9zx" event={"ID":"5b184273-dd22-4a59-a680-9bddcaf7ea6a","Type":"ContainerStarted","Data":"ead444e469a90720db813d3fad43b3b1c9a2f7112ad416c125195d48936d7253"} Jul 30 00:26:53.143363 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:26:53.143312 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-zk9zx" podStartSLOduration=1.186051961 podStartE2EDuration="4.14329628s" totalImagesPullingTime="2.957244319s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:26:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:26:53.141496432 +0000 UTC m=+577.200252401" watchObservedRunningTime="2026-07-30 00:26:53.14329628 +0000 UTC m=+577.202052248" Jul 30 00:27:04.108378 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:04.108340 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:27:04.212012 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:04.211965 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:27:04.523849 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:04.523806 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:27:16.371620 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:16.371594 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-b92tw_0ef8f120-f712-4ed9-adbd-40f785d4aa9d/console-operator/1.log" Jul 30 00:27:16.378721 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:16.378696 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-b92tw_0ef8f120-f712-4ed9-adbd-40f785d4aa9d/console-operator/1.log" Jul 30 00:27:17.149120 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:17.149079 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:27:17.278791 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:17.278754 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:27:18.933052 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:18.933015 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-zk9zx"] Jul 30 00:27:18.934475 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:18.933257 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-zk9zx" podUID="5b184273-dd22-4a59-a680-9bddcaf7ea6a" containerName="authorino" containerID="cri-o://ead444e469a90720db813d3fad43b3b1c9a2f7112ad416c125195d48936d7253" gracePeriod=30 Jul 30 00:27:19.184675 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:19.184615 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-zk9zx" Jul 30 00:27:19.252091 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:19.252051 2563 generic.go:356] "Generic (PLEG): container finished" podID="5b184273-dd22-4a59-a680-9bddcaf7ea6a" containerID="ead444e469a90720db813d3fad43b3b1c9a2f7112ad416c125195d48936d7253" exitCode=0 Jul 30 00:27:19.252233 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:19.252085 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-zk9zx" event={"ID":"5b184273-dd22-4a59-a680-9bddcaf7ea6a","Type":"ContainerDied","Data":"ead444e469a90720db813d3fad43b3b1c9a2f7112ad416c125195d48936d7253"} Jul 30 00:27:19.252233 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:19.252129 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-zk9zx" event={"ID":"5b184273-dd22-4a59-a680-9bddcaf7ea6a","Type":"ContainerDied","Data":"007a671fa88e9ba8795bac07e5da1d729799563afa8653bb1c16c8a946d3d3e6"} Jul 30 00:27:19.252233 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:19.252147 2563 scope.go:117] "RemoveContainer" containerID="ead444e469a90720db813d3fad43b3b1c9a2f7112ad416c125195d48936d7253" Jul 30 00:27:19.252233 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:19.252097 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-zk9zx" Jul 30 00:27:19.261586 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:19.261569 2563 scope.go:117] "RemoveContainer" containerID="ead444e469a90720db813d3fad43b3b1c9a2f7112ad416c125195d48936d7253" Jul 30 00:27:19.261856 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:27:19.261834 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ead444e469a90720db813d3fad43b3b1c9a2f7112ad416c125195d48936d7253\": container with ID starting with ead444e469a90720db813d3fad43b3b1c9a2f7112ad416c125195d48936d7253 not found: ID does not exist" containerID="ead444e469a90720db813d3fad43b3b1c9a2f7112ad416c125195d48936d7253" Jul 30 00:27:19.261959 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:19.261863 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ead444e469a90720db813d3fad43b3b1c9a2f7112ad416c125195d48936d7253"} err="failed to get container status \"ead444e469a90720db813d3fad43b3b1c9a2f7112ad416c125195d48936d7253\": rpc error: code = NotFound desc = could not find container \"ead444e469a90720db813d3fad43b3b1c9a2f7112ad416c125195d48936d7253\": container with ID starting with ead444e469a90720db813d3fad43b3b1c9a2f7112ad416c125195d48936d7253 not found: ID does not exist" Jul 30 00:27:19.277395 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:19.277372 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8rrlg\" (UniqueName: \"kubernetes.io/projected/5b184273-dd22-4a59-a680-9bddcaf7ea6a-kube-api-access-8rrlg\") pod \"5b184273-dd22-4a59-a680-9bddcaf7ea6a\" (UID: \"5b184273-dd22-4a59-a680-9bddcaf7ea6a\") " Jul 30 00:27:19.279259 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:19.279232 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b184273-dd22-4a59-a680-9bddcaf7ea6a-kube-api-access-8rrlg" (OuterVolumeSpecName: "kube-api-access-8rrlg") pod "5b184273-dd22-4a59-a680-9bddcaf7ea6a" (UID: "5b184273-dd22-4a59-a680-9bddcaf7ea6a"). InnerVolumeSpecName "kube-api-access-8rrlg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 00:27:19.378931 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:19.378894 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8rrlg\" (UniqueName: \"kubernetes.io/projected/5b184273-dd22-4a59-a680-9bddcaf7ea6a-kube-api-access-8rrlg\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:27:19.578108 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:19.578072 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-zk9zx"] Jul 30 00:27:19.586982 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:19.586954 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-zk9zx"] Jul 30 00:27:19.703101 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:19.703067 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:27:20.475668 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:20.475635 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b184273-dd22-4a59-a680-9bddcaf7ea6a" path="/var/lib/kubelet/pods/5b184273-dd22-4a59-a680-9bddcaf7ea6a/volumes" Jul 30 00:27:20.857116 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:20.857080 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:27:22.638385 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:22.638353 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:27:23.325205 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:23.325177 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:27:23.404901 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:23.404866 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:27:23.461682 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:23.461640 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:27:23.495107 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:23.495076 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-bd495bbc5-8697p"] Jul 30 00:27:23.495450 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:23.495438 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b184273-dd22-4a59-a680-9bddcaf7ea6a" containerName="authorino" Jul 30 00:27:23.495493 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:23.495452 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b184273-dd22-4a59-a680-9bddcaf7ea6a" containerName="authorino" Jul 30 00:27:23.495541 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:23.495511 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b184273-dd22-4a59-a680-9bddcaf7ea6a" containerName="authorino" Jul 30 00:27:23.498296 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:23.498278 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-bd495bbc5-8697p" Jul 30 00:27:23.500711 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:23.500684 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 30 00:27:23.500711 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:23.500707 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 30 00:27:23.500917 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:23.500709 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-94z2g\"" Jul 30 00:27:23.507497 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:23.507455 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-bd495bbc5-8697p"] Jul 30 00:27:23.549624 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:23.549601 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:27:23.618924 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:23.618865 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1cf550e0-db96-46a1-b7f5-56be853a5ba4-webhook-cert\") pod \"maas-controller-bd495bbc5-8697p\" (UID: \"1cf550e0-db96-46a1-b7f5-56be853a5ba4\") " pod="opendatahub/maas-controller-bd495bbc5-8697p" Jul 30 00:27:23.618924 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:23.618894 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hlnb\" (UniqueName: \"kubernetes.io/projected/1cf550e0-db96-46a1-b7f5-56be853a5ba4-kube-api-access-8hlnb\") pod \"maas-controller-bd495bbc5-8697p\" (UID: \"1cf550e0-db96-46a1-b7f5-56be853a5ba4\") " pod="opendatahub/maas-controller-bd495bbc5-8697p" Jul 30 00:27:23.719608 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:23.719577 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1cf550e0-db96-46a1-b7f5-56be853a5ba4-webhook-cert\") pod \"maas-controller-bd495bbc5-8697p\" (UID: \"1cf550e0-db96-46a1-b7f5-56be853a5ba4\") " pod="opendatahub/maas-controller-bd495bbc5-8697p" Jul 30 00:27:23.719608 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:23.719610 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hlnb\" (UniqueName: \"kubernetes.io/projected/1cf550e0-db96-46a1-b7f5-56be853a5ba4-kube-api-access-8hlnb\") pod \"maas-controller-bd495bbc5-8697p\" (UID: \"1cf550e0-db96-46a1-b7f5-56be853a5ba4\") " pod="opendatahub/maas-controller-bd495bbc5-8697p" Jul 30 00:27:23.721974 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:23.721951 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1cf550e0-db96-46a1-b7f5-56be853a5ba4-webhook-cert\") pod \"maas-controller-bd495bbc5-8697p\" (UID: \"1cf550e0-db96-46a1-b7f5-56be853a5ba4\") " pod="opendatahub/maas-controller-bd495bbc5-8697p" Jul 30 00:27:23.727072 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:23.727051 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hlnb\" (UniqueName: \"kubernetes.io/projected/1cf550e0-db96-46a1-b7f5-56be853a5ba4-kube-api-access-8hlnb\") pod \"maas-controller-bd495bbc5-8697p\" (UID: \"1cf550e0-db96-46a1-b7f5-56be853a5ba4\") " pod="opendatahub/maas-controller-bd495bbc5-8697p" Jul 30 00:27:23.809838 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:23.809809 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-bd495bbc5-8697p" Jul 30 00:27:23.942964 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:23.942915 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-bd495bbc5-8697p"] Jul 30 00:27:23.951967 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:27:23.951934 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cf550e0_db96_46a1_b7f5_56be853a5ba4.slice/crio-b20662c52436d35069780791d4fb4d98a98a6da71f1c84acf23c2646e55bca66 WatchSource:0}: Error finding container b20662c52436d35069780791d4fb4d98a98a6da71f1c84acf23c2646e55bca66: Status 404 returned error can't find the container with id b20662c52436d35069780791d4fb4d98a98a6da71f1c84acf23c2646e55bca66 Jul 30 00:27:24.281043 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:24.281005 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-bd495bbc5-8697p" event={"ID":"1cf550e0-db96-46a1-b7f5-56be853a5ba4","Type":"ContainerStarted","Data":"b20662c52436d35069780791d4fb4d98a98a6da71f1c84acf23c2646e55bca66"} Jul 30 00:27:26.133328 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:26.133281 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:27:26.227278 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:26.227236 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:27:26.292779 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:26.292728 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-bd495bbc5-8697p" event={"ID":"1cf550e0-db96-46a1-b7f5-56be853a5ba4","Type":"ContainerStarted","Data":"ae8b1c192e49cc40f819c79ccf130316275473ef0d0503604d881bbe0992dcc6"} Jul 30 00:27:26.292937 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:26.292846 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-bd495bbc5-8697p" Jul 30 00:27:26.315453 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:26.315401 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-bd495bbc5-8697p" podStartSLOduration=1.13795027 podStartE2EDuration="3.315383195s" totalImagesPullingTime="2.177432925s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:27:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:27:26.313846702 +0000 UTC m=+610.372602670" watchObservedRunningTime="2026-07-30 00:27:26.315383195 +0000 UTC m=+610.374139162" Jul 30 00:27:29.333498 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:29.333457 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:27:29.434984 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:29.434951 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:27:37.300879 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:37.300844 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-bd495bbc5-8697p" Jul 30 00:27:56.870123 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:56.870081 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:27:56.942872 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:56.942829 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:27:58.273453 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:58.273415 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:27:58.550593 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:58.550501 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:27:58.646998 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:58.646964 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:27:59.020317 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:59.020283 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:27:59.095153 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:27:59.095116 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:28:07.495555 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:07.495516 2563 generic.go:356] "Generic (PLEG): container finished" podID="70445eac-de72-40b2-ad0e-92667eb8856d" containerID="b192cb8ec311b25d511861d87c96d73144698fa10e79ebabdc0eb4a29400f9cb" exitCode=0 Jul 30 00:28:07.496002 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:07.495590 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gf4jt" event={"ID":"70445eac-de72-40b2-ad0e-92667eb8856d","Type":"ContainerDied","Data":"b192cb8ec311b25d511861d87c96d73144698fa10e79ebabdc0eb4a29400f9cb"} Jul 30 00:28:07.496002 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:07.495961 2563 scope.go:117] "RemoveContainer" containerID="b192cb8ec311b25d511861d87c96d73144698fa10e79ebabdc0eb4a29400f9cb" Jul 30 00:28:07.496882 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:07.496852 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:28:07.503171 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:07.497437 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:28:07.503171 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:07.497615 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:28:08.501649 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:08.501614 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gf4jt" event={"ID":"70445eac-de72-40b2-ad0e-92667eb8856d","Type":"ContainerStarted","Data":"decf093556f515b39cc74772420d7f713601b37bfb89b79cf62bccf78f019b6b"} Jul 30 00:28:09.227433 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:09.227393 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gf4jt" Jul 30 00:28:09.227657 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:09.227632 2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-gf4jt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 30 00:28:09.227709 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:09.227682 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gf4jt" podUID="70445eac-de72-40b2-ad0e-92667eb8856d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 30 00:28:10.227693 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:10.227657 2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-gf4jt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 30 00:28:10.228109 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:10.227709 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gf4jt" podUID="70445eac-de72-40b2-ad0e-92667eb8856d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 30 00:28:11.019727 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:11.019674 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:28:11.185773 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:11.185721 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:28:11.227253 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:11.227226 2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-gf4jt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 30 00:28:11.227370 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:11.227275 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gf4jt" podUID="70445eac-de72-40b2-ad0e-92667eb8856d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 30 00:28:12.227215 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:12.227178 2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-gf4jt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 30 00:28:12.227585 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:12.227236 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gf4jt" podUID="70445eac-de72-40b2-ad0e-92667eb8856d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 30 00:28:13.231048 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:13.231018 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gf4jt" Jul 30 00:28:13.231467 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:13.231348 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gf4jt" Jul 30 00:28:13.232104 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:13.232082 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gf4jt" Jul 30 00:28:24.213303 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:24.213268 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ch4fl"] Jul 30 00:28:24.217730 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:24.217705 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ch4fl" Jul 30 00:28:24.221285 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:24.221256 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-fwqxf\"" Jul 30 00:28:24.221421 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:24.221335 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 30 00:28:24.221421 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:24.221344 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 30 00:28:24.221421 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:24.221336 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 30 00:28:24.226206 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:24.226184 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ch4fl"] Jul 30 00:28:24.241462 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:24.241440 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:28:24.333296 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:24.333262 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvzv9\" (UniqueName: \"kubernetes.io/projected/3391cc48-9da3-4104-9a82-3b2d70e24a30-kube-api-access-vvzv9\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ch4fl\" (UID: \"3391cc48-9da3-4104-9a82-3b2d70e24a30\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ch4fl" Jul 30 00:28:24.333492 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:24.333312 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3391cc48-9da3-4104-9a82-3b2d70e24a30-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ch4fl\" (UID: \"3391cc48-9da3-4104-9a82-3b2d70e24a30\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ch4fl" Jul 30 00:28:24.333492 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:24.333346 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3391cc48-9da3-4104-9a82-3b2d70e24a30-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ch4fl\" (UID: \"3391cc48-9da3-4104-9a82-3b2d70e24a30\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ch4fl" Jul 30 00:28:24.333492 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:24.333467 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3391cc48-9da3-4104-9a82-3b2d70e24a30-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ch4fl\" (UID: \"3391cc48-9da3-4104-9a82-3b2d70e24a30\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ch4fl" Jul 30 00:28:24.333646 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:24.333521 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3391cc48-9da3-4104-9a82-3b2d70e24a30-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ch4fl\" (UID: \"3391cc48-9da3-4104-9a82-3b2d70e24a30\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ch4fl" Jul 30 00:28:24.333646 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:24.333580 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3391cc48-9da3-4104-9a82-3b2d70e24a30-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ch4fl\" (UID: \"3391cc48-9da3-4104-9a82-3b2d70e24a30\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ch4fl" Jul 30 00:28:24.434752 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:24.434704 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3391cc48-9da3-4104-9a82-3b2d70e24a30-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ch4fl\" (UID: \"3391cc48-9da3-4104-9a82-3b2d70e24a30\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ch4fl" Jul 30 00:28:24.434942 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:24.434768 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3391cc48-9da3-4104-9a82-3b2d70e24a30-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ch4fl\" (UID: \"3391cc48-9da3-4104-9a82-3b2d70e24a30\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ch4fl" Jul 30 00:28:24.434942 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:24.434806 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3391cc48-9da3-4104-9a82-3b2d70e24a30-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ch4fl\" (UID: \"3391cc48-9da3-4104-9a82-3b2d70e24a30\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ch4fl" Jul 30 00:28:24.434942 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:24.434833 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvzv9\" (UniqueName: \"kubernetes.io/projected/3391cc48-9da3-4104-9a82-3b2d70e24a30-kube-api-access-vvzv9\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ch4fl\" (UID: \"3391cc48-9da3-4104-9a82-3b2d70e24a30\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ch4fl" Jul 30 00:28:24.434942 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:24.434864 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3391cc48-9da3-4104-9a82-3b2d70e24a30-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ch4fl\" (UID: \"3391cc48-9da3-4104-9a82-3b2d70e24a30\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ch4fl" Jul 30 00:28:24.434942 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:24.434888 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3391cc48-9da3-4104-9a82-3b2d70e24a30-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ch4fl\" (UID: \"3391cc48-9da3-4104-9a82-3b2d70e24a30\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ch4fl" Jul 30 00:28:24.435309 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:24.435288 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3391cc48-9da3-4104-9a82-3b2d70e24a30-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ch4fl\" (UID: \"3391cc48-9da3-4104-9a82-3b2d70e24a30\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ch4fl" Jul 30 00:28:24.435309 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:24.435298 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3391cc48-9da3-4104-9a82-3b2d70e24a30-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ch4fl\" (UID: \"3391cc48-9da3-4104-9a82-3b2d70e24a30\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ch4fl" Jul 30 00:28:24.435418 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:24.435329 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3391cc48-9da3-4104-9a82-3b2d70e24a30-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ch4fl\" (UID: \"3391cc48-9da3-4104-9a82-3b2d70e24a30\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ch4fl" Jul 30 00:28:24.437220 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:24.437196 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3391cc48-9da3-4104-9a82-3b2d70e24a30-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ch4fl\" (UID: \"3391cc48-9da3-4104-9a82-3b2d70e24a30\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ch4fl" Jul 30 00:28:24.437373 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:24.437269 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3391cc48-9da3-4104-9a82-3b2d70e24a30-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ch4fl\" (UID: \"3391cc48-9da3-4104-9a82-3b2d70e24a30\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ch4fl" Jul 30 00:28:24.443051 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:24.443019 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvzv9\" (UniqueName: \"kubernetes.io/projected/3391cc48-9da3-4104-9a82-3b2d70e24a30-kube-api-access-vvzv9\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ch4fl\" (UID: \"3391cc48-9da3-4104-9a82-3b2d70e24a30\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ch4fl" Jul 30 00:28:24.530503 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:24.530470 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ch4fl" Jul 30 00:28:24.884986 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:24.884865 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ch4fl"] Jul 30 00:28:24.890323 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:28:24.890293 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3391cc48_9da3_4104_9a82_3b2d70e24a30.slice/crio-2cff81ae72b24b5edae51f2d1c59e697b43d81598d44188a8545a0780f216a3c WatchSource:0}: Error finding container 2cff81ae72b24b5edae51f2d1c59e697b43d81598d44188a8545a0780f216a3c: Status 404 returned error can't find the container with id 2cff81ae72b24b5edae51f2d1c59e697b43d81598d44188a8545a0780f216a3c Jul 30 00:28:25.584312 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:25.584276 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ch4fl" event={"ID":"3391cc48-9da3-4104-9a82-3b2d70e24a30","Type":"ContainerStarted","Data":"2cff81ae72b24b5edae51f2d1c59e697b43d81598d44188a8545a0780f216a3c"} Jul 30 00:28:26.590835 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:26.590792 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ch4fl" event={"ID":"3391cc48-9da3-4104-9a82-3b2d70e24a30","Type":"ContainerStarted","Data":"0838c3874e3237af8941391b8135d38db6da3616f48488ecb66460aa582b3199"} Jul 30 00:28:26.607320 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:26.607269 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ch4fl" podStartSLOduration=1.499431602 podStartE2EDuration="2.607254591s" totalImagesPullingTime="1.107822989s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:28:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:28:26.605478276 +0000 UTC m=+670.664234244" watchObservedRunningTime="2026-07-30 00:28:26.607254591 +0000 UTC m=+670.666010558" Jul 30 00:28:33.516038 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:33.516000 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-xhzsb"] Jul 30 00:28:33.518721 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:33.518700 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xhzsb" Jul 30 00:28:33.521366 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:33.521340 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 30 00:28:33.530645 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:33.530620 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-xhzsb"] Jul 30 00:28:33.615670 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:33.615637 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/690de04c-56c5-4cf6-a383-5de5291d1161-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xhzsb\" (UID: \"690de04c-56c5-4cf6-a383-5de5291d1161\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xhzsb" Jul 30 00:28:33.615670 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:33.615657 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:28:33.615958 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:33.615701 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4s9n\" (UniqueName: \"kubernetes.io/projected/690de04c-56c5-4cf6-a383-5de5291d1161-kube-api-access-r4s9n\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xhzsb\" (UID: \"690de04c-56c5-4cf6-a383-5de5291d1161\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xhzsb" Jul 30 00:28:33.615958 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:33.615784 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/690de04c-56c5-4cf6-a383-5de5291d1161-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xhzsb\" (UID: \"690de04c-56c5-4cf6-a383-5de5291d1161\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xhzsb" Jul 30 00:28:33.615958 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:33.615825 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/690de04c-56c5-4cf6-a383-5de5291d1161-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xhzsb\" (UID: \"690de04c-56c5-4cf6-a383-5de5291d1161\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xhzsb" Jul 30 00:28:33.615958 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:33.615867 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/690de04c-56c5-4cf6-a383-5de5291d1161-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xhzsb\" (UID: \"690de04c-56c5-4cf6-a383-5de5291d1161\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xhzsb" Jul 30 00:28:33.616165 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:33.615961 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/690de04c-56c5-4cf6-a383-5de5291d1161-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xhzsb\" (UID: \"690de04c-56c5-4cf6-a383-5de5291d1161\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xhzsb" Jul 30 00:28:33.717456 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:33.717413 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/690de04c-56c5-4cf6-a383-5de5291d1161-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xhzsb\" (UID: \"690de04c-56c5-4cf6-a383-5de5291d1161\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xhzsb" Jul 30 00:28:33.717642 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:33.717469 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/690de04c-56c5-4cf6-a383-5de5291d1161-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xhzsb\" (UID: \"690de04c-56c5-4cf6-a383-5de5291d1161\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xhzsb" Jul 30 00:28:33.717642 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:33.717513 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/690de04c-56c5-4cf6-a383-5de5291d1161-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xhzsb\" (UID: \"690de04c-56c5-4cf6-a383-5de5291d1161\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xhzsb" Jul 30 00:28:33.717642 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:33.717567 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/690de04c-56c5-4cf6-a383-5de5291d1161-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xhzsb\" (UID: \"690de04c-56c5-4cf6-a383-5de5291d1161\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xhzsb" Jul 30 00:28:33.717642 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:33.717630 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/690de04c-56c5-4cf6-a383-5de5291d1161-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xhzsb\" (UID: \"690de04c-56c5-4cf6-a383-5de5291d1161\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xhzsb" Jul 30 00:28:33.717900 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:33.717658 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4s9n\" (UniqueName: \"kubernetes.io/projected/690de04c-56c5-4cf6-a383-5de5291d1161-kube-api-access-r4s9n\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xhzsb\" (UID: \"690de04c-56c5-4cf6-a383-5de5291d1161\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xhzsb" Jul 30 00:28:33.717900 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:33.717851 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/690de04c-56c5-4cf6-a383-5de5291d1161-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xhzsb\" (UID: \"690de04c-56c5-4cf6-a383-5de5291d1161\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xhzsb" Jul 30 00:28:33.718003 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:33.717902 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/690de04c-56c5-4cf6-a383-5de5291d1161-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xhzsb\" (UID: \"690de04c-56c5-4cf6-a383-5de5291d1161\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xhzsb" Jul 30 00:28:33.718051 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:33.718013 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/690de04c-56c5-4cf6-a383-5de5291d1161-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xhzsb\" (UID: \"690de04c-56c5-4cf6-a383-5de5291d1161\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xhzsb" Jul 30 00:28:33.719962 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:33.719938 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/690de04c-56c5-4cf6-a383-5de5291d1161-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xhzsb\" (UID: \"690de04c-56c5-4cf6-a383-5de5291d1161\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xhzsb" Jul 30 00:28:33.720186 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:33.720169 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/690de04c-56c5-4cf6-a383-5de5291d1161-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xhzsb\" (UID: \"690de04c-56c5-4cf6-a383-5de5291d1161\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xhzsb" Jul 30 00:28:33.725527 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:33.725503 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4s9n\" (UniqueName: \"kubernetes.io/projected/690de04c-56c5-4cf6-a383-5de5291d1161-kube-api-access-r4s9n\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xhzsb\" (UID: \"690de04c-56c5-4cf6-a383-5de5291d1161\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xhzsb" Jul 30 00:28:33.828962 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:33.828876 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xhzsb" Jul 30 00:28:33.960252 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:33.960223 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-xhzsb"] Jul 30 00:28:33.962426 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:28:33.962399 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod690de04c_56c5_4cf6_a383_5de5291d1161.slice/crio-abab9a13b5dc4ef9b13879ac5fe42088186aa93efba6bbdac1dbd670efe6a010 WatchSource:0}: Error finding container abab9a13b5dc4ef9b13879ac5fe42088186aa93efba6bbdac1dbd670efe6a010: Status 404 returned error can't find the container with id abab9a13b5dc4ef9b13879ac5fe42088186aa93efba6bbdac1dbd670efe6a010 Jul 30 00:28:34.531547 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:34.531517 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ch4fl" Jul 30 00:28:34.531944 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:34.531568 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ch4fl" Jul 30 00:28:34.543861 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:34.543837 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ch4fl" Jul 30 00:28:34.631931 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:34.631898 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xhzsb" event={"ID":"690de04c-56c5-4cf6-a383-5de5291d1161","Type":"ContainerStarted","Data":"d31c75b44dea8679869e184b7673be7a6983b7ce9e37d1d63557968835450846"} Jul 30 00:28:34.632101 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:34.631953 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xhzsb" event={"ID":"690de04c-56c5-4cf6-a383-5de5291d1161","Type":"ContainerStarted","Data":"abab9a13b5dc4ef9b13879ac5fe42088186aa93efba6bbdac1dbd670efe6a010"} Jul 30 00:28:34.643104 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:34.643079 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ch4fl" Jul 30 00:28:34.653781 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:34.653713 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xhzsb" podStartSLOduration=1.492697701 podStartE2EDuration="1.653698639s" totalImagesPullingTime="161.000938ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:28:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:28:34.651543612 +0000 UTC m=+678.710299580" watchObservedRunningTime="2026-07-30 00:28:34.653698639 +0000 UTC m=+678.712454606" Jul 30 00:28:43.829853 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:43.829817 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xhzsb" Jul 30 00:28:43.830279 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:43.829875 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xhzsb" Jul 30 00:28:43.842155 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:43.842124 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xhzsb" Jul 30 00:28:43.924962 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:43.924927 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:28:44.693193 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:44.693159 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xhzsb" Jul 30 00:28:49.035648 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:49.035609 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:28:54.207020 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:54.206975 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:28:57.244271 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:28:57.244234 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:29:10.914800 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:29:10.914704 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:30:00.145964 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:30:00.145930 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29756190-nchqp"] Jul 30 00:30:00.149350 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:30:00.149333 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756190-nchqp" Jul 30 00:30:00.151836 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:30:00.151816 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-v8q7j\"" Jul 30 00:30:00.157527 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:30:00.157504 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29756190-nchqp"] Jul 30 00:30:00.161016 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:30:00.160996 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756190-nchqp" Jul 30 00:30:00.502381 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:30:00.502325 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29756190-nchqp"] Jul 30 00:30:00.504809 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:30:00.504778 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2447b8f_3d7d_48f2_abaf_fd773fd8ecee.slice/crio-4652674629bcdce2e7ad2cb85b9259794e3a8d0da460757e575778b21812c75b WatchSource:0}: Error finding container 4652674629bcdce2e7ad2cb85b9259794e3a8d0da460757e575778b21812c75b: Status 404 returned error can't find the container with id 4652674629bcdce2e7ad2cb85b9259794e3a8d0da460757e575778b21812c75b Jul 30 00:30:00.506539 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:30:00.506522 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 30 00:30:01.061032 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:30:01.060996 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756190-nchqp" event={"ID":"f2447b8f-3d7d-48f2-abaf-fd773fd8ecee","Type":"ContainerStarted","Data":"4652674629bcdce2e7ad2cb85b9259794e3a8d0da460757e575778b21812c75b"} Jul 30 00:30:05.083558 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:30:05.083531 2563 generic.go:356] "Generic (PLEG): container finished" podID="f2447b8f-3d7d-48f2-abaf-fd773fd8ecee" containerID="fda1e51704e76bc4d3dcfdc3ce0fb2ff3efa1ae0945f5b914b66b5b56bde92b9" exitCode=0 Jul 30 00:30:05.083870 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:30:05.083612 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756190-nchqp" event={"ID":"f2447b8f-3d7d-48f2-abaf-fd773fd8ecee","Type":"ContainerDied","Data":"fda1e51704e76bc4d3dcfdc3ce0fb2ff3efa1ae0945f5b914b66b5b56bde92b9"} Jul 30 00:30:06.223726 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:30:06.223704 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756190-nchqp" Jul 30 00:30:07.094774 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:30:07.094721 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756190-nchqp" Jul 30 00:30:07.094942 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:30:07.094720 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756190-nchqp" event={"ID":"f2447b8f-3d7d-48f2-abaf-fd773fd8ecee","Type":"ContainerDied","Data":"4652674629bcdce2e7ad2cb85b9259794e3a8d0da460757e575778b21812c75b"} Jul 30 00:30:07.094942 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:30:07.094866 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4652674629bcdce2e7ad2cb85b9259794e3a8d0da460757e575778b21812c75b" Jul 30 00:30:15.601989 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:30:15.601958 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-54cd44d85-rjqq4"] Jul 30 00:30:15.602358 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:30:15.602344 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2447b8f-3d7d-48f2-abaf-fd773fd8ecee" containerName="cleanup" Jul 30 00:30:15.602403 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:30:15.602359 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2447b8f-3d7d-48f2-abaf-fd773fd8ecee" containerName="cleanup" Jul 30 00:30:15.602439 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:30:15.602421 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2447b8f-3d7d-48f2-abaf-fd773fd8ecee" containerName="cleanup" Jul 30 00:30:15.605383 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:30:15.605366 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-54cd44d85-rjqq4" Jul 30 00:30:15.608678 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:30:15.608656 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-cjb69\"" Jul 30 00:30:15.608824 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:30:15.608654 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 30 00:30:15.613969 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:30:15.613943 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-54cd44d85-rjqq4"] Jul 30 00:30:15.678529 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:30:15.678507 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbzf4\" (UniqueName: \"kubernetes.io/projected/9e19c7ad-fe70-44a0-a535-cb2889a76b52-kube-api-access-dbzf4\") pod \"authorino-54cd44d85-rjqq4\" (UID: \"9e19c7ad-fe70-44a0-a535-cb2889a76b52\") " pod="kuadrant-system/authorino-54cd44d85-rjqq4" Jul 30 00:30:15.678642 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:30:15.678567 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9e19c7ad-fe70-44a0-a535-cb2889a76b52-tls-cert\") pod \"authorino-54cd44d85-rjqq4\" (UID: \"9e19c7ad-fe70-44a0-a535-cb2889a76b52\") " pod="kuadrant-system/authorino-54cd44d85-rjqq4" Jul 30 00:30:15.780006 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:30:15.779972 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbzf4\" (UniqueName: \"kubernetes.io/projected/9e19c7ad-fe70-44a0-a535-cb2889a76b52-kube-api-access-dbzf4\") pod \"authorino-54cd44d85-rjqq4\" (UID: \"9e19c7ad-fe70-44a0-a535-cb2889a76b52\") " pod="kuadrant-system/authorino-54cd44d85-rjqq4" Jul 30 00:30:15.780168 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:30:15.780073 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9e19c7ad-fe70-44a0-a535-cb2889a76b52-tls-cert\") pod \"authorino-54cd44d85-rjqq4\" (UID: \"9e19c7ad-fe70-44a0-a535-cb2889a76b52\") " pod="kuadrant-system/authorino-54cd44d85-rjqq4" Jul 30 00:30:15.782583 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:30:15.782558 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9e19c7ad-fe70-44a0-a535-cb2889a76b52-tls-cert\") pod \"authorino-54cd44d85-rjqq4\" (UID: \"9e19c7ad-fe70-44a0-a535-cb2889a76b52\") " pod="kuadrant-system/authorino-54cd44d85-rjqq4" Jul 30 00:30:15.791128 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:30:15.791107 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbzf4\" (UniqueName: \"kubernetes.io/projected/9e19c7ad-fe70-44a0-a535-cb2889a76b52-kube-api-access-dbzf4\") pod \"authorino-54cd44d85-rjqq4\" (UID: \"9e19c7ad-fe70-44a0-a535-cb2889a76b52\") " pod="kuadrant-system/authorino-54cd44d85-rjqq4" Jul 30 00:30:15.914769 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:30:15.914679 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-54cd44d85-rjqq4" Jul 30 00:30:16.047941 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:30:16.047910 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-54cd44d85-rjqq4"] Jul 30 00:30:16.051554 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:30:16.051529 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9e19c7ad_fe70_44a0_a535_cb2889a76b52.slice/crio-3af31e1a0ecfe88518e8a748ef71a88116f9560e326c1472cea2eace98edaa44 WatchSource:0}: Error finding container 3af31e1a0ecfe88518e8a748ef71a88116f9560e326c1472cea2eace98edaa44: Status 404 returned error can't find the container with id 3af31e1a0ecfe88518e8a748ef71a88116f9560e326c1472cea2eace98edaa44 Jul 30 00:30:16.141184 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:30:16.141152 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-54cd44d85-rjqq4" event={"ID":"9e19c7ad-fe70-44a0-a535-cb2889a76b52","Type":"ContainerStarted","Data":"3af31e1a0ecfe88518e8a748ef71a88116f9560e326c1472cea2eace98edaa44"} Jul 30 00:30:17.148125 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:30:17.148083 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-54cd44d85-rjqq4" event={"ID":"9e19c7ad-fe70-44a0-a535-cb2889a76b52","Type":"ContainerStarted","Data":"05082860b72311cae924be09bf567f5c4e9139b329f073ae7fef4eb612e3f77d"} Jul 30 00:30:17.168540 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:30:17.168482 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-54cd44d85-rjqq4" podStartSLOduration=1.706092 podStartE2EDuration="2.168465723s" totalImagesPullingTime="462.373723ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:30:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:30:17.167345134 +0000 UTC m=+781.226101105" watchObservedRunningTime="2026-07-30 00:30:17.168465723 +0000 UTC m=+781.227221691" Jul 30 00:30:17.687155 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:30:17.687114 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:30:17.754076 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:30:17.754038 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:30:18.734230 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:30:18.734197 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:30:18.833018 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:30:18.832971 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:31:06.804445 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:31:06.804412 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-bd495bbc5-8697p"] Jul 30 00:31:06.804937 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:31:06.804627 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-bd495bbc5-8697p" podUID="1cf550e0-db96-46a1-b7f5-56be853a5ba4" containerName="manager" containerID="cri-o://ae8b1c192e49cc40f819c79ccf130316275473ef0d0503604d881bbe0992dcc6" gracePeriod=30 Jul 30 00:31:07.062370 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:31:07.062311 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-bd495bbc5-8697p" Jul 30 00:31:07.144663 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:31:07.144629 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8hlnb\" (UniqueName: \"kubernetes.io/projected/1cf550e0-db96-46a1-b7f5-56be853a5ba4-kube-api-access-8hlnb\") pod \"1cf550e0-db96-46a1-b7f5-56be853a5ba4\" (UID: \"1cf550e0-db96-46a1-b7f5-56be853a5ba4\") " Jul 30 00:31:07.144861 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:31:07.144751 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1cf550e0-db96-46a1-b7f5-56be853a5ba4-webhook-cert\") pod \"1cf550e0-db96-46a1-b7f5-56be853a5ba4\" (UID: \"1cf550e0-db96-46a1-b7f5-56be853a5ba4\") " Jul 30 00:31:07.146833 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:31:07.146808 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cf550e0-db96-46a1-b7f5-56be853a5ba4-kube-api-access-8hlnb" (OuterVolumeSpecName: "kube-api-access-8hlnb") pod "1cf550e0-db96-46a1-b7f5-56be853a5ba4" (UID: "1cf550e0-db96-46a1-b7f5-56be853a5ba4"). InnerVolumeSpecName "kube-api-access-8hlnb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 00:31:07.146897 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:31:07.146878 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1cf550e0-db96-46a1-b7f5-56be853a5ba4-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "1cf550e0-db96-46a1-b7f5-56be853a5ba4" (UID: "1cf550e0-db96-46a1-b7f5-56be853a5ba4"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 00:31:07.246348 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:31:07.246318 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8hlnb\" (UniqueName: \"kubernetes.io/projected/1cf550e0-db96-46a1-b7f5-56be853a5ba4-kube-api-access-8hlnb\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:31:07.246348 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:31:07.246342 2563 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1cf550e0-db96-46a1-b7f5-56be853a5ba4-webhook-cert\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:31:07.416714 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:31:07.416630 2563 generic.go:356] "Generic (PLEG): container finished" podID="1cf550e0-db96-46a1-b7f5-56be853a5ba4" containerID="ae8b1c192e49cc40f819c79ccf130316275473ef0d0503604d881bbe0992dcc6" exitCode=0 Jul 30 00:31:07.416714 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:31:07.416671 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-bd495bbc5-8697p" Jul 30 00:31:07.416714 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:31:07.416683 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-bd495bbc5-8697p" event={"ID":"1cf550e0-db96-46a1-b7f5-56be853a5ba4","Type":"ContainerDied","Data":"ae8b1c192e49cc40f819c79ccf130316275473ef0d0503604d881bbe0992dcc6"} Jul 30 00:31:07.416954 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:31:07.416722 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-bd495bbc5-8697p" event={"ID":"1cf550e0-db96-46a1-b7f5-56be853a5ba4","Type":"ContainerDied","Data":"b20662c52436d35069780791d4fb4d98a98a6da71f1c84acf23c2646e55bca66"} Jul 30 00:31:07.416954 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:31:07.416768 2563 scope.go:117] "RemoveContainer" containerID="ae8b1c192e49cc40f819c79ccf130316275473ef0d0503604d881bbe0992dcc6" Jul 30 00:31:07.431998 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:31:07.431978 2563 scope.go:117] "RemoveContainer" containerID="ae8b1c192e49cc40f819c79ccf130316275473ef0d0503604d881bbe0992dcc6" Jul 30 00:31:07.432301 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:31:07.432282 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae8b1c192e49cc40f819c79ccf130316275473ef0d0503604d881bbe0992dcc6\": container with ID starting with ae8b1c192e49cc40f819c79ccf130316275473ef0d0503604d881bbe0992dcc6 not found: ID does not exist" containerID="ae8b1c192e49cc40f819c79ccf130316275473ef0d0503604d881bbe0992dcc6" Jul 30 00:31:07.432356 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:31:07.432310 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae8b1c192e49cc40f819c79ccf130316275473ef0d0503604d881bbe0992dcc6"} err="failed to get container status \"ae8b1c192e49cc40f819c79ccf130316275473ef0d0503604d881bbe0992dcc6\": rpc error: code = NotFound desc = could not find container \"ae8b1c192e49cc40f819c79ccf130316275473ef0d0503604d881bbe0992dcc6\": container with ID starting with ae8b1c192e49cc40f819c79ccf130316275473ef0d0503604d881bbe0992dcc6 not found: ID does not exist" Jul 30 00:31:07.441203 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:31:07.441179 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-bd495bbc5-8697p"] Jul 30 00:31:07.444977 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:31:07.444958 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-bd495bbc5-8697p"] Jul 30 00:31:08.476118 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:31:08.476085 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1cf550e0-db96-46a1-b7f5-56be853a5ba4" path="/var/lib/kubelet/pods/1cf550e0-db96-46a1-b7f5-56be853a5ba4/volumes" Jul 30 00:32:16.404195 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:32:16.404109 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-b92tw_0ef8f120-f712-4ed9-adbd-40f785d4aa9d/console-operator/1.log" Jul 30 00:32:16.408523 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:32:16.408500 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-b92tw_0ef8f120-f712-4ed9-adbd-40f785d4aa9d/console-operator/1.log" Jul 30 00:32:27.819906 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:32:27.819869 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:32:33.907820 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:32:33.907787 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:32:53.430403 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:32:53.430364 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:33:14.201334 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:33:14.201291 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:33:14.306435 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:33:14.306403 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:33:14.397868 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:33:14.397833 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:33:20.189912 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:33:20.189879 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:33:20.240814 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:33:20.240779 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:33:20.316087 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:33:20.316057 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:33:20.416087 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:33:20.416048 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:33:20.515553 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:33:20.515518 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:33:26.212470 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:33:26.212433 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:33:26.347707 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:33:26.347676 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:36:16.785953 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:36:16.785910 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:37:04.093223 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:37:04.093182 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:37:16.436577 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:37:16.436550 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-b92tw_0ef8f120-f712-4ed9-adbd-40f785d4aa9d/console-operator/1.log" Jul 30 00:37:16.437096 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:37:16.437080 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-b92tw_0ef8f120-f712-4ed9-adbd-40f785d4aa9d/console-operator/1.log" Jul 30 00:37:31.626471 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:37:31.626438 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tmqb4"] Jul 30 00:37:31.626920 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:37:31.626621 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tmqb4" podUID="92d878fa-dd6e-4025-abc3-35f3cbf1d85b" containerName="manager" containerID="cri-o://6fea561e469d1bbb559718ae6b2cdf0d2f293bcdccbe9542e801cb699ade03d5" gracePeriod=10 Jul 30 00:37:35.281389 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:37:35.281362 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tmqb4" Jul 30 00:37:35.435393 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:37:35.435304 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-db4rq\" (UniqueName: \"kubernetes.io/projected/92d878fa-dd6e-4025-abc3-35f3cbf1d85b-kube-api-access-db4rq\") pod \"92d878fa-dd6e-4025-abc3-35f3cbf1d85b\" (UID: \"92d878fa-dd6e-4025-abc3-35f3cbf1d85b\") " Jul 30 00:37:35.435393 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:37:35.435351 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/92d878fa-dd6e-4025-abc3-35f3cbf1d85b-extensions-socket-volume\") pod \"92d878fa-dd6e-4025-abc3-35f3cbf1d85b\" (UID: \"92d878fa-dd6e-4025-abc3-35f3cbf1d85b\") " Jul 30 00:37:35.435818 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:37:35.435788 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92d878fa-dd6e-4025-abc3-35f3cbf1d85b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "92d878fa-dd6e-4025-abc3-35f3cbf1d85b" (UID: "92d878fa-dd6e-4025-abc3-35f3cbf1d85b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:37:35.437518 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:37:35.437488 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92d878fa-dd6e-4025-abc3-35f3cbf1d85b-kube-api-access-db4rq" (OuterVolumeSpecName: "kube-api-access-db4rq") pod "92d878fa-dd6e-4025-abc3-35f3cbf1d85b" (UID: "92d878fa-dd6e-4025-abc3-35f3cbf1d85b"). InnerVolumeSpecName "kube-api-access-db4rq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 00:37:35.437943 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:37:35.437917 2563 generic.go:356] "Generic (PLEG): container finished" podID="92d878fa-dd6e-4025-abc3-35f3cbf1d85b" containerID="6fea561e469d1bbb559718ae6b2cdf0d2f293bcdccbe9542e801cb699ade03d5" exitCode=0 Jul 30 00:37:35.438063 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:37:35.437971 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tmqb4" Jul 30 00:37:35.438063 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:37:35.437998 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tmqb4" event={"ID":"92d878fa-dd6e-4025-abc3-35f3cbf1d85b","Type":"ContainerDied","Data":"6fea561e469d1bbb559718ae6b2cdf0d2f293bcdccbe9542e801cb699ade03d5"} Jul 30 00:37:35.438063 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:37:35.438054 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tmqb4" event={"ID":"92d878fa-dd6e-4025-abc3-35f3cbf1d85b","Type":"ContainerDied","Data":"945b656648e41ba7c752a566f8683f5607e22ef521992ff4262fca0952d1876f"} Jul 30 00:37:35.438238 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:37:35.438084 2563 scope.go:117] "RemoveContainer" containerID="6fea561e469d1bbb559718ae6b2cdf0d2f293bcdccbe9542e801cb699ade03d5" Jul 30 00:37:35.453644 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:37:35.453624 2563 scope.go:117] "RemoveContainer" containerID="6fea561e469d1bbb559718ae6b2cdf0d2f293bcdccbe9542e801cb699ade03d5" Jul 30 00:37:35.453934 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:37:35.453911 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6fea561e469d1bbb559718ae6b2cdf0d2f293bcdccbe9542e801cb699ade03d5\": container with ID starting with 6fea561e469d1bbb559718ae6b2cdf0d2f293bcdccbe9542e801cb699ade03d5 not found: ID does not exist" containerID="6fea561e469d1bbb559718ae6b2cdf0d2f293bcdccbe9542e801cb699ade03d5" Jul 30 00:37:35.454003 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:37:35.453942 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fea561e469d1bbb559718ae6b2cdf0d2f293bcdccbe9542e801cb699ade03d5"} err="failed to get container status \"6fea561e469d1bbb559718ae6b2cdf0d2f293bcdccbe9542e801cb699ade03d5\": rpc error: code = NotFound desc = could not find container \"6fea561e469d1bbb559718ae6b2cdf0d2f293bcdccbe9542e801cb699ade03d5\": container with ID starting with 6fea561e469d1bbb559718ae6b2cdf0d2f293bcdccbe9542e801cb699ade03d5 not found: ID does not exist" Jul 30 00:37:35.463926 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:37:35.463904 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tmqb4"] Jul 30 00:37:35.469326 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:37:35.469303 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tmqb4"] Jul 30 00:37:35.536914 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:37:35.536879 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-db4rq\" (UniqueName: \"kubernetes.io/projected/92d878fa-dd6e-4025-abc3-35f3cbf1d85b-kube-api-access-db4rq\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:37:35.536914 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:37:35.536910 2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/92d878fa-dd6e-4025-abc3-35f3cbf1d85b-extensions-socket-volume\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:37:36.474915 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:37:36.474882 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92d878fa-dd6e-4025-abc3-35f3cbf1d85b" path="/var/lib/kubelet/pods/92d878fa-dd6e-4025-abc3-35f3cbf1d85b/volumes" Jul 30 00:38:32.676193 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:38:32.676154 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-k9mh2"] Jul 30 00:38:32.676592 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:38:32.676530 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92d878fa-dd6e-4025-abc3-35f3cbf1d85b" containerName="manager" Jul 30 00:38:32.676592 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:38:32.676540 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="92d878fa-dd6e-4025-abc3-35f3cbf1d85b" containerName="manager" Jul 30 00:38:32.676592 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:38:32.676552 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cf550e0-db96-46a1-b7f5-56be853a5ba4" containerName="manager" Jul 30 00:38:32.676592 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:38:32.676557 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cf550e0-db96-46a1-b7f5-56be853a5ba4" containerName="manager" Jul 30 00:38:32.676724 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:38:32.676624 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="92d878fa-dd6e-4025-abc3-35f3cbf1d85b" containerName="manager" Jul 30 00:38:32.676724 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:38:32.676633 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cf550e0-db96-46a1-b7f5-56be853a5ba4" containerName="manager" Jul 30 00:38:32.680943 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:38:32.680926 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-k9mh2" Jul 30 00:38:32.683280 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:38:32.683258 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-hmwnt\"" Jul 30 00:38:32.690044 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:38:32.690017 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-k9mh2"] Jul 30 00:38:32.716597 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:38:32.716565 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xx6xr\" (UniqueName: \"kubernetes.io/projected/71b960ca-fb8c-4fcd-afff-88a9627a3b23-kube-api-access-xx6xr\") pod \"kuadrant-operator-controller-manager-86878c769f-k9mh2\" (UID: \"71b960ca-fb8c-4fcd-afff-88a9627a3b23\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-k9mh2" Jul 30 00:38:32.716776 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:38:32.716661 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/71b960ca-fb8c-4fcd-afff-88a9627a3b23-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-k9mh2\" (UID: \"71b960ca-fb8c-4fcd-afff-88a9627a3b23\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-k9mh2" Jul 30 00:38:32.817247 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:38:32.817216 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xx6xr\" (UniqueName: \"kubernetes.io/projected/71b960ca-fb8c-4fcd-afff-88a9627a3b23-kube-api-access-xx6xr\") pod \"kuadrant-operator-controller-manager-86878c769f-k9mh2\" (UID: \"71b960ca-fb8c-4fcd-afff-88a9627a3b23\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-k9mh2" Jul 30 00:38:32.817400 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:38:32.817274 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/71b960ca-fb8c-4fcd-afff-88a9627a3b23-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-k9mh2\" (UID: \"71b960ca-fb8c-4fcd-afff-88a9627a3b23\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-k9mh2" Jul 30 00:38:32.817602 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:38:32.817577 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/71b960ca-fb8c-4fcd-afff-88a9627a3b23-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-k9mh2\" (UID: \"71b960ca-fb8c-4fcd-afff-88a9627a3b23\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-k9mh2" Jul 30 00:38:32.825502 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:38:32.825475 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xx6xr\" (UniqueName: \"kubernetes.io/projected/71b960ca-fb8c-4fcd-afff-88a9627a3b23-kube-api-access-xx6xr\") pod \"kuadrant-operator-controller-manager-86878c769f-k9mh2\" (UID: \"71b960ca-fb8c-4fcd-afff-88a9627a3b23\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-k9mh2" Jul 30 00:38:32.991827 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:38:32.991727 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-k9mh2" Jul 30 00:38:33.134021 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:38:33.133997 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-k9mh2"] Jul 30 00:38:33.136134 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:38:33.136107 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71b960ca_fb8c_4fcd_afff_88a9627a3b23.slice/crio-82875514280b707e73822b62e1d7bf5341a2b8a7a64b48006602e75ccb5e4f74 WatchSource:0}: Error finding container 82875514280b707e73822b62e1d7bf5341a2b8a7a64b48006602e75ccb5e4f74: Status 404 returned error can't find the container with id 82875514280b707e73822b62e1d7bf5341a2b8a7a64b48006602e75ccb5e4f74 Jul 30 00:38:33.727370 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:38:33.727328 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-k9mh2" event={"ID":"71b960ca-fb8c-4fcd-afff-88a9627a3b23","Type":"ContainerStarted","Data":"b4a4e220bbb72cf71170eabb456dc041b3d6c98f66a41755603437d43e72873c"} Jul 30 00:38:33.727883 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:38:33.727387 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-k9mh2" event={"ID":"71b960ca-fb8c-4fcd-afff-88a9627a3b23","Type":"ContainerStarted","Data":"82875514280b707e73822b62e1d7bf5341a2b8a7a64b48006602e75ccb5e4f74"} Jul 30 00:38:33.727883 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:38:33.727434 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-k9mh2" Jul 30 00:38:33.748042 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:38:33.747993 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-k9mh2" podStartSLOduration=1.747976828 podStartE2EDuration="1.747976828s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:38:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:38:33.74647136 +0000 UTC m=+1277.805227339" watchObservedRunningTime="2026-07-30 00:38:33.747976828 +0000 UTC m=+1277.806732796" Jul 30 00:38:44.735266 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:38:44.735235 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-k9mh2" Jul 30 00:39:54.801727 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:39:54.801651 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:41:25.056939 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:41:25.056849 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:41:32.643994 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:41:32.643958 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:41:56.871615 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:41:56.871577 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:41:57.044511 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:41:57.044476 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:42:16.474834 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:42:16.474805 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-b92tw_0ef8f120-f712-4ed9-adbd-40f785d4aa9d/console-operator/1.log" Jul 30 00:42:16.475266 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:42:16.475046 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-b92tw_0ef8f120-f712-4ed9-adbd-40f785d4aa9d/console-operator/1.log" Jul 30 00:42:25.851693 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:42:25.851650 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:42:25.904633 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:42:25.904601 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:42:27.003331 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:42:27.003300 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:07.886065 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:07.886029 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:23.925979 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:23.925944 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:26.555489 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:26.555454 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:29.440561 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:29.440517 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:29.573882 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:29.573852 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:29.859395 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:29.859358 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:35.700311 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:35.700273 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:35.766647 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:35.766611 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:35.926123 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:35.926085 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:36.201576 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:36.201542 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx"] Jul 30 00:45:36.205688 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:36.205663 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx" Jul 30 00:45:36.208213 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:36.208192 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-a5930c62-openshift-default-dockercfg-75tth\"" Jul 30 00:45:36.217036 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:36.217015 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx"] Jul 30 00:45:36.219593 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:36.219569 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:36.302445 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:36.302408 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff35d36a-d137-457e-8dfc-8110aa63cf92-istiod-ca-cert\") pod \"e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx\" (UID: \"ff35d36a-d137-457e-8dfc-8110aa63cf92\") " pod="openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx" Jul 30 00:45:36.302641 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:36.302457 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5hfk\" (UniqueName: \"kubernetes.io/projected/ff35d36a-d137-457e-8dfc-8110aa63cf92-kube-api-access-l5hfk\") pod \"e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx\" (UID: \"ff35d36a-d137-457e-8dfc-8110aa63cf92\") " pod="openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx" Jul 30 00:45:36.302641 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:36.302550 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff35d36a-d137-457e-8dfc-8110aa63cf92-credential-socket\") pod \"e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx\" (UID: \"ff35d36a-d137-457e-8dfc-8110aa63cf92\") " pod="openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx" Jul 30 00:45:36.302797 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:36.302643 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff35d36a-d137-457e-8dfc-8110aa63cf92-workload-socket\") pod \"e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx\" (UID: \"ff35d36a-d137-457e-8dfc-8110aa63cf92\") " pod="openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx" Jul 30 00:45:36.302797 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:36.302690 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff35d36a-d137-457e-8dfc-8110aa63cf92-workload-certs\") pod \"e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx\" (UID: \"ff35d36a-d137-457e-8dfc-8110aa63cf92\") " pod="openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx" Jul 30 00:45:36.302797 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:36.302729 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff35d36a-d137-457e-8dfc-8110aa63cf92-istio-envoy\") pod \"e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx\" (UID: \"ff35d36a-d137-457e-8dfc-8110aa63cf92\") " pod="openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx" Jul 30 00:45:36.302966 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:36.302801 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff35d36a-d137-457e-8dfc-8110aa63cf92-istio-data\") pod \"e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx\" (UID: \"ff35d36a-d137-457e-8dfc-8110aa63cf92\") " pod="openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx" Jul 30 00:45:36.302966 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:36.302826 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff35d36a-d137-457e-8dfc-8110aa63cf92-istio-podinfo\") pod \"e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx\" (UID: \"ff35d36a-d137-457e-8dfc-8110aa63cf92\") " pod="openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx" Jul 30 00:45:36.302966 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:36.302848 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff35d36a-d137-457e-8dfc-8110aa63cf92-istio-token\") pod \"e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx\" (UID: \"ff35d36a-d137-457e-8dfc-8110aa63cf92\") " pod="openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx" Jul 30 00:45:36.403424 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:36.403385 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5hfk\" (UniqueName: \"kubernetes.io/projected/ff35d36a-d137-457e-8dfc-8110aa63cf92-kube-api-access-l5hfk\") pod \"e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx\" (UID: \"ff35d36a-d137-457e-8dfc-8110aa63cf92\") " pod="openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx" Jul 30 00:45:36.403605 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:36.403432 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff35d36a-d137-457e-8dfc-8110aa63cf92-credential-socket\") pod \"e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx\" (UID: \"ff35d36a-d137-457e-8dfc-8110aa63cf92\") " pod="openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx" Jul 30 00:45:36.403605 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:36.403479 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff35d36a-d137-457e-8dfc-8110aa63cf92-workload-socket\") pod \"e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx\" (UID: \"ff35d36a-d137-457e-8dfc-8110aa63cf92\") " pod="openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx" Jul 30 00:45:36.403605 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:36.403506 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff35d36a-d137-457e-8dfc-8110aa63cf92-workload-certs\") pod \"e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx\" (UID: \"ff35d36a-d137-457e-8dfc-8110aa63cf92\") " pod="openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx" Jul 30 00:45:36.403605 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:36.403538 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff35d36a-d137-457e-8dfc-8110aa63cf92-istio-envoy\") pod \"e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx\" (UID: \"ff35d36a-d137-457e-8dfc-8110aa63cf92\") " pod="openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx" Jul 30 00:45:36.403605 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:36.403575 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff35d36a-d137-457e-8dfc-8110aa63cf92-istio-data\") pod \"e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx\" (UID: \"ff35d36a-d137-457e-8dfc-8110aa63cf92\") " pod="openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx" Jul 30 00:45:36.403605 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:36.403600 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff35d36a-d137-457e-8dfc-8110aa63cf92-istio-podinfo\") pod \"e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx\" (UID: \"ff35d36a-d137-457e-8dfc-8110aa63cf92\") " pod="openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx" Jul 30 00:45:36.403971 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:36.403623 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff35d36a-d137-457e-8dfc-8110aa63cf92-istio-token\") pod \"e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx\" (UID: \"ff35d36a-d137-457e-8dfc-8110aa63cf92\") " pod="openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx" Jul 30 00:45:36.403971 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:36.403653 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff35d36a-d137-457e-8dfc-8110aa63cf92-istiod-ca-cert\") pod \"e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx\" (UID: \"ff35d36a-d137-457e-8dfc-8110aa63cf92\") " pod="openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx" Jul 30 00:45:36.404079 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:36.404036 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff35d36a-d137-457e-8dfc-8110aa63cf92-credential-socket\") pod \"e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx\" (UID: \"ff35d36a-d137-457e-8dfc-8110aa63cf92\") " pod="openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx" Jul 30 00:45:36.404224 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:36.404195 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff35d36a-d137-457e-8dfc-8110aa63cf92-istio-data\") pod \"e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx\" (UID: \"ff35d36a-d137-457e-8dfc-8110aa63cf92\") " pod="openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx" Jul 30 00:45:36.404351 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:36.404236 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff35d36a-d137-457e-8dfc-8110aa63cf92-workload-certs\") pod \"e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx\" (UID: \"ff35d36a-d137-457e-8dfc-8110aa63cf92\") " pod="openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx" Jul 30 00:45:36.404408 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:36.404360 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff35d36a-d137-457e-8dfc-8110aa63cf92-workload-socket\") pod \"e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx\" (UID: \"ff35d36a-d137-457e-8dfc-8110aa63cf92\") " pod="openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx" Jul 30 00:45:36.404462 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:36.404442 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff35d36a-d137-457e-8dfc-8110aa63cf92-istiod-ca-cert\") pod \"e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx\" (UID: \"ff35d36a-d137-457e-8dfc-8110aa63cf92\") " pod="openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx" Jul 30 00:45:36.406032 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:36.406010 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff35d36a-d137-457e-8dfc-8110aa63cf92-istio-envoy\") pod \"e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx\" (UID: \"ff35d36a-d137-457e-8dfc-8110aa63cf92\") " pod="openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx" Jul 30 00:45:36.406250 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:36.406228 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff35d36a-d137-457e-8dfc-8110aa63cf92-istio-podinfo\") pod \"e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx\" (UID: \"ff35d36a-d137-457e-8dfc-8110aa63cf92\") " pod="openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx" Jul 30 00:45:36.416962 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:36.416936 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff35d36a-d137-457e-8dfc-8110aa63cf92-istio-token\") pod \"e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx\" (UID: \"ff35d36a-d137-457e-8dfc-8110aa63cf92\") " pod="openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx" Jul 30 00:45:36.417443 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:36.417425 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5hfk\" (UniqueName: \"kubernetes.io/projected/ff35d36a-d137-457e-8dfc-8110aa63cf92-kube-api-access-l5hfk\") pod \"e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx\" (UID: \"ff35d36a-d137-457e-8dfc-8110aa63cf92\") " pod="openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx" Jul 30 00:45:36.478501 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:36.478422 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:36.516727 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:36.516689 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx" Jul 30 00:45:36.656145 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:36.656073 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx"] Jul 30 00:45:36.659834 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:45:36.659794 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff35d36a_d137_457e_8dfc_8110aa63cf92.slice/crio-37b8c3454df399efb625d8332f1d7cc2fff599cb11b8344fab3769cd7feff375 WatchSource:0}: Error finding container 37b8c3454df399efb625d8332f1d7cc2fff599cb11b8344fab3769cd7feff375: Status 404 returned error can't find the container with id 37b8c3454df399efb625d8332f1d7cc2fff599cb11b8344fab3769cd7feff375 Jul 30 00:45:36.662547 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:36.662508 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:45:36.662677 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:36.662592 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:45:36.662677 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:36.662646 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:45:36.905546 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:36.905501 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx" event={"ID":"ff35d36a-d137-457e-8dfc-8110aa63cf92","Type":"ContainerStarted","Data":"ad373f0699e5f2a995c1105a91936a83ee6f40da6225b57056f78d2128aefcd6"} Jul 30 00:45:36.905933 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:36.905575 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx" event={"ID":"ff35d36a-d137-457e-8dfc-8110aa63cf92","Type":"ContainerStarted","Data":"37b8c3454df399efb625d8332f1d7cc2fff599cb11b8344fab3769cd7feff375"} Jul 30 00:45:36.923848 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:36.923793 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx" podStartSLOduration=0.923778439 podStartE2EDuration="923.778439ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:45:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:45:36.923213233 +0000 UTC m=+1700.981969201" watchObservedRunningTime="2026-07-30 00:45:36.923778439 +0000 UTC m=+1700.982534407" Jul 30 00:45:37.517056 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:37.517021 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx" Jul 30 00:45:37.521981 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:37.521953 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx" Jul 30 00:45:37.910949 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:37.910846 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx" Jul 30 00:45:37.911943 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:37.911919 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx" Jul 30 00:45:38.333362 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:38.333329 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:38.435187 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:38.435148 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:39.617564 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:39.617524 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:39.767593 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:39.767549 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:40.048514 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:40.048479 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:40.138962 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:40.138926 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:45:52.912444 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:52.912406 2563 patch_prober.go:28] interesting pod/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 30 00:45:52.912909 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:52.912462 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx" podUID="ff35d36a-d137-457e-8dfc-8110aa63cf92" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 30 00:45:53.014582 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:53.014547 2563 generic.go:356] "Generic (PLEG): container finished" podID="ff35d36a-d137-457e-8dfc-8110aa63cf92" containerID="ad373f0699e5f2a995c1105a91936a83ee6f40da6225b57056f78d2128aefcd6" exitCode=0 Jul 30 00:45:53.014765 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:53.014621 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx" event={"ID":"ff35d36a-d137-457e-8dfc-8110aa63cf92","Type":"ContainerDied","Data":"ad373f0699e5f2a995c1105a91936a83ee6f40da6225b57056f78d2128aefcd6"} Jul 30 00:45:53.015078 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:53.015063 2563 scope.go:117] "RemoveContainer" containerID="ad373f0699e5f2a995c1105a91936a83ee6f40da6225b57056f78d2128aefcd6" Jul 30 00:45:53.016115 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:53.016086 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:45:53.016181 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:53.016169 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:45:53.016216 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:53.016203 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:45:54.022231 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:54.022197 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx" event={"ID":"ff35d36a-d137-457e-8dfc-8110aa63cf92","Type":"ContainerStarted","Data":"352cf374b489f08a73a34fb3c09b2ac2fdc79b037b1634b11ed5d124c8debe24"} Jul 30 00:45:54.517635 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:54.517603 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx" Jul 30 00:45:54.517986 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:54.517956 2563 patch_prober.go:28] interesting pod/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 30 00:45:54.518097 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:54.518018 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx" podUID="ff35d36a-d137-457e-8dfc-8110aa63cf92" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 30 00:45:55.517988 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:55.517954 2563 patch_prober.go:28] interesting pod/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 30 00:45:55.518457 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:55.518008 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx" podUID="ff35d36a-d137-457e-8dfc-8110aa63cf92" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 30 00:45:56.517235 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:56.517200 2563 patch_prober.go:28] interesting pod/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 30 00:45:56.517415 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:56.517257 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx" podUID="ff35d36a-d137-457e-8dfc-8110aa63cf92" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 30 00:45:57.517574 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:57.517534 2563 patch_prober.go:28] interesting pod/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 30 00:45:57.518012 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:57.517601 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx" podUID="ff35d36a-d137-457e-8dfc-8110aa63cf92" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 30 00:45:58.520725 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:58.520691 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx" Jul 30 00:45:58.521216 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:58.521117 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx" Jul 30 00:45:58.521704 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:45:58.521686 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx" Jul 30 00:46:10.013873 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:10.013834 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:46:10.131870 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:10.131835 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:46:10.231856 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:10.231826 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:46:10.291395 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:10.291306 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:46:10.361404 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:10.361367 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx"] Jul 30 00:46:10.361597 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:10.361581 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx" podUID="ff35d36a-d137-457e-8dfc-8110aa63cf92" containerName="istio-proxy" containerID="cri-o://352cf374b489f08a73a34fb3c09b2ac2fdc79b037b1634b11ed5d124c8debe24" gracePeriod=30 Jul 30 00:46:10.392823 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:10.392787 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:46:10.589300 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:10.589216 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:46:10.821538 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:10.821501 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:46:10.917716 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:10.917635 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:46:11.240384 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:11.240296 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:46:11.278616 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:11.278587 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk"] Jul 30 00:46:11.282526 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:11.282505 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk" Jul 30 00:46:11.285043 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:11.285026 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-837664ed-gw-openshift-default-dockercfg-wvnzm\"" Jul 30 00:46:11.294129 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:11.294104 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk"] Jul 30 00:46:11.316929 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:11.316893 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:46:11.324983 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:11.324954 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96eb3aeb-161b-4bd6-a26e-659d1e2dd215-istio-podinfo\") pod \"e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk\" (UID: \"96eb3aeb-161b-4bd6-a26e-659d1e2dd215\") " pod="openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk" Jul 30 00:46:11.325094 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:11.324995 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96eb3aeb-161b-4bd6-a26e-659d1e2dd215-credential-socket\") pod \"e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk\" (UID: \"96eb3aeb-161b-4bd6-a26e-659d1e2dd215\") " pod="openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk" Jul 30 00:46:11.325094 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:11.325016 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96eb3aeb-161b-4bd6-a26e-659d1e2dd215-workload-certs\") pod \"e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk\" (UID: \"96eb3aeb-161b-4bd6-a26e-659d1e2dd215\") " pod="openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk" Jul 30 00:46:11.325094 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:11.325072 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96eb3aeb-161b-4bd6-a26e-659d1e2dd215-istiod-ca-cert\") pod \"e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk\" (UID: \"96eb3aeb-161b-4bd6-a26e-659d1e2dd215\") " pod="openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk" Jul 30 00:46:11.325217 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:11.325108 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96eb3aeb-161b-4bd6-a26e-659d1e2dd215-istio-data\") pod \"e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk\" (UID: \"96eb3aeb-161b-4bd6-a26e-659d1e2dd215\") " pod="openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk" Jul 30 00:46:11.325217 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:11.325140 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4s5c\" (UniqueName: \"kubernetes.io/projected/96eb3aeb-161b-4bd6-a26e-659d1e2dd215-kube-api-access-v4s5c\") pod \"e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk\" (UID: \"96eb3aeb-161b-4bd6-a26e-659d1e2dd215\") " pod="openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk" Jul 30 00:46:11.325217 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:11.325185 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96eb3aeb-161b-4bd6-a26e-659d1e2dd215-istio-token\") pod \"e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk\" (UID: \"96eb3aeb-161b-4bd6-a26e-659d1e2dd215\") " pod="openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk" Jul 30 00:46:11.325217 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:11.325213 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96eb3aeb-161b-4bd6-a26e-659d1e2dd215-istio-envoy\") pod \"e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk\" (UID: \"96eb3aeb-161b-4bd6-a26e-659d1e2dd215\") " pod="openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk" Jul 30 00:46:11.325343 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:11.325240 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96eb3aeb-161b-4bd6-a26e-659d1e2dd215-workload-socket\") pod \"e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk\" (UID: \"96eb3aeb-161b-4bd6-a26e-659d1e2dd215\") " pod="openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk" Jul 30 00:46:11.426359 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:11.426322 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96eb3aeb-161b-4bd6-a26e-659d1e2dd215-istio-podinfo\") pod \"e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk\" (UID: \"96eb3aeb-161b-4bd6-a26e-659d1e2dd215\") " pod="openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk" Jul 30 00:46:11.426567 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:11.426386 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96eb3aeb-161b-4bd6-a26e-659d1e2dd215-credential-socket\") pod \"e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk\" (UID: \"96eb3aeb-161b-4bd6-a26e-659d1e2dd215\") " pod="openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk" Jul 30 00:46:11.426567 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:11.426417 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96eb3aeb-161b-4bd6-a26e-659d1e2dd215-workload-certs\") pod \"e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk\" (UID: \"96eb3aeb-161b-4bd6-a26e-659d1e2dd215\") " pod="openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk" Jul 30 00:46:11.426567 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:11.426464 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96eb3aeb-161b-4bd6-a26e-659d1e2dd215-istiod-ca-cert\") pod \"e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk\" (UID: \"96eb3aeb-161b-4bd6-a26e-659d1e2dd215\") " pod="openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk" Jul 30 00:46:11.426567 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:11.426496 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96eb3aeb-161b-4bd6-a26e-659d1e2dd215-istio-data\") pod \"e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk\" (UID: \"96eb3aeb-161b-4bd6-a26e-659d1e2dd215\") " pod="openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk" Jul 30 00:46:11.426567 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:11.426547 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4s5c\" (UniqueName: \"kubernetes.io/projected/96eb3aeb-161b-4bd6-a26e-659d1e2dd215-kube-api-access-v4s5c\") pod \"e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk\" (UID: \"96eb3aeb-161b-4bd6-a26e-659d1e2dd215\") " pod="openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk" Jul 30 00:46:11.426863 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:11.426588 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96eb3aeb-161b-4bd6-a26e-659d1e2dd215-istio-token\") pod \"e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk\" (UID: \"96eb3aeb-161b-4bd6-a26e-659d1e2dd215\") " pod="openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk" Jul 30 00:46:11.426863 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:11.426617 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96eb3aeb-161b-4bd6-a26e-659d1e2dd215-istio-envoy\") pod \"e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk\" (UID: \"96eb3aeb-161b-4bd6-a26e-659d1e2dd215\") " pod="openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk" Jul 30 00:46:11.426863 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:11.426656 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96eb3aeb-161b-4bd6-a26e-659d1e2dd215-workload-socket\") pod \"e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk\" (UID: \"96eb3aeb-161b-4bd6-a26e-659d1e2dd215\") " pod="openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk" Jul 30 00:46:11.427026 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:11.426923 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96eb3aeb-161b-4bd6-a26e-659d1e2dd215-workload-certs\") pod \"e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk\" (UID: \"96eb3aeb-161b-4bd6-a26e-659d1e2dd215\") " pod="openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk" Jul 30 00:46:11.427218 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:11.427198 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96eb3aeb-161b-4bd6-a26e-659d1e2dd215-credential-socket\") pod \"e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk\" (UID: \"96eb3aeb-161b-4bd6-a26e-659d1e2dd215\") " pod="openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk" Jul 30 00:46:11.427286 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:11.427255 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96eb3aeb-161b-4bd6-a26e-659d1e2dd215-istio-data\") pod \"e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk\" (UID: \"96eb3aeb-161b-4bd6-a26e-659d1e2dd215\") " pod="openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk" Jul 30 00:46:11.427347 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:11.427313 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96eb3aeb-161b-4bd6-a26e-659d1e2dd215-workload-socket\") pod \"e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk\" (UID: \"96eb3aeb-161b-4bd6-a26e-659d1e2dd215\") " pod="openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk" Jul 30 00:46:11.427675 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:11.427651 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96eb3aeb-161b-4bd6-a26e-659d1e2dd215-istiod-ca-cert\") pod \"e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk\" (UID: \"96eb3aeb-161b-4bd6-a26e-659d1e2dd215\") " pod="openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk" Jul 30 00:46:11.429168 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:11.429145 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96eb3aeb-161b-4bd6-a26e-659d1e2dd215-istio-podinfo\") pod \"e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk\" (UID: \"96eb3aeb-161b-4bd6-a26e-659d1e2dd215\") " pod="openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk" Jul 30 00:46:11.429371 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:11.429351 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96eb3aeb-161b-4bd6-a26e-659d1e2dd215-istio-envoy\") pod \"e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk\" (UID: \"96eb3aeb-161b-4bd6-a26e-659d1e2dd215\") " pod="openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk" Jul 30 00:46:11.438773 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:11.438748 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96eb3aeb-161b-4bd6-a26e-659d1e2dd215-istio-token\") pod \"e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk\" (UID: \"96eb3aeb-161b-4bd6-a26e-659d1e2dd215\") " pod="openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk" Jul 30 00:46:11.438980 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:11.438964 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4s5c\" (UniqueName: \"kubernetes.io/projected/96eb3aeb-161b-4bd6-a26e-659d1e2dd215-kube-api-access-v4s5c\") pod \"e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk\" (UID: \"96eb3aeb-161b-4bd6-a26e-659d1e2dd215\") " pod="openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk" Jul 30 00:46:11.593378 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:11.593335 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk" Jul 30 00:46:11.744443 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:11.744420 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk"] Jul 30 00:46:11.746390 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:46:11.746358 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96eb3aeb_161b_4bd6_a26e_659d1e2dd215.slice/crio-1d8c3b6f38afd473e4de48a71b950e67d83ac6d2b214db1d694bedaa31d6380d WatchSource:0}: Error finding container 1d8c3b6f38afd473e4de48a71b950e67d83ac6d2b214db1d694bedaa31d6380d: Status 404 returned error can't find the container with id 1d8c3b6f38afd473e4de48a71b950e67d83ac6d2b214db1d694bedaa31d6380d Jul 30 00:46:11.748596 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:11.748565 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:46:11.748696 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:11.748627 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:46:11.748696 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:11.748654 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:46:11.942806 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:11.942704 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:46:12.124814 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:12.124776 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk" event={"ID":"96eb3aeb-161b-4bd6-a26e-659d1e2dd215","Type":"ContainerStarted","Data":"1c2e7f2add896d824f4a76fa0a3d28b5f22d8671c7f8c23563d5a4ddfd8c79ad"} Jul 30 00:46:12.124980 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:12.124838 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk" event={"ID":"96eb3aeb-161b-4bd6-a26e-659d1e2dd215","Type":"ContainerStarted","Data":"1d8c3b6f38afd473e4de48a71b950e67d83ac6d2b214db1d694bedaa31d6380d"} Jul 30 00:46:12.145617 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:12.145565 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk" podStartSLOduration=1.145549177 podStartE2EDuration="1.145549177s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:46:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:46:12.143239767 +0000 UTC m=+1736.201995735" watchObservedRunningTime="2026-07-30 00:46:12.145549177 +0000 UTC m=+1736.204305147" Jul 30 00:46:12.593996 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:12.593957 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk" Jul 30 00:46:12.598689 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:12.598665 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk" Jul 30 00:46:13.130191 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:13.130162 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk" Jul 30 00:46:13.130979 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:13.130961 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk" Jul 30 00:46:13.523598 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:13.523562 2563 patch_prober.go:28] interesting pod/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 30 00:46:13.523797 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:13.523632 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx" podUID="ff35d36a-d137-457e-8dfc-8110aa63cf92" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 30 00:46:13.721525 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:13.721492 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:46:13.820115 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:13.820030 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:46:14.033127 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:14.033097 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:46:14.193940 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:14.193852 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:46:14.286749 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:14.286707 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:46:15.621848 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:15.621793 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx" Jul 30 00:46:15.672131 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:15.672098 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff35d36a-d137-457e-8dfc-8110aa63cf92-istio-data\") pod \"ff35d36a-d137-457e-8dfc-8110aa63cf92\" (UID: \"ff35d36a-d137-457e-8dfc-8110aa63cf92\") " Jul 30 00:46:15.672293 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:15.672138 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff35d36a-d137-457e-8dfc-8110aa63cf92-workload-certs\") pod \"ff35d36a-d137-457e-8dfc-8110aa63cf92\" (UID: \"ff35d36a-d137-457e-8dfc-8110aa63cf92\") " Jul 30 00:46:15.672293 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:15.672169 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff35d36a-d137-457e-8dfc-8110aa63cf92-istio-podinfo\") pod \"ff35d36a-d137-457e-8dfc-8110aa63cf92\" (UID: \"ff35d36a-d137-457e-8dfc-8110aa63cf92\") " Jul 30 00:46:15.672293 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:15.672224 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff35d36a-d137-457e-8dfc-8110aa63cf92-credential-socket\") pod \"ff35d36a-d137-457e-8dfc-8110aa63cf92\" (UID: \"ff35d36a-d137-457e-8dfc-8110aa63cf92\") " Jul 30 00:46:15.672293 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:15.672286 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff35d36a-d137-457e-8dfc-8110aa63cf92-istio-token\") pod \"ff35d36a-d137-457e-8dfc-8110aa63cf92\" (UID: \"ff35d36a-d137-457e-8dfc-8110aa63cf92\") " Jul 30 00:46:15.672501 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:15.672314 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff35d36a-d137-457e-8dfc-8110aa63cf92-workload-socket\") pod \"ff35d36a-d137-457e-8dfc-8110aa63cf92\" (UID: \"ff35d36a-d137-457e-8dfc-8110aa63cf92\") " Jul 30 00:46:15.672501 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:15.672352 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l5hfk\" (UniqueName: \"kubernetes.io/projected/ff35d36a-d137-457e-8dfc-8110aa63cf92-kube-api-access-l5hfk\") pod \"ff35d36a-d137-457e-8dfc-8110aa63cf92\" (UID: \"ff35d36a-d137-457e-8dfc-8110aa63cf92\") " Jul 30 00:46:15.672501 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:15.672391 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff35d36a-d137-457e-8dfc-8110aa63cf92-istiod-ca-cert\") pod \"ff35d36a-d137-457e-8dfc-8110aa63cf92\" (UID: \"ff35d36a-d137-457e-8dfc-8110aa63cf92\") " Jul 30 00:46:15.672501 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:15.672415 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff35d36a-d137-457e-8dfc-8110aa63cf92-istio-envoy\") pod \"ff35d36a-d137-457e-8dfc-8110aa63cf92\" (UID: \"ff35d36a-d137-457e-8dfc-8110aa63cf92\") " Jul 30 00:46:15.672501 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:15.672452 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff35d36a-d137-457e-8dfc-8110aa63cf92-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ff35d36a-d137-457e-8dfc-8110aa63cf92" (UID: "ff35d36a-d137-457e-8dfc-8110aa63cf92"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:46:15.672783 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:15.672656 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff35d36a-d137-457e-8dfc-8110aa63cf92-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ff35d36a-d137-457e-8dfc-8110aa63cf92" (UID: "ff35d36a-d137-457e-8dfc-8110aa63cf92"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:46:15.672859 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:15.672827 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff35d36a-d137-457e-8dfc-8110aa63cf92-workload-socket\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:46:15.672859 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:15.672850 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff35d36a-d137-457e-8dfc-8110aa63cf92-workload-certs\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:46:15.672964 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:15.672826 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff35d36a-d137-457e-8dfc-8110aa63cf92-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ff35d36a-d137-457e-8dfc-8110aa63cf92" (UID: "ff35d36a-d137-457e-8dfc-8110aa63cf92"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:46:15.673172 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:15.673138 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff35d36a-d137-457e-8dfc-8110aa63cf92-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ff35d36a-d137-457e-8dfc-8110aa63cf92" (UID: "ff35d36a-d137-457e-8dfc-8110aa63cf92"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 00:46:15.673172 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:15.673157 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff35d36a-d137-457e-8dfc-8110aa63cf92-istio-data" (OuterVolumeSpecName: "istio-data") pod "ff35d36a-d137-457e-8dfc-8110aa63cf92" (UID: "ff35d36a-d137-457e-8dfc-8110aa63cf92"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:46:15.674870 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:15.674846 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ff35d36a-d137-457e-8dfc-8110aa63cf92-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ff35d36a-d137-457e-8dfc-8110aa63cf92" (UID: "ff35d36a-d137-457e-8dfc-8110aa63cf92"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 30 00:46:15.674870 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:15.674864 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff35d36a-d137-457e-8dfc-8110aa63cf92-kube-api-access-l5hfk" (OuterVolumeSpecName: "kube-api-access-l5hfk") pod "ff35d36a-d137-457e-8dfc-8110aa63cf92" (UID: "ff35d36a-d137-457e-8dfc-8110aa63cf92"). InnerVolumeSpecName "kube-api-access-l5hfk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 00:46:15.675026 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:15.674903 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff35d36a-d137-457e-8dfc-8110aa63cf92-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ff35d36a-d137-457e-8dfc-8110aa63cf92" (UID: "ff35d36a-d137-457e-8dfc-8110aa63cf92"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:46:15.675094 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:15.675072 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff35d36a-d137-457e-8dfc-8110aa63cf92-istio-token" (OuterVolumeSpecName: "istio-token") pod "ff35d36a-d137-457e-8dfc-8110aa63cf92" (UID: "ff35d36a-d137-457e-8dfc-8110aa63cf92"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 00:46:15.774287 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:15.774244 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff35d36a-d137-457e-8dfc-8110aa63cf92-istio-data\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:46:15.774287 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:15.774286 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff35d36a-d137-457e-8dfc-8110aa63cf92-istio-podinfo\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:46:15.774485 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:15.774302 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff35d36a-d137-457e-8dfc-8110aa63cf92-credential-socket\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:46:15.774485 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:15.774317 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff35d36a-d137-457e-8dfc-8110aa63cf92-istio-token\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:46:15.774485 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:15.774334 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l5hfk\" (UniqueName: \"kubernetes.io/projected/ff35d36a-d137-457e-8dfc-8110aa63cf92-kube-api-access-l5hfk\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:46:15.774485 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:15.774348 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff35d36a-d137-457e-8dfc-8110aa63cf92-istiod-ca-cert\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:46:15.774485 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:15.774363 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff35d36a-d137-457e-8dfc-8110aa63cf92-istio-envoy\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:46:16.145664 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:16.145624 2563 generic.go:356] "Generic (PLEG): container finished" podID="ff35d36a-d137-457e-8dfc-8110aa63cf92" containerID="352cf374b489f08a73a34fb3c09b2ac2fdc79b037b1634b11ed5d124c8debe24" exitCode=0 Jul 30 00:46:16.145879 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:16.145666 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx" event={"ID":"ff35d36a-d137-457e-8dfc-8110aa63cf92","Type":"ContainerDied","Data":"352cf374b489f08a73a34fb3c09b2ac2fdc79b037b1634b11ed5d124c8debe24"} Jul 30 00:46:16.145879 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:16.145724 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx" event={"ID":"ff35d36a-d137-457e-8dfc-8110aa63cf92","Type":"ContainerDied","Data":"37b8c3454df399efb625d8332f1d7cc2fff599cb11b8344fab3769cd7feff375"} Jul 30 00:46:16.145879 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:16.145679 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx" Jul 30 00:46:16.145879 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:16.145769 2563 scope.go:117] "RemoveContainer" containerID="352cf374b489f08a73a34fb3c09b2ac2fdc79b037b1634b11ed5d124c8debe24" Jul 30 00:46:16.159459 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:16.159438 2563 scope.go:117] "RemoveContainer" containerID="ad373f0699e5f2a995c1105a91936a83ee6f40da6225b57056f78d2128aefcd6" Jul 30 00:46:16.169256 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:16.169233 2563 scope.go:117] "RemoveContainer" containerID="352cf374b489f08a73a34fb3c09b2ac2fdc79b037b1634b11ed5d124c8debe24" Jul 30 00:46:16.169522 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:46:16.169503 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"352cf374b489f08a73a34fb3c09b2ac2fdc79b037b1634b11ed5d124c8debe24\": container with ID starting with 352cf374b489f08a73a34fb3c09b2ac2fdc79b037b1634b11ed5d124c8debe24 not found: ID does not exist" containerID="352cf374b489f08a73a34fb3c09b2ac2fdc79b037b1634b11ed5d124c8debe24" Jul 30 00:46:16.169573 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:16.169555 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"352cf374b489f08a73a34fb3c09b2ac2fdc79b037b1634b11ed5d124c8debe24"} err="failed to get container status \"352cf374b489f08a73a34fb3c09b2ac2fdc79b037b1634b11ed5d124c8debe24\": rpc error: code = NotFound desc = could not find container \"352cf374b489f08a73a34fb3c09b2ac2fdc79b037b1634b11ed5d124c8debe24\": container with ID starting with 352cf374b489f08a73a34fb3c09b2ac2fdc79b037b1634b11ed5d124c8debe24 not found: ID does not exist" Jul 30 00:46:16.169623 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:16.169576 2563 scope.go:117] "RemoveContainer" containerID="ad373f0699e5f2a995c1105a91936a83ee6f40da6225b57056f78d2128aefcd6" Jul 30 00:46:16.169829 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:46:16.169804 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad373f0699e5f2a995c1105a91936a83ee6f40da6225b57056f78d2128aefcd6\": container with ID starting with ad373f0699e5f2a995c1105a91936a83ee6f40da6225b57056f78d2128aefcd6 not found: ID does not exist" containerID="ad373f0699e5f2a995c1105a91936a83ee6f40da6225b57056f78d2128aefcd6" Jul 30 00:46:16.169871 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:16.169838 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad373f0699e5f2a995c1105a91936a83ee6f40da6225b57056f78d2128aefcd6"} err="failed to get container status \"ad373f0699e5f2a995c1105a91936a83ee6f40da6225b57056f78d2128aefcd6\": rpc error: code = NotFound desc = could not find container \"ad373f0699e5f2a995c1105a91936a83ee6f40da6225b57056f78d2128aefcd6\": container with ID starting with ad373f0699e5f2a995c1105a91936a83ee6f40da6225b57056f78d2128aefcd6 not found: ID does not exist" Jul 30 00:46:16.178374 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:16.178348 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx"] Jul 30 00:46:16.182174 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:16.182152 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-a5930c62-openshift-default-85f76b8998-w6bmx"] Jul 30 00:46:16.477325 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:16.477231 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff35d36a-d137-457e-8dfc-8110aa63cf92" path="/var/lib/kubelet/pods/ff35d36a-d137-457e-8dfc-8110aa63cf92/volumes" Jul 30 00:46:28.132021 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:28.131932 2563 patch_prober.go:28] interesting pod/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 30 00:46:28.132021 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:28.132006 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk" podUID="96eb3aeb-161b-4bd6-a26e-659d1e2dd215" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 30 00:46:28.213727 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:28.213692 2563 generic.go:356] "Generic (PLEG): container finished" podID="96eb3aeb-161b-4bd6-a26e-659d1e2dd215" containerID="1c2e7f2add896d824f4a76fa0a3d28b5f22d8671c7f8c23563d5a4ddfd8c79ad" exitCode=0 Jul 30 00:46:28.213919 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:28.213766 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk" event={"ID":"96eb3aeb-161b-4bd6-a26e-659d1e2dd215","Type":"ContainerDied","Data":"1c2e7f2add896d824f4a76fa0a3d28b5f22d8671c7f8c23563d5a4ddfd8c79ad"} Jul 30 00:46:28.214208 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:28.214191 2563 scope.go:117] "RemoveContainer" containerID="1c2e7f2add896d824f4a76fa0a3d28b5f22d8671c7f8c23563d5a4ddfd8c79ad" Jul 30 00:46:28.215192 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:28.215167 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:46:28.215244 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:28.215235 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:46:28.215283 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:28.215266 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:46:29.220840 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:29.220803 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk" event={"ID":"96eb3aeb-161b-4bd6-a26e-659d1e2dd215","Type":"ContainerStarted","Data":"6a4cbc93afdf98e5203f0eae94e711acb6761c711048ed1719f0810d89a0295a"} Jul 30 00:46:29.594440 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:29.594398 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk" Jul 30 00:46:29.594678 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:29.594656 2563 patch_prober.go:28] interesting pod/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 30 00:46:29.594726 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:29.594703 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk" podUID="96eb3aeb-161b-4bd6-a26e-659d1e2dd215" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 30 00:46:30.594844 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:30.594803 2563 patch_prober.go:28] interesting pod/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 30 00:46:30.595260 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:30.594879 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk" podUID="96eb3aeb-161b-4bd6-a26e-659d1e2dd215" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 30 00:46:31.594000 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:31.593957 2563 patch_prober.go:28] interesting pod/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 30 00:46:31.594176 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:31.594050 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk" podUID="96eb3aeb-161b-4bd6-a26e-659d1e2dd215" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 30 00:46:32.594187 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:32.594147 2563 patch_prober.go:28] interesting pod/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 30 00:46:32.594592 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:32.594243 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk" podUID="96eb3aeb-161b-4bd6-a26e-659d1e2dd215" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 30 00:46:33.598671 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:33.598639 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk" Jul 30 00:46:33.599086 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:33.598938 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk" Jul 30 00:46:33.599571 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:33.599554 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk" Jul 30 00:46:44.989906 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:44.989875 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:46:45.062106 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:45.062072 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:46:45.164139 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:45.164104 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:46:45.221135 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:45.221102 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:46:45.276585 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:45.276551 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk"] Jul 30 00:46:45.276834 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:45.276792 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk" podUID="96eb3aeb-161b-4bd6-a26e-659d1e2dd215" containerName="istio-proxy" containerID="cri-o://6a4cbc93afdf98e5203f0eae94e711acb6761c711048ed1719f0810d89a0295a" gracePeriod=30 Jul 30 00:46:45.300155 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:45.300126 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:46:45.514037 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:45.513999 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:46:45.730777 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:45.730655 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:46:45.994511 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:45.994424 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9"] Jul 30 00:46:45.994922 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:45.994906 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff35d36a-d137-457e-8dfc-8110aa63cf92" containerName="istio-proxy" Jul 30 00:46:45.994967 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:45.994924 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff35d36a-d137-457e-8dfc-8110aa63cf92" containerName="istio-proxy" Jul 30 00:46:45.994967 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:45.994936 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff35d36a-d137-457e-8dfc-8110aa63cf92" containerName="istio-proxy" Jul 30 00:46:45.994967 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:45.994941 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff35d36a-d137-457e-8dfc-8110aa63cf92" containerName="istio-proxy" Jul 30 00:46:45.995069 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:45.995012 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff35d36a-d137-457e-8dfc-8110aa63cf92" containerName="istio-proxy" Jul 30 00:46:45.995069 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:45.995024 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff35d36a-d137-457e-8dfc-8110aa63cf92" containerName="istio-proxy" Jul 30 00:46:45.998481 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:45.998462 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9" Jul 30 00:46:46.001497 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:46.001472 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-5213aadc-openshift-default-dockercfg-zwpt9\"" Jul 30 00:46:46.009118 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:46.009092 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9"] Jul 30 00:46:46.022254 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:46.022218 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:46:46.165869 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:46.165835 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f6736bc-86b8-4799-a5ac-cf21540272fa-istio-token\") pod \"e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9\" (UID: \"9f6736bc-86b8-4799-a5ac-cf21540272fa\") " pod="openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9" Jul 30 00:46:46.165869 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:46.165878 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f6736bc-86b8-4799-a5ac-cf21540272fa-workload-socket\") pod \"e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9\" (UID: \"9f6736bc-86b8-4799-a5ac-cf21540272fa\") " pod="openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9" Jul 30 00:46:46.166135 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:46.165897 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f6736bc-86b8-4799-a5ac-cf21540272fa-workload-certs\") pod \"e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9\" (UID: \"9f6736bc-86b8-4799-a5ac-cf21540272fa\") " pod="openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9" Jul 30 00:46:46.166135 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:46.165916 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f6736bc-86b8-4799-a5ac-cf21540272fa-istio-data\") pod \"e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9\" (UID: \"9f6736bc-86b8-4799-a5ac-cf21540272fa\") " pod="openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9" Jul 30 00:46:46.166135 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:46.166050 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f6736bc-86b8-4799-a5ac-cf21540272fa-credential-socket\") pod \"e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9\" (UID: \"9f6736bc-86b8-4799-a5ac-cf21540272fa\") " pod="openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9" Jul 30 00:46:46.166135 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:46.166104 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f6736bc-86b8-4799-a5ac-cf21540272fa-istiod-ca-cert\") pod \"e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9\" (UID: \"9f6736bc-86b8-4799-a5ac-cf21540272fa\") " pod="openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9" Jul 30 00:46:46.166347 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:46.166158 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qllvv\" (UniqueName: \"kubernetes.io/projected/9f6736bc-86b8-4799-a5ac-cf21540272fa-kube-api-access-qllvv\") pod \"e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9\" (UID: \"9f6736bc-86b8-4799-a5ac-cf21540272fa\") " pod="openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9" Jul 30 00:46:46.166347 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:46.166185 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f6736bc-86b8-4799-a5ac-cf21540272fa-istio-envoy\") pod \"e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9\" (UID: \"9f6736bc-86b8-4799-a5ac-cf21540272fa\") " pod="openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9" Jul 30 00:46:46.166347 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:46.166208 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f6736bc-86b8-4799-a5ac-cf21540272fa-istio-podinfo\") pod \"e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9\" (UID: \"9f6736bc-86b8-4799-a5ac-cf21540272fa\") " pod="openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9" Jul 30 00:46:46.267040 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:46.267006 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f6736bc-86b8-4799-a5ac-cf21540272fa-istio-podinfo\") pod \"e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9\" (UID: \"9f6736bc-86b8-4799-a5ac-cf21540272fa\") " pod="openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9" Jul 30 00:46:46.267227 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:46.267049 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f6736bc-86b8-4799-a5ac-cf21540272fa-istio-token\") pod \"e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9\" (UID: \"9f6736bc-86b8-4799-a5ac-cf21540272fa\") " pod="openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9" Jul 30 00:46:46.267227 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:46.267079 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f6736bc-86b8-4799-a5ac-cf21540272fa-workload-socket\") pod \"e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9\" (UID: \"9f6736bc-86b8-4799-a5ac-cf21540272fa\") " pod="openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9" Jul 30 00:46:46.267227 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:46.267103 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f6736bc-86b8-4799-a5ac-cf21540272fa-workload-certs\") pod \"e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9\" (UID: \"9f6736bc-86b8-4799-a5ac-cf21540272fa\") " pod="openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9" Jul 30 00:46:46.267227 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:46.267126 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f6736bc-86b8-4799-a5ac-cf21540272fa-istio-data\") pod \"e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9\" (UID: \"9f6736bc-86b8-4799-a5ac-cf21540272fa\") " pod="openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9" Jul 30 00:46:46.267227 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:46.267199 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f6736bc-86b8-4799-a5ac-cf21540272fa-credential-socket\") pod \"e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9\" (UID: \"9f6736bc-86b8-4799-a5ac-cf21540272fa\") " pod="openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9" Jul 30 00:46:46.267504 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:46.267250 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f6736bc-86b8-4799-a5ac-cf21540272fa-istiod-ca-cert\") pod \"e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9\" (UID: \"9f6736bc-86b8-4799-a5ac-cf21540272fa\") " pod="openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9" Jul 30 00:46:46.267504 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:46.267302 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qllvv\" (UniqueName: \"kubernetes.io/projected/9f6736bc-86b8-4799-a5ac-cf21540272fa-kube-api-access-qllvv\") pod \"e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9\" (UID: \"9f6736bc-86b8-4799-a5ac-cf21540272fa\") " pod="openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9" Jul 30 00:46:46.267504 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:46.267331 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f6736bc-86b8-4799-a5ac-cf21540272fa-istio-envoy\") pod \"e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9\" (UID: \"9f6736bc-86b8-4799-a5ac-cf21540272fa\") " pod="openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9" Jul 30 00:46:46.267660 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:46.267597 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f6736bc-86b8-4799-a5ac-cf21540272fa-istio-data\") pod \"e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9\" (UID: \"9f6736bc-86b8-4799-a5ac-cf21540272fa\") " pod="openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9" Jul 30 00:46:46.267879 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:46.267850 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f6736bc-86b8-4799-a5ac-cf21540272fa-workload-socket\") pod \"e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9\" (UID: \"9f6736bc-86b8-4799-a5ac-cf21540272fa\") " pod="openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9" Jul 30 00:46:46.268004 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:46.267890 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f6736bc-86b8-4799-a5ac-cf21540272fa-workload-certs\") pod \"e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9\" (UID: \"9f6736bc-86b8-4799-a5ac-cf21540272fa\") " pod="openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9" Jul 30 00:46:46.268107 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:46.268087 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f6736bc-86b8-4799-a5ac-cf21540272fa-credential-socket\") pod \"e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9\" (UID: \"9f6736bc-86b8-4799-a5ac-cf21540272fa\") " pod="openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9" Jul 30 00:46:46.268447 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:46.268425 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f6736bc-86b8-4799-a5ac-cf21540272fa-istiod-ca-cert\") pod \"e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9\" (UID: \"9f6736bc-86b8-4799-a5ac-cf21540272fa\") " pod="openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9" Jul 30 00:46:46.269815 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:46.269794 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f6736bc-86b8-4799-a5ac-cf21540272fa-istio-envoy\") pod \"e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9\" (UID: \"9f6736bc-86b8-4799-a5ac-cf21540272fa\") " pod="openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9" Jul 30 00:46:46.270130 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:46.270106 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f6736bc-86b8-4799-a5ac-cf21540272fa-istio-podinfo\") pod \"e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9\" (UID: \"9f6736bc-86b8-4799-a5ac-cf21540272fa\") " pod="openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9" Jul 30 00:46:46.276136 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:46.276112 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f6736bc-86b8-4799-a5ac-cf21540272fa-istio-token\") pod \"e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9\" (UID: \"9f6736bc-86b8-4799-a5ac-cf21540272fa\") " pod="openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9" Jul 30 00:46:46.276484 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:46.276464 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qllvv\" (UniqueName: \"kubernetes.io/projected/9f6736bc-86b8-4799-a5ac-cf21540272fa-kube-api-access-qllvv\") pod \"e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9\" (UID: \"9f6736bc-86b8-4799-a5ac-cf21540272fa\") " pod="openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9" Jul 30 00:46:46.312543 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:46.312512 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9" Jul 30 00:46:46.493894 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:46.493866 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9"] Jul 30 00:46:46.496150 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:46:46.496117 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f6736bc_86b8_4799_a5ac_cf21540272fa.slice/crio-4af92fea75374e3310a3e3d52f8543d05238b09cde8f120b679d28323d53e3e5 WatchSource:0}: Error finding container 4af92fea75374e3310a3e3d52f8543d05238b09cde8f120b679d28323d53e3e5: Status 404 returned error can't find the container with id 4af92fea75374e3310a3e3d52f8543d05238b09cde8f120b679d28323d53e3e5 Jul 30 00:46:46.498505 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:46.498466 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:46:46.498615 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:46.498544 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:46:46.498615 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:46.498575 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:46:46.663009 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:46.662962 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:46:47.315797 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:47.315729 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9" event={"ID":"9f6736bc-86b8-4799-a5ac-cf21540272fa","Type":"ContainerStarted","Data":"1d1df52901436ca5e9eafdc8d6e2eaa78d5f65a8f2432afe3d64b9223c402554"} Jul 30 00:46:47.316185 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:47.315823 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9" event={"ID":"9f6736bc-86b8-4799-a5ac-cf21540272fa","Type":"ContainerStarted","Data":"4af92fea75374e3310a3e3d52f8543d05238b09cde8f120b679d28323d53e3e5"} Jul 30 00:46:47.343260 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:47.343203 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9" podStartSLOduration=2.343187183 podStartE2EDuration="2.343187183s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:46:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:46:47.341628672 +0000 UTC m=+1771.400384670" watchObservedRunningTime="2026-07-30 00:46:47.343187183 +0000 UTC m=+1771.401943150" Jul 30 00:46:48.312854 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:48.312814 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9" Jul 30 00:46:48.317981 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:48.317942 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9" Jul 30 00:46:48.321537 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:48.321513 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9" Jul 30 00:46:48.322372 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:48.322351 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9" Jul 30 00:46:48.601706 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:48.601619 2563 patch_prober.go:28] interesting pod/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused Jul 30 00:46:48.601706 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:48.601685 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk" podUID="96eb3aeb-161b-4bd6-a26e-659d1e2dd215" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 30 00:46:48.669888 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:48.669848 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:46:48.789879 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:48.789824 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:46:49.915405 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:49.915375 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:46:50.140429 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:50.140384 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:46:50.441576 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:50.441520 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:46:50.532666 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:50.532634 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:46:50.543367 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:50.543309 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk" Jul 30 00:46:50.608516 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:50.608483 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96eb3aeb-161b-4bd6-a26e-659d1e2dd215-credential-socket\") pod \"96eb3aeb-161b-4bd6-a26e-659d1e2dd215\" (UID: \"96eb3aeb-161b-4bd6-a26e-659d1e2dd215\") " Jul 30 00:46:50.608710 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:50.608558 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96eb3aeb-161b-4bd6-a26e-659d1e2dd215-istio-data\") pod \"96eb3aeb-161b-4bd6-a26e-659d1e2dd215\" (UID: \"96eb3aeb-161b-4bd6-a26e-659d1e2dd215\") " Jul 30 00:46:50.608710 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:50.608591 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96eb3aeb-161b-4bd6-a26e-659d1e2dd215-istio-token\") pod \"96eb3aeb-161b-4bd6-a26e-659d1e2dd215\" (UID: \"96eb3aeb-161b-4bd6-a26e-659d1e2dd215\") " Jul 30 00:46:50.608710 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:50.608640 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96eb3aeb-161b-4bd6-a26e-659d1e2dd215-workload-certs\") pod \"96eb3aeb-161b-4bd6-a26e-659d1e2dd215\" (UID: \"96eb3aeb-161b-4bd6-a26e-659d1e2dd215\") " Jul 30 00:46:50.608710 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:50.608669 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96eb3aeb-161b-4bd6-a26e-659d1e2dd215-workload-socket\") pod \"96eb3aeb-161b-4bd6-a26e-659d1e2dd215\" (UID: \"96eb3aeb-161b-4bd6-a26e-659d1e2dd215\") " Jul 30 00:46:50.608971 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:50.608725 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v4s5c\" (UniqueName: \"kubernetes.io/projected/96eb3aeb-161b-4bd6-a26e-659d1e2dd215-kube-api-access-v4s5c\") pod \"96eb3aeb-161b-4bd6-a26e-659d1e2dd215\" (UID: \"96eb3aeb-161b-4bd6-a26e-659d1e2dd215\") " Jul 30 00:46:50.608971 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:50.608807 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96eb3aeb-161b-4bd6-a26e-659d1e2dd215-istio-podinfo\") pod \"96eb3aeb-161b-4bd6-a26e-659d1e2dd215\" (UID: \"96eb3aeb-161b-4bd6-a26e-659d1e2dd215\") " Jul 30 00:46:50.608971 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:50.608842 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96eb3aeb-161b-4bd6-a26e-659d1e2dd215-istiod-ca-cert\") pod \"96eb3aeb-161b-4bd6-a26e-659d1e2dd215\" (UID: \"96eb3aeb-161b-4bd6-a26e-659d1e2dd215\") " Jul 30 00:46:50.608971 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:50.608879 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96eb3aeb-161b-4bd6-a26e-659d1e2dd215-istio-envoy\") pod \"96eb3aeb-161b-4bd6-a26e-659d1e2dd215\" (UID: \"96eb3aeb-161b-4bd6-a26e-659d1e2dd215\") " Jul 30 00:46:50.608971 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:50.608936 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96eb3aeb-161b-4bd6-a26e-659d1e2dd215-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "96eb3aeb-161b-4bd6-a26e-659d1e2dd215" (UID: "96eb3aeb-161b-4bd6-a26e-659d1e2dd215"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:46:50.608971 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:50.608953 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96eb3aeb-161b-4bd6-a26e-659d1e2dd215-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "96eb3aeb-161b-4bd6-a26e-659d1e2dd215" (UID: "96eb3aeb-161b-4bd6-a26e-659d1e2dd215"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:46:50.609271 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:50.609060 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96eb3aeb-161b-4bd6-a26e-659d1e2dd215-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "96eb3aeb-161b-4bd6-a26e-659d1e2dd215" (UID: "96eb3aeb-161b-4bd6-a26e-659d1e2dd215"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:46:50.609336 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:50.609295 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96eb3aeb-161b-4bd6-a26e-659d1e2dd215-workload-socket\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:46:50.609336 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:50.609316 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96eb3aeb-161b-4bd6-a26e-659d1e2dd215-credential-socket\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:46:50.609336 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:50.609332 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96eb3aeb-161b-4bd6-a26e-659d1e2dd215-workload-certs\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:46:50.609482 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:50.609397 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96eb3aeb-161b-4bd6-a26e-659d1e2dd215-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "96eb3aeb-161b-4bd6-a26e-659d1e2dd215" (UID: "96eb3aeb-161b-4bd6-a26e-659d1e2dd215"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 00:46:50.609773 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:50.609724 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96eb3aeb-161b-4bd6-a26e-659d1e2dd215-istio-data" (OuterVolumeSpecName: "istio-data") pod "96eb3aeb-161b-4bd6-a26e-659d1e2dd215" (UID: "96eb3aeb-161b-4bd6-a26e-659d1e2dd215"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:46:50.611123 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:50.611044 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96eb3aeb-161b-4bd6-a26e-659d1e2dd215-istio-token" (OuterVolumeSpecName: "istio-token") pod "96eb3aeb-161b-4bd6-a26e-659d1e2dd215" (UID: "96eb3aeb-161b-4bd6-a26e-659d1e2dd215"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 00:46:50.611231 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:50.611163 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96eb3aeb-161b-4bd6-a26e-659d1e2dd215-kube-api-access-v4s5c" (OuterVolumeSpecName: "kube-api-access-v4s5c") pod "96eb3aeb-161b-4bd6-a26e-659d1e2dd215" (UID: "96eb3aeb-161b-4bd6-a26e-659d1e2dd215"). InnerVolumeSpecName "kube-api-access-v4s5c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 00:46:50.611383 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:50.611357 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96eb3aeb-161b-4bd6-a26e-659d1e2dd215-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "96eb3aeb-161b-4bd6-a26e-659d1e2dd215" (UID: "96eb3aeb-161b-4bd6-a26e-659d1e2dd215"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:46:50.611512 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:50.611433 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/96eb3aeb-161b-4bd6-a26e-659d1e2dd215-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "96eb3aeb-161b-4bd6-a26e-659d1e2dd215" (UID: "96eb3aeb-161b-4bd6-a26e-659d1e2dd215"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 30 00:46:50.710417 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:50.710386 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96eb3aeb-161b-4bd6-a26e-659d1e2dd215-istio-data\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:46:50.710417 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:50.710413 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96eb3aeb-161b-4bd6-a26e-659d1e2dd215-istio-token\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:46:50.710417 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:50.710422 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v4s5c\" (UniqueName: \"kubernetes.io/projected/96eb3aeb-161b-4bd6-a26e-659d1e2dd215-kube-api-access-v4s5c\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:46:50.710649 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:50.710430 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96eb3aeb-161b-4bd6-a26e-659d1e2dd215-istio-podinfo\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:46:50.710649 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:50.710441 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96eb3aeb-161b-4bd6-a26e-659d1e2dd215-istiod-ca-cert\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:46:50.710649 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:50.710449 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96eb3aeb-161b-4bd6-a26e-659d1e2dd215-istio-envoy\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:46:51.337355 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:51.337317 2563 generic.go:356] "Generic (PLEG): container finished" podID="96eb3aeb-161b-4bd6-a26e-659d1e2dd215" containerID="6a4cbc93afdf98e5203f0eae94e711acb6761c711048ed1719f0810d89a0295a" exitCode=0 Jul 30 00:46:51.337776 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:51.337369 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk" Jul 30 00:46:51.337776 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:51.337397 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk" event={"ID":"96eb3aeb-161b-4bd6-a26e-659d1e2dd215","Type":"ContainerDied","Data":"6a4cbc93afdf98e5203f0eae94e711acb6761c711048ed1719f0810d89a0295a"} Jul 30 00:46:51.337776 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:51.337449 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk" event={"ID":"96eb3aeb-161b-4bd6-a26e-659d1e2dd215","Type":"ContainerDied","Data":"1d8c3b6f38afd473e4de48a71b950e67d83ac6d2b214db1d694bedaa31d6380d"} Jul 30 00:46:51.337776 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:51.337473 2563 scope.go:117] "RemoveContainer" containerID="6a4cbc93afdf98e5203f0eae94e711acb6761c711048ed1719f0810d89a0295a" Jul 30 00:46:51.346686 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:51.346667 2563 scope.go:117] "RemoveContainer" containerID="1c2e7f2add896d824f4a76fa0a3d28b5f22d8671c7f8c23563d5a4ddfd8c79ad" Jul 30 00:46:51.354710 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:51.354686 2563 scope.go:117] "RemoveContainer" containerID="6a4cbc93afdf98e5203f0eae94e711acb6761c711048ed1719f0810d89a0295a" Jul 30 00:46:51.355009 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:46:51.354991 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a4cbc93afdf98e5203f0eae94e711acb6761c711048ed1719f0810d89a0295a\": container with ID starting with 6a4cbc93afdf98e5203f0eae94e711acb6761c711048ed1719f0810d89a0295a not found: ID does not exist" containerID="6a4cbc93afdf98e5203f0eae94e711acb6761c711048ed1719f0810d89a0295a" Jul 30 00:46:51.355088 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:51.355022 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a4cbc93afdf98e5203f0eae94e711acb6761c711048ed1719f0810d89a0295a"} err="failed to get container status \"6a4cbc93afdf98e5203f0eae94e711acb6761c711048ed1719f0810d89a0295a\": rpc error: code = NotFound desc = could not find container \"6a4cbc93afdf98e5203f0eae94e711acb6761c711048ed1719f0810d89a0295a\": container with ID starting with 6a4cbc93afdf98e5203f0eae94e711acb6761c711048ed1719f0810d89a0295a not found: ID does not exist" Jul 30 00:46:51.355088 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:51.355045 2563 scope.go:117] "RemoveContainer" containerID="1c2e7f2add896d824f4a76fa0a3d28b5f22d8671c7f8c23563d5a4ddfd8c79ad" Jul 30 00:46:51.355294 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:46:51.355278 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c2e7f2add896d824f4a76fa0a3d28b5f22d8671c7f8c23563d5a4ddfd8c79ad\": container with ID starting with 1c2e7f2add896d824f4a76fa0a3d28b5f22d8671c7f8c23563d5a4ddfd8c79ad not found: ID does not exist" containerID="1c2e7f2add896d824f4a76fa0a3d28b5f22d8671c7f8c23563d5a4ddfd8c79ad" Jul 30 00:46:51.355348 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:51.355300 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c2e7f2add896d824f4a76fa0a3d28b5f22d8671c7f8c23563d5a4ddfd8c79ad"} err="failed to get container status \"1c2e7f2add896d824f4a76fa0a3d28b5f22d8671c7f8c23563d5a4ddfd8c79ad\": rpc error: code = NotFound desc = could not find container \"1c2e7f2add896d824f4a76fa0a3d28b5f22d8671c7f8c23563d5a4ddfd8c79ad\": container with ID starting with 1c2e7f2add896d824f4a76fa0a3d28b5f22d8671c7f8c23563d5a4ddfd8c79ad not found: ID does not exist" Jul 30 00:46:51.358499 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:51.358473 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk"] Jul 30 00:46:51.362902 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:51.362870 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-837664ed-gw-openshift-default-84f6fbc484-nx8zk"] Jul 30 00:46:52.475295 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:46:52.475258 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96eb3aeb-161b-4bd6-a26e-659d1e2dd215" path="/var/lib/kubelet/pods/96eb3aeb-161b-4bd6-a26e-659d1e2dd215/volumes" Jul 30 00:47:03.323497 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:03.323459 2563 patch_prober.go:28] interesting pod/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 30 00:47:03.323925 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:03.323527 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9" podUID="9f6736bc-86b8-4799-a5ac-cf21540272fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 30 00:47:03.407358 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:03.407319 2563 generic.go:356] "Generic (PLEG): container finished" podID="9f6736bc-86b8-4799-a5ac-cf21540272fa" containerID="1d1df52901436ca5e9eafdc8d6e2eaa78d5f65a8f2432afe3d64b9223c402554" exitCode=0 Jul 30 00:47:03.407529 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:03.407391 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9" event={"ID":"9f6736bc-86b8-4799-a5ac-cf21540272fa","Type":"ContainerDied","Data":"1d1df52901436ca5e9eafdc8d6e2eaa78d5f65a8f2432afe3d64b9223c402554"} Jul 30 00:47:03.407833 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:03.407817 2563 scope.go:117] "RemoveContainer" containerID="1d1df52901436ca5e9eafdc8d6e2eaa78d5f65a8f2432afe3d64b9223c402554" Jul 30 00:47:03.408832 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:03.408806 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:47:03.408930 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:03.408869 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:47:03.408930 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:03.408898 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:47:04.413853 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:04.413819 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9" event={"ID":"9f6736bc-86b8-4799-a5ac-cf21540272fa","Type":"ContainerStarted","Data":"c711c7625cd1db709049849707d041504bceb55a3673e3a7810f2f5903e124ee"} Jul 30 00:47:05.313221 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:05.313182 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9" Jul 30 00:47:05.313400 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:05.313359 2563 patch_prober.go:28] interesting pod/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 30 00:47:05.313445 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:05.313410 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9" podUID="9f6736bc-86b8-4799-a5ac-cf21540272fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 30 00:47:06.313634 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:06.313594 2563 patch_prober.go:28] interesting pod/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 30 00:47:06.314051 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:06.313659 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9" podUID="9f6736bc-86b8-4799-a5ac-cf21540272fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 30 00:47:07.313110 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:07.313071 2563 patch_prober.go:28] interesting pod/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 30 00:47:07.313297 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:07.313131 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9" podUID="9f6736bc-86b8-4799-a5ac-cf21540272fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 30 00:47:08.317361 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:08.317330 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9" Jul 30 00:47:08.317764 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:08.317583 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9" Jul 30 00:47:08.318167 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:08.318146 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9" Jul 30 00:47:16.508443 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:16.508417 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-b92tw_0ef8f120-f712-4ed9-adbd-40f785d4aa9d/console-operator/1.log" Jul 30 00:47:16.510271 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:16.510245 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-b92tw_0ef8f120-f712-4ed9-adbd-40f785d4aa9d/console-operator/1.log" Jul 30 00:47:20.376279 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:20.376241 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:47:20.440286 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:20.440252 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:47:20.496547 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:20.496514 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:47:20.552659 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:20.552628 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:47:21.584433 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:21.584392 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9"] Jul 30 00:47:21.584863 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:21.584639 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9" podUID="9f6736bc-86b8-4799-a5ac-cf21540272fa" containerName="istio-proxy" containerID="cri-o://c711c7625cd1db709049849707d041504bceb55a3673e3a7810f2f5903e124ee" gracePeriod=30 Jul 30 00:47:21.606212 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:21.606182 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:47:21.794131 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:21.794101 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:47:21.849344 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:21.849261 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:47:22.002876 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:22.002820 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:47:22.255729 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:22.255696 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk"] Jul 30 00:47:22.256323 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:22.256303 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96eb3aeb-161b-4bd6-a26e-659d1e2dd215" containerName="istio-proxy" Jul 30 00:47:22.256389 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:22.256327 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="96eb3aeb-161b-4bd6-a26e-659d1e2dd215" containerName="istio-proxy" Jul 30 00:47:22.256389 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:22.256353 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96eb3aeb-161b-4bd6-a26e-659d1e2dd215" containerName="istio-proxy" Jul 30 00:47:22.256389 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:22.256362 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="96eb3aeb-161b-4bd6-a26e-659d1e2dd215" containerName="istio-proxy" Jul 30 00:47:22.256505 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:22.256479 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="96eb3aeb-161b-4bd6-a26e-659d1e2dd215" containerName="istio-proxy" Jul 30 00:47:22.256505 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:22.256490 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="96eb3aeb-161b-4bd6-a26e-659d1e2dd215" containerName="istio-proxy" Jul 30 00:47:22.260150 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:22.260127 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk" Jul 30 00:47:22.262863 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:22.262798 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-06160795-openshift-default-dockercfg-wg2kx\"" Jul 30 00:47:22.272918 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:22.272893 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk"] Jul 30 00:47:22.286285 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:22.286263 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:47:22.385849 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:22.385817 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:47:22.413114 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:22.413077 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471-istio-podinfo\") pod \"e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk\" (UID: \"b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471\") " pod="openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk" Jul 30 00:47:22.413231 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:22.413130 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471-workload-certs\") pod \"e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk\" (UID: \"b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471\") " pod="openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk" Jul 30 00:47:22.413231 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:22.413199 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471-credential-socket\") pod \"e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk\" (UID: \"b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471\") " pod="openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk" Jul 30 00:47:22.413303 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:22.413240 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471-istiod-ca-cert\") pod \"e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk\" (UID: \"b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471\") " pod="openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk" Jul 30 00:47:22.413303 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:22.413266 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9h2j\" (UniqueName: \"kubernetes.io/projected/b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471-kube-api-access-l9h2j\") pod \"e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk\" (UID: \"b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471\") " pod="openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk" Jul 30 00:47:22.413373 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:22.413306 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471-istio-envoy\") pod \"e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk\" (UID: \"b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471\") " pod="openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk" Jul 30 00:47:22.413373 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:22.413350 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471-istio-data\") pod \"e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk\" (UID: \"b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471\") " pod="openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk" Jul 30 00:47:22.413373 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:22.413370 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471-istio-token\") pod \"e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk\" (UID: \"b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471\") " pod="openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk" Jul 30 00:47:22.413464 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:22.413401 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471-workload-socket\") pod \"e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk\" (UID: \"b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471\") " pod="openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk" Jul 30 00:47:22.514161 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:22.514069 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471-istio-podinfo\") pod \"e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk\" (UID: \"b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471\") " pod="openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk" Jul 30 00:47:22.514161 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:22.514115 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471-workload-certs\") pod \"e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk\" (UID: \"b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471\") " pod="openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk" Jul 30 00:47:22.514419 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:22.514164 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471-credential-socket\") pod \"e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk\" (UID: \"b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471\") " pod="openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk" Jul 30 00:47:22.514419 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:22.514200 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471-istiod-ca-cert\") pod \"e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk\" (UID: \"b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471\") " pod="openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk" Jul 30 00:47:22.514419 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:22.514235 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9h2j\" (UniqueName: \"kubernetes.io/projected/b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471-kube-api-access-l9h2j\") pod \"e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk\" (UID: \"b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471\") " pod="openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk" Jul 30 00:47:22.514419 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:22.514261 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471-istio-envoy\") pod \"e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk\" (UID: \"b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471\") " pod="openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk" Jul 30 00:47:22.514419 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:22.514309 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471-istio-data\") pod \"e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk\" (UID: \"b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471\") " pod="openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk" Jul 30 00:47:22.514419 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:22.514335 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471-istio-token\") pod \"e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk\" (UID: \"b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471\") " pod="openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk" Jul 30 00:47:22.514727 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:22.514530 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471-workload-socket\") pod \"e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk\" (UID: \"b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471\") " pod="openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk" Jul 30 00:47:22.514727 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:22.514594 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471-workload-certs\") pod \"e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk\" (UID: \"b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471\") " pod="openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk" Jul 30 00:47:22.514727 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:22.514693 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471-credential-socket\") pod \"e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk\" (UID: \"b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471\") " pod="openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk" Jul 30 00:47:22.514932 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:22.514883 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471-workload-socket\") pod \"e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk\" (UID: \"b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471\") " pod="openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk" Jul 30 00:47:22.514932 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:22.514921 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471-istio-data\") pod \"e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk\" (UID: \"b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471\") " pod="openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk" Jul 30 00:47:22.515050 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:22.515024 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471-istiod-ca-cert\") pod \"e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk\" (UID: \"b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471\") " pod="openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk" Jul 30 00:47:22.516573 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:22.516547 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471-istio-envoy\") pod \"e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk\" (UID: \"b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471\") " pod="openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk" Jul 30 00:47:22.516686 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:22.516672 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471-istio-podinfo\") pod \"e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk\" (UID: \"b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471\") " pod="openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk" Jul 30 00:47:22.522065 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:22.522037 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471-istio-token\") pod \"e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk\" (UID: \"b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471\") " pod="openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk" Jul 30 00:47:22.522230 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:22.522212 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9h2j\" (UniqueName: \"kubernetes.io/projected/b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471-kube-api-access-l9h2j\") pod \"e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk\" (UID: \"b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471\") " pod="openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk" Jul 30 00:47:22.572910 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:22.572868 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk" Jul 30 00:47:22.715975 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:22.715947 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk"] Jul 30 00:47:22.717480 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:47:22.717453 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1b4f8f3_a9f4_48c5_bcf3_ad4fb4316471.slice/crio-e1c9a4747a0f97337570e97cf3bfcc0d49f68ac7fecd42ecb1109d0cf4a5ed03 WatchSource:0}: Error finding container e1c9a4747a0f97337570e97cf3bfcc0d49f68ac7fecd42ecb1109d0cf4a5ed03: Status 404 returned error can't find the container with id e1c9a4747a0f97337570e97cf3bfcc0d49f68ac7fecd42ecb1109d0cf4a5ed03 Jul 30 00:47:22.719797 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:22.719766 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:47:22.719897 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:22.719828 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:47:22.719897 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:22.719876 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:47:23.319636 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:23.319596 2563 patch_prober.go:28] interesting pod/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused Jul 30 00:47:23.319833 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:23.319652 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9" podUID="9f6736bc-86b8-4799-a5ac-cf21540272fa" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 30 00:47:23.510790 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:23.510727 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk" event={"ID":"b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471","Type":"ContainerStarted","Data":"c7f36a67e7805ab5e829bdbf3c80a0de4f22731a7b76dbebfe5a2ba32d097566"} Jul 30 00:47:23.510991 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:23.510803 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk" event={"ID":"b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471","Type":"ContainerStarted","Data":"e1c9a4747a0f97337570e97cf3bfcc0d49f68ac7fecd42ecb1109d0cf4a5ed03"} Jul 30 00:47:23.533642 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:23.533594 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk" podStartSLOduration=1.533578455 podStartE2EDuration="1.533578455s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:47:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:47:23.530831983 +0000 UTC m=+1807.589587954" watchObservedRunningTime="2026-07-30 00:47:23.533578455 +0000 UTC m=+1807.592334423" Jul 30 00:47:23.573632 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:23.573557 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk" Jul 30 00:47:23.578329 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:23.578307 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk" Jul 30 00:47:24.516067 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:24.516023 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk" Jul 30 00:47:24.516978 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:24.516959 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk" Jul 30 00:47:24.853953 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:24.853913 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:47:24.953563 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:24.953531 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:47:26.016201 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:26.016173 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:47:26.153252 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:26.153222 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:47:26.459400 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:26.459296 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:47:26.560217 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:26.560183 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:47:26.849529 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:26.849506 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9" Jul 30 00:47:26.959957 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:26.959923 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f6736bc-86b8-4799-a5ac-cf21540272fa-workload-socket\") pod \"9f6736bc-86b8-4799-a5ac-cf21540272fa\" (UID: \"9f6736bc-86b8-4799-a5ac-cf21540272fa\") " Jul 30 00:47:26.959957 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:26.959960 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qllvv\" (UniqueName: \"kubernetes.io/projected/9f6736bc-86b8-4799-a5ac-cf21540272fa-kube-api-access-qllvv\") pod \"9f6736bc-86b8-4799-a5ac-cf21540272fa\" (UID: \"9f6736bc-86b8-4799-a5ac-cf21540272fa\") " Jul 30 00:47:26.960206 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:26.960016 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f6736bc-86b8-4799-a5ac-cf21540272fa-workload-certs\") pod \"9f6736bc-86b8-4799-a5ac-cf21540272fa\" (UID: \"9f6736bc-86b8-4799-a5ac-cf21540272fa\") " Jul 30 00:47:26.960206 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:26.960046 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f6736bc-86b8-4799-a5ac-cf21540272fa-istio-podinfo\") pod \"9f6736bc-86b8-4799-a5ac-cf21540272fa\" (UID: \"9f6736bc-86b8-4799-a5ac-cf21540272fa\") " Jul 30 00:47:26.960206 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:26.960075 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f6736bc-86b8-4799-a5ac-cf21540272fa-credential-socket\") pod \"9f6736bc-86b8-4799-a5ac-cf21540272fa\" (UID: \"9f6736bc-86b8-4799-a5ac-cf21540272fa\") " Jul 30 00:47:26.960206 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:26.960121 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f6736bc-86b8-4799-a5ac-cf21540272fa-istio-envoy\") pod \"9f6736bc-86b8-4799-a5ac-cf21540272fa\" (UID: \"9f6736bc-86b8-4799-a5ac-cf21540272fa\") " Jul 30 00:47:26.960409 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:26.960243 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f6736bc-86b8-4799-a5ac-cf21540272fa-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9f6736bc-86b8-4799-a5ac-cf21540272fa" (UID: "9f6736bc-86b8-4799-a5ac-cf21540272fa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:47:26.960409 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:26.960278 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f6736bc-86b8-4799-a5ac-cf21540272fa-istio-token\") pod \"9f6736bc-86b8-4799-a5ac-cf21540272fa\" (UID: \"9f6736bc-86b8-4799-a5ac-cf21540272fa\") " Jul 30 00:47:26.960409 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:26.960344 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f6736bc-86b8-4799-a5ac-cf21540272fa-istiod-ca-cert\") pod \"9f6736bc-86b8-4799-a5ac-cf21540272fa\" (UID: \"9f6736bc-86b8-4799-a5ac-cf21540272fa\") " Jul 30 00:47:26.960409 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:26.960373 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f6736bc-86b8-4799-a5ac-cf21540272fa-istio-data\") pod \"9f6736bc-86b8-4799-a5ac-cf21540272fa\" (UID: \"9f6736bc-86b8-4799-a5ac-cf21540272fa\") " Jul 30 00:47:26.960715 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:26.960691 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f6736bc-86b8-4799-a5ac-cf21540272fa-workload-socket\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:47:26.960965 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:26.960939 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f6736bc-86b8-4799-a5ac-cf21540272fa-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9f6736bc-86b8-4799-a5ac-cf21540272fa" (UID: "9f6736bc-86b8-4799-a5ac-cf21540272fa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 00:47:26.961333 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:26.961309 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f6736bc-86b8-4799-a5ac-cf21540272fa-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9f6736bc-86b8-4799-a5ac-cf21540272fa" (UID: "9f6736bc-86b8-4799-a5ac-cf21540272fa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:47:26.961402 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:26.961380 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f6736bc-86b8-4799-a5ac-cf21540272fa-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9f6736bc-86b8-4799-a5ac-cf21540272fa" (UID: "9f6736bc-86b8-4799-a5ac-cf21540272fa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:47:26.961531 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:26.961509 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f6736bc-86b8-4799-a5ac-cf21540272fa-istio-data" (OuterVolumeSpecName: "istio-data") pod "9f6736bc-86b8-4799-a5ac-cf21540272fa" (UID: "9f6736bc-86b8-4799-a5ac-cf21540272fa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:47:26.962446 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:26.962415 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f6736bc-86b8-4799-a5ac-cf21540272fa-istio-token" (OuterVolumeSpecName: "istio-token") pod "9f6736bc-86b8-4799-a5ac-cf21540272fa" (UID: "9f6736bc-86b8-4799-a5ac-cf21540272fa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 00:47:26.962578 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:26.962559 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f6736bc-86b8-4799-a5ac-cf21540272fa-kube-api-access-qllvv" (OuterVolumeSpecName: "kube-api-access-qllvv") pod "9f6736bc-86b8-4799-a5ac-cf21540272fa" (UID: "9f6736bc-86b8-4799-a5ac-cf21540272fa"). InnerVolumeSpecName "kube-api-access-qllvv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 00:47:26.962673 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:26.962654 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9f6736bc-86b8-4799-a5ac-cf21540272fa-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9f6736bc-86b8-4799-a5ac-cf21540272fa" (UID: "9f6736bc-86b8-4799-a5ac-cf21540272fa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 30 00:47:26.963365 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:26.963350 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f6736bc-86b8-4799-a5ac-cf21540272fa-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9f6736bc-86b8-4799-a5ac-cf21540272fa" (UID: "9f6736bc-86b8-4799-a5ac-cf21540272fa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:47:27.061618 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:27.061519 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f6736bc-86b8-4799-a5ac-cf21540272fa-istiod-ca-cert\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:47:27.061618 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:27.061552 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f6736bc-86b8-4799-a5ac-cf21540272fa-istio-data\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:47:27.061618 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:27.061561 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qllvv\" (UniqueName: \"kubernetes.io/projected/9f6736bc-86b8-4799-a5ac-cf21540272fa-kube-api-access-qllvv\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:47:27.061618 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:27.061570 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f6736bc-86b8-4799-a5ac-cf21540272fa-workload-certs\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:47:27.061618 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:27.061580 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f6736bc-86b8-4799-a5ac-cf21540272fa-istio-podinfo\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:47:27.061618 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:27.061587 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f6736bc-86b8-4799-a5ac-cf21540272fa-credential-socket\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:47:27.061618 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:27.061595 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f6736bc-86b8-4799-a5ac-cf21540272fa-istio-envoy\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:47:27.061618 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:27.061603 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f6736bc-86b8-4799-a5ac-cf21540272fa-istio-token\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:47:27.531713 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:27.531677 2563 generic.go:356] "Generic (PLEG): container finished" podID="9f6736bc-86b8-4799-a5ac-cf21540272fa" containerID="c711c7625cd1db709049849707d041504bceb55a3673e3a7810f2f5903e124ee" exitCode=0 Jul 30 00:47:27.531921 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:27.531726 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9" Jul 30 00:47:27.531921 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:27.531765 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9" event={"ID":"9f6736bc-86b8-4799-a5ac-cf21540272fa","Type":"ContainerDied","Data":"c711c7625cd1db709049849707d041504bceb55a3673e3a7810f2f5903e124ee"} Jul 30 00:47:27.531921 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:27.531819 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9" event={"ID":"9f6736bc-86b8-4799-a5ac-cf21540272fa","Type":"ContainerDied","Data":"4af92fea75374e3310a3e3d52f8543d05238b09cde8f120b679d28323d53e3e5"} Jul 30 00:47:27.531921 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:27.531846 2563 scope.go:117] "RemoveContainer" containerID="c711c7625cd1db709049849707d041504bceb55a3673e3a7810f2f5903e124ee" Jul 30 00:47:27.542099 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:27.542080 2563 scope.go:117] "RemoveContainer" containerID="1d1df52901436ca5e9eafdc8d6e2eaa78d5f65a8f2432afe3d64b9223c402554" Jul 30 00:47:27.552942 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:27.552908 2563 scope.go:117] "RemoveContainer" containerID="c711c7625cd1db709049849707d041504bceb55a3673e3a7810f2f5903e124ee" Jul 30 00:47:27.553359 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:47:27.553336 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c711c7625cd1db709049849707d041504bceb55a3673e3a7810f2f5903e124ee\": container with ID starting with c711c7625cd1db709049849707d041504bceb55a3673e3a7810f2f5903e124ee not found: ID does not exist" containerID="c711c7625cd1db709049849707d041504bceb55a3673e3a7810f2f5903e124ee" Jul 30 00:47:27.553474 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:27.553371 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c711c7625cd1db709049849707d041504bceb55a3673e3a7810f2f5903e124ee"} err="failed to get container status \"c711c7625cd1db709049849707d041504bceb55a3673e3a7810f2f5903e124ee\": rpc error: code = NotFound desc = could not find container \"c711c7625cd1db709049849707d041504bceb55a3673e3a7810f2f5903e124ee\": container with ID starting with c711c7625cd1db709049849707d041504bceb55a3673e3a7810f2f5903e124ee not found: ID does not exist" Jul 30 00:47:27.553474 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:27.553395 2563 scope.go:117] "RemoveContainer" containerID="1d1df52901436ca5e9eafdc8d6e2eaa78d5f65a8f2432afe3d64b9223c402554" Jul 30 00:47:27.553716 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:47:27.553685 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d1df52901436ca5e9eafdc8d6e2eaa78d5f65a8f2432afe3d64b9223c402554\": container with ID starting with 1d1df52901436ca5e9eafdc8d6e2eaa78d5f65a8f2432afe3d64b9223c402554 not found: ID does not exist" containerID="1d1df52901436ca5e9eafdc8d6e2eaa78d5f65a8f2432afe3d64b9223c402554" Jul 30 00:47:27.553794 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:27.553725 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d1df52901436ca5e9eafdc8d6e2eaa78d5f65a8f2432afe3d64b9223c402554"} err="failed to get container status \"1d1df52901436ca5e9eafdc8d6e2eaa78d5f65a8f2432afe3d64b9223c402554\": rpc error: code = NotFound desc = could not find container \"1d1df52901436ca5e9eafdc8d6e2eaa78d5f65a8f2432afe3d64b9223c402554\": container with ID starting with 1d1df52901436ca5e9eafdc8d6e2eaa78d5f65a8f2432afe3d64b9223c402554 not found: ID does not exist" Jul 30 00:47:27.557451 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:27.557428 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9"] Jul 30 00:47:27.559913 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:27.559887 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-5213aadc-openshift-default-5df5799b88-9d4g9"] Jul 30 00:47:28.475777 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:28.475726 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f6736bc-86b8-4799-a5ac-cf21540272fa" path="/var/lib/kubelet/pods/9f6736bc-86b8-4799-a5ac-cf21540272fa/volumes" Jul 30 00:47:38.609225 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:38.609192 2563 generic.go:356] "Generic (PLEG): container finished" podID="b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471" containerID="c7f36a67e7805ab5e829bdbf3c80a0de4f22731a7b76dbebfe5a2ba32d097566" exitCode=0 Jul 30 00:47:38.609682 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:38.609271 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk" event={"ID":"b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471","Type":"ContainerDied","Data":"c7f36a67e7805ab5e829bdbf3c80a0de4f22731a7b76dbebfe5a2ba32d097566"} Jul 30 00:47:38.609682 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:38.609675 2563 scope.go:117] "RemoveContainer" containerID="c7f36a67e7805ab5e829bdbf3c80a0de4f22731a7b76dbebfe5a2ba32d097566" Jul 30 00:47:38.610751 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:38.610699 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:47:38.610856 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:38.610785 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:47:38.610856 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:38.610816 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:47:39.516207 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:39.516175 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk" Jul 30 00:47:39.617641 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:39.617600 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk" event={"ID":"b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471","Type":"ContainerStarted","Data":"5c8dad4e8826fb760008ccc160ccb89d5b9a22af1410feb76dbe04212fe245a1"} Jul 30 00:47:40.573512 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:40.573474 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk" Jul 30 00:47:40.574876 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:40.574844 2563 patch_prober.go:28] interesting pod/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 30 00:47:40.575013 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:40.574906 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk" podUID="b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 30 00:47:41.573996 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:41.573949 2563 patch_prober.go:28] interesting pod/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 30 00:47:41.574504 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:41.574041 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk" podUID="b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 30 00:47:42.574527 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:42.574483 2563 patch_prober.go:28] interesting pod/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 30 00:47:42.575030 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:42.574549 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk" podUID="b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 30 00:47:43.573752 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:43.573691 2563 patch_prober.go:28] interesting pod/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 30 00:47:43.574064 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:43.574032 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk" podUID="b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 30 00:47:44.578058 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:44.578030 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk" Jul 30 00:47:44.578451 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:44.578321 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk" Jul 30 00:47:44.579085 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:44.579066 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk" Jul 30 00:47:56.733958 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:56.733920 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:47:56.796374 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:56.796344 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:47:56.852454 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:56.852414 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:47:56.906711 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:56.906676 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:47:57.014170 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:57.014135 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk"] Jul 30 00:47:57.014431 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:57.014398 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk" podUID="b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471" containerName="istio-proxy" containerID="cri-o://5c8dad4e8826fb760008ccc160ccb89d5b9a22af1410feb76dbe04212fe245a1" gracePeriod=30 Jul 30 00:47:57.037220 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:57.037182 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:47:57.251421 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:57.251385 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:47:57.688750 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:57.688701 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:47:57.791609 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:57.791577 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:47:57.881226 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:57.881191 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:47:59.256153 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:59.256114 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:47:59.581034 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:59.580948 2563 patch_prober.go:28] interesting pod/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 30 00:47:59.581034 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:47:59.581013 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk" podUID="b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 30 00:48:02.269951 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:48:02.269924 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk" Jul 30 00:48:02.395949 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:48:02.395848 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471-istio-token\") pod \"b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471\" (UID: \"b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471\") " Jul 30 00:48:02.395949 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:48:02.395910 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471-istiod-ca-cert\") pod \"b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471\" (UID: \"b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471\") " Jul 30 00:48:02.395949 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:48:02.395938 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471-workload-certs\") pod \"b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471\" (UID: \"b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471\") " Jul 30 00:48:02.396249 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:48:02.395963 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471-istio-podinfo\") pod \"b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471\" (UID: \"b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471\") " Jul 30 00:48:02.396249 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:48:02.396005 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l9h2j\" (UniqueName: \"kubernetes.io/projected/b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471-kube-api-access-l9h2j\") pod \"b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471\" (UID: \"b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471\") " Jul 30 00:48:02.396249 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:48:02.396028 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471-istio-data\") pod \"b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471\" (UID: \"b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471\") " Jul 30 00:48:02.396249 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:48:02.396050 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471-workload-socket\") pod \"b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471\" (UID: \"b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471\") " Jul 30 00:48:02.396249 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:48:02.396087 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471-credential-socket\") pod \"b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471\" (UID: \"b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471\") " Jul 30 00:48:02.396249 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:48:02.396109 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471-istio-envoy\") pod \"b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471\" (UID: \"b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471\") " Jul 30 00:48:02.396549 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:48:02.396457 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471" (UID: "b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:48:02.396549 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:48:02.396463 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471" (UID: "b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 00:48:02.396660 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:48:02.396580 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471" (UID: "b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:48:02.396795 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:48:02.396772 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471" (UID: "b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:48:02.397041 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:48:02.397011 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471-istio-data" (OuterVolumeSpecName: "istio-data") pod "b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471" (UID: "b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:48:02.398267 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:48:02.398239 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471" (UID: "b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 30 00:48:02.398353 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:48:02.398297 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471-kube-api-access-l9h2j" (OuterVolumeSpecName: "kube-api-access-l9h2j") pod "b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471" (UID: "b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471"). InnerVolumeSpecName "kube-api-access-l9h2j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 00:48:02.398353 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:48:02.398322 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471-istio-token" (OuterVolumeSpecName: "istio-token") pod "b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471" (UID: "b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 00:48:02.398353 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:48:02.398345 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471" (UID: "b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:48:02.496827 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:48:02.496791 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471-istio-token\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:48:02.496827 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:48:02.496822 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471-istiod-ca-cert\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:48:02.497020 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:48:02.496834 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471-workload-certs\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:48:02.497020 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:48:02.496845 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471-istio-podinfo\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:48:02.497020 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:48:02.496857 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l9h2j\" (UniqueName: \"kubernetes.io/projected/b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471-kube-api-access-l9h2j\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:48:02.497020 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:48:02.496868 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471-istio-data\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:48:02.497020 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:48:02.496879 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471-workload-socket\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:48:02.497020 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:48:02.496890 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471-credential-socket\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:48:02.497020 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:48:02.496901 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471-istio-envoy\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:48:02.744136 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:48:02.744044 2563 generic.go:356] "Generic (PLEG): container finished" podID="b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471" containerID="5c8dad4e8826fb760008ccc160ccb89d5b9a22af1410feb76dbe04212fe245a1" exitCode=0 Jul 30 00:48:02.744136 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:48:02.744097 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk" Jul 30 00:48:02.744386 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:48:02.744131 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk" event={"ID":"b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471","Type":"ContainerDied","Data":"5c8dad4e8826fb760008ccc160ccb89d5b9a22af1410feb76dbe04212fe245a1"} Jul 30 00:48:02.744386 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:48:02.744192 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk" event={"ID":"b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471","Type":"ContainerDied","Data":"e1c9a4747a0f97337570e97cf3bfcc0d49f68ac7fecd42ecb1109d0cf4a5ed03"} Jul 30 00:48:02.744386 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:48:02.744224 2563 scope.go:117] "RemoveContainer" containerID="5c8dad4e8826fb760008ccc160ccb89d5b9a22af1410feb76dbe04212fe245a1" Jul 30 00:48:02.755690 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:48:02.755668 2563 scope.go:117] "RemoveContainer" containerID="c7f36a67e7805ab5e829bdbf3c80a0de4f22731a7b76dbebfe5a2ba32d097566" Jul 30 00:48:02.764562 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:48:02.764528 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk"] Jul 30 00:48:02.765460 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:48:02.765443 2563 scope.go:117] "RemoveContainer" containerID="5c8dad4e8826fb760008ccc160ccb89d5b9a22af1410feb76dbe04212fe245a1" Jul 30 00:48:02.765988 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:48:02.765783 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c8dad4e8826fb760008ccc160ccb89d5b9a22af1410feb76dbe04212fe245a1\": container with ID starting with 5c8dad4e8826fb760008ccc160ccb89d5b9a22af1410feb76dbe04212fe245a1 not found: ID does not exist" containerID="5c8dad4e8826fb760008ccc160ccb89d5b9a22af1410feb76dbe04212fe245a1" Jul 30 00:48:02.765988 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:48:02.765825 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c8dad4e8826fb760008ccc160ccb89d5b9a22af1410feb76dbe04212fe245a1"} err="failed to get container status \"5c8dad4e8826fb760008ccc160ccb89d5b9a22af1410feb76dbe04212fe245a1\": rpc error: code = NotFound desc = could not find container \"5c8dad4e8826fb760008ccc160ccb89d5b9a22af1410feb76dbe04212fe245a1\": container with ID starting with 5c8dad4e8826fb760008ccc160ccb89d5b9a22af1410feb76dbe04212fe245a1 not found: ID does not exist" Jul 30 00:48:02.765988 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:48:02.765911 2563 scope.go:117] "RemoveContainer" containerID="c7f36a67e7805ab5e829bdbf3c80a0de4f22731a7b76dbebfe5a2ba32d097566" Jul 30 00:48:02.766647 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:48:02.766616 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7f36a67e7805ab5e829bdbf3c80a0de4f22731a7b76dbebfe5a2ba32d097566\": container with ID starting with c7f36a67e7805ab5e829bdbf3c80a0de4f22731a7b76dbebfe5a2ba32d097566 not found: ID does not exist" containerID="c7f36a67e7805ab5e829bdbf3c80a0de4f22731a7b76dbebfe5a2ba32d097566" Jul 30 00:48:02.766769 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:48:02.766654 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7f36a67e7805ab5e829bdbf3c80a0de4f22731a7b76dbebfe5a2ba32d097566"} err="failed to get container status \"c7f36a67e7805ab5e829bdbf3c80a0de4f22731a7b76dbebfe5a2ba32d097566\": rpc error: code = NotFound desc = could not find container \"c7f36a67e7805ab5e829bdbf3c80a0de4f22731a7b76dbebfe5a2ba32d097566\": container with ID starting with c7f36a67e7805ab5e829bdbf3c80a0de4f22731a7b76dbebfe5a2ba32d097566 not found: ID does not exist" Jul 30 00:48:02.768662 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:48:02.768633 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-06160795-openshift-default-c9fbbfdc6-l8sdk"] Jul 30 00:48:04.476273 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:48:04.476234 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471" path="/var/lib/kubelet/pods/b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471/volumes" Jul 30 00:48:05.049132 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:48:05.049088 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:48:05.154043 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:48:05.154010 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:48:05.248794 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:48:05.248755 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:48:05.670585 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:48:05.670553 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:48:31.614431 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:48:31.614397 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:48:37.942301 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:48:37.942263 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:48:38.050445 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:48:38.050386 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:48:39.293149 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:48:39.293067 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:48:53.972722 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:48:53.972688 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:49:01.209922 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:01.209889 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:49:01.310381 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:01.310348 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:49:10.418504 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:10.418457 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:49:11.206540 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:11.206505 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:49:17.604361 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:17.604319 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:49:17.725326 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:17.725214 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:49:17.850272 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:17.850239 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:49:17.968546 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:17.968466 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:49:29.284146 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:29.284094 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:49:29.381789 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:29.381754 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:49:30.647611 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:30.647575 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:49:31.089565 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:31.089529 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:49:37.126497 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:37.126449 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:49:37.224145 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:37.224113 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:49:37.725462 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:37.725407 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:49:43.828367 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:43.828334 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:49:43.930290 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:43.930257 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:49:44.053683 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:44.053653 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:49:44.328171 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:44.328138 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm"] Jul 30 00:49:44.328623 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:44.328608 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f6736bc-86b8-4799-a5ac-cf21540272fa" containerName="istio-proxy" Jul 30 00:49:44.328623 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:44.328624 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f6736bc-86b8-4799-a5ac-cf21540272fa" containerName="istio-proxy" Jul 30 00:49:44.328744 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:44.328644 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471" containerName="istio-proxy" Jul 30 00:49:44.328744 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:44.328650 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471" containerName="istio-proxy" Jul 30 00:49:44.328744 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:44.328665 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471" containerName="istio-proxy" Jul 30 00:49:44.328744 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:44.328672 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471" containerName="istio-proxy" Jul 30 00:49:44.328744 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:44.328726 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471" containerName="istio-proxy" Jul 30 00:49:44.328904 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:44.328750 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f6736bc-86b8-4799-a5ac-cf21540272fa" containerName="istio-proxy" Jul 30 00:49:44.328904 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:44.328762 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1b4f8f3-a9f4-48c5-bcf3-ad4fb4316471" containerName="istio-proxy" Jul 30 00:49:44.332184 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:44.332161 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm" Jul 30 00:49:44.334634 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:44.334614 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-819bad55-openshift-default-dockercfg-r74bk\"" Jul 30 00:49:44.350434 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:44.350404 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm"] Jul 30 00:49:44.356772 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:44.352827 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:49:44.437682 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:44.437646 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b2baa76-311c-457b-b557-c9e729111a45-workload-socket\") pod \"e2e-mt-819bad55-openshift-default-577c6f5684-wlszm\" (UID: \"9b2baa76-311c-457b-b557-c9e729111a45\") " pod="openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm" Jul 30 00:49:44.437910 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:44.437693 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b2baa76-311c-457b-b557-c9e729111a45-workload-certs\") pod \"e2e-mt-819bad55-openshift-default-577c6f5684-wlszm\" (UID: \"9b2baa76-311c-457b-b557-c9e729111a45\") " pod="openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm" Jul 30 00:49:44.437910 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:44.437711 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b2baa76-311c-457b-b557-c9e729111a45-istio-token\") pod \"e2e-mt-819bad55-openshift-default-577c6f5684-wlszm\" (UID: \"9b2baa76-311c-457b-b557-c9e729111a45\") " pod="openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm" Jul 30 00:49:44.437910 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:44.437796 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b2baa76-311c-457b-b557-c9e729111a45-istio-podinfo\") pod \"e2e-mt-819bad55-openshift-default-577c6f5684-wlszm\" (UID: \"9b2baa76-311c-457b-b557-c9e729111a45\") " pod="openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm" Jul 30 00:49:44.437910 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:44.437853 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b2baa76-311c-457b-b557-c9e729111a45-istio-envoy\") pod \"e2e-mt-819bad55-openshift-default-577c6f5684-wlszm\" (UID: \"9b2baa76-311c-457b-b557-c9e729111a45\") " pod="openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm" Jul 30 00:49:44.437910 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:44.437873 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b2baa76-311c-457b-b557-c9e729111a45-istio-data\") pod \"e2e-mt-819bad55-openshift-default-577c6f5684-wlszm\" (UID: \"9b2baa76-311c-457b-b557-c9e729111a45\") " pod="openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm" Jul 30 00:49:44.437910 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:44.437898 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjs5b\" (UniqueName: \"kubernetes.io/projected/9b2baa76-311c-457b-b557-c9e729111a45-kube-api-access-tjs5b\") pod \"e2e-mt-819bad55-openshift-default-577c6f5684-wlszm\" (UID: \"9b2baa76-311c-457b-b557-c9e729111a45\") " pod="openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm" Jul 30 00:49:44.438255 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:44.437952 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b2baa76-311c-457b-b557-c9e729111a45-istiod-ca-cert\") pod \"e2e-mt-819bad55-openshift-default-577c6f5684-wlszm\" (UID: \"9b2baa76-311c-457b-b557-c9e729111a45\") " pod="openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm" Jul 30 00:49:44.438255 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:44.438004 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b2baa76-311c-457b-b557-c9e729111a45-credential-socket\") pod \"e2e-mt-819bad55-openshift-default-577c6f5684-wlszm\" (UID: \"9b2baa76-311c-457b-b557-c9e729111a45\") " pod="openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm" Jul 30 00:49:44.538593 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:44.538558 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b2baa76-311c-457b-b557-c9e729111a45-istio-podinfo\") pod \"e2e-mt-819bad55-openshift-default-577c6f5684-wlszm\" (UID: \"9b2baa76-311c-457b-b557-c9e729111a45\") " pod="openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm" Jul 30 00:49:44.538593 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:44.538594 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b2baa76-311c-457b-b557-c9e729111a45-istio-envoy\") pod \"e2e-mt-819bad55-openshift-default-577c6f5684-wlszm\" (UID: \"9b2baa76-311c-457b-b557-c9e729111a45\") " pod="openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm" Jul 30 00:49:44.538893 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:44.538613 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b2baa76-311c-457b-b557-c9e729111a45-istio-data\") pod \"e2e-mt-819bad55-openshift-default-577c6f5684-wlszm\" (UID: \"9b2baa76-311c-457b-b557-c9e729111a45\") " pod="openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm" Jul 30 00:49:44.538893 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:44.538730 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjs5b\" (UniqueName: \"kubernetes.io/projected/9b2baa76-311c-457b-b557-c9e729111a45-kube-api-access-tjs5b\") pod \"e2e-mt-819bad55-openshift-default-577c6f5684-wlszm\" (UID: \"9b2baa76-311c-457b-b557-c9e729111a45\") " pod="openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm" Jul 30 00:49:44.538893 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:44.538812 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b2baa76-311c-457b-b557-c9e729111a45-istiod-ca-cert\") pod \"e2e-mt-819bad55-openshift-default-577c6f5684-wlszm\" (UID: \"9b2baa76-311c-457b-b557-c9e729111a45\") " pod="openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm" Jul 30 00:49:44.538893 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:44.538892 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b2baa76-311c-457b-b557-c9e729111a45-credential-socket\") pod \"e2e-mt-819bad55-openshift-default-577c6f5684-wlszm\" (UID: \"9b2baa76-311c-457b-b557-c9e729111a45\") " pod="openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm" Jul 30 00:49:44.539189 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:44.538979 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b2baa76-311c-457b-b557-c9e729111a45-workload-socket\") pod \"e2e-mt-819bad55-openshift-default-577c6f5684-wlszm\" (UID: \"9b2baa76-311c-457b-b557-c9e729111a45\") " pod="openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm" Jul 30 00:49:44.539189 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:44.539019 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b2baa76-311c-457b-b557-c9e729111a45-workload-certs\") pod \"e2e-mt-819bad55-openshift-default-577c6f5684-wlszm\" (UID: \"9b2baa76-311c-457b-b557-c9e729111a45\") " pod="openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm" Jul 30 00:49:44.539189 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:44.539050 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b2baa76-311c-457b-b557-c9e729111a45-istio-token\") pod \"e2e-mt-819bad55-openshift-default-577c6f5684-wlszm\" (UID: \"9b2baa76-311c-457b-b557-c9e729111a45\") " pod="openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm" Jul 30 00:49:44.539189 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:44.539163 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b2baa76-311c-457b-b557-c9e729111a45-istio-data\") pod \"e2e-mt-819bad55-openshift-default-577c6f5684-wlszm\" (UID: \"9b2baa76-311c-457b-b557-c9e729111a45\") " pod="openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm" Jul 30 00:49:44.539399 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:44.539298 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b2baa76-311c-457b-b557-c9e729111a45-credential-socket\") pod \"e2e-mt-819bad55-openshift-default-577c6f5684-wlszm\" (UID: \"9b2baa76-311c-457b-b557-c9e729111a45\") " pod="openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm" Jul 30 00:49:44.539458 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:44.539421 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b2baa76-311c-457b-b557-c9e729111a45-workload-certs\") pod \"e2e-mt-819bad55-openshift-default-577c6f5684-wlszm\" (UID: \"9b2baa76-311c-457b-b557-c9e729111a45\") " pod="openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm" Jul 30 00:49:44.539755 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:44.539709 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b2baa76-311c-457b-b557-c9e729111a45-istiod-ca-cert\") pod \"e2e-mt-819bad55-openshift-default-577c6f5684-wlszm\" (UID: \"9b2baa76-311c-457b-b557-c9e729111a45\") " pod="openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm" Jul 30 00:49:44.539978 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:44.539953 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b2baa76-311c-457b-b557-c9e729111a45-workload-socket\") pod \"e2e-mt-819bad55-openshift-default-577c6f5684-wlszm\" (UID: \"9b2baa76-311c-457b-b557-c9e729111a45\") " pod="openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm" Jul 30 00:49:44.541263 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:44.541241 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b2baa76-311c-457b-b557-c9e729111a45-istio-envoy\") pod \"e2e-mt-819bad55-openshift-default-577c6f5684-wlszm\" (UID: \"9b2baa76-311c-457b-b557-c9e729111a45\") " pod="openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm" Jul 30 00:49:44.541368 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:44.541306 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b2baa76-311c-457b-b557-c9e729111a45-istio-podinfo\") pod \"e2e-mt-819bad55-openshift-default-577c6f5684-wlszm\" (UID: \"9b2baa76-311c-457b-b557-c9e729111a45\") " pod="openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm" Jul 30 00:49:44.546826 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:44.546799 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b2baa76-311c-457b-b557-c9e729111a45-istio-token\") pod \"e2e-mt-819bad55-openshift-default-577c6f5684-wlszm\" (UID: \"9b2baa76-311c-457b-b557-c9e729111a45\") " pod="openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm" Jul 30 00:49:44.546957 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:44.546938 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjs5b\" (UniqueName: \"kubernetes.io/projected/9b2baa76-311c-457b-b557-c9e729111a45-kube-api-access-tjs5b\") pod \"e2e-mt-819bad55-openshift-default-577c6f5684-wlszm\" (UID: \"9b2baa76-311c-457b-b557-c9e729111a45\") " pod="openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm" Jul 30 00:49:44.548265 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:44.548241 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:49:44.643227 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:44.643134 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm" Jul 30 00:49:44.787721 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:44.787691 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm"] Jul 30 00:49:44.790044 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:49:44.790013 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b2baa76_311c_457b_b557_c9e729111a45.slice/crio-2e395ce58ae980d01018165d0adc59001803f5109823e8ceea4a7f1a4ef00afb WatchSource:0}: Error finding container 2e395ce58ae980d01018165d0adc59001803f5109823e8ceea4a7f1a4ef00afb: Status 404 returned error can't find the container with id 2e395ce58ae980d01018165d0adc59001803f5109823e8ceea4a7f1a4ef00afb Jul 30 00:49:44.792558 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:44.792523 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:49:44.792657 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:44.792587 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:49:44.792657 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:44.792614 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:49:45.271006 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:45.270972 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm" event={"ID":"9b2baa76-311c-457b-b557-c9e729111a45","Type":"ContainerStarted","Data":"481532bcf491ea63cf100f513077a0343354d58f27a3a671549501c0262258fe"} Jul 30 00:49:45.271384 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:45.271028 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm" event={"ID":"9b2baa76-311c-457b-b557-c9e729111a45","Type":"ContainerStarted","Data":"2e395ce58ae980d01018165d0adc59001803f5109823e8ceea4a7f1a4ef00afb"} Jul 30 00:49:45.289467 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:45.289424 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm" podStartSLOduration=1.2894096689999999 podStartE2EDuration="1.289409669s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:49:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:49:45.288143592 +0000 UTC m=+1949.346899559" watchObservedRunningTime="2026-07-30 00:49:45.289409669 +0000 UTC m=+1949.348165636" Jul 30 00:49:45.643690 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:45.643603 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm" Jul 30 00:49:45.648909 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:45.648872 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm" Jul 30 00:49:46.276694 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:46.276660 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm" Jul 30 00:49:46.277552 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:46.277534 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm" Jul 30 00:49:47.143018 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:47.142985 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:49:47.247808 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:47.247778 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:49:48.316601 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:48.316570 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:49:48.559652 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:48.559620 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:49:48.876805 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:48.876770 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:49:48.974348 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:48.974313 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:49:58.423390 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:49:58.423356 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:50:01.395059 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:01.394609 2563 patch_prober.go:28] interesting pod/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": read tcp 10.132.0.2:42006->10.132.0.54:15021: read: connection reset by peer" start-of-body= Jul 30 00:50:01.395059 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:01.394722 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm" podUID="9b2baa76-311c-457b-b557-c9e729111a45" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": read tcp 10.132.0.2:42006->10.132.0.54:15021: read: connection reset by peer" Jul 30 00:50:02.368660 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:02.368624 2563 generic.go:356] "Generic (PLEG): container finished" podID="9b2baa76-311c-457b-b557-c9e729111a45" containerID="481532bcf491ea63cf100f513077a0343354d58f27a3a671549501c0262258fe" exitCode=0 Jul 30 00:50:02.368856 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:02.368681 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm" event={"ID":"9b2baa76-311c-457b-b557-c9e729111a45","Type":"ContainerDied","Data":"481532bcf491ea63cf100f513077a0343354d58f27a3a671549501c0262258fe"} Jul 30 00:50:02.369185 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:02.369170 2563 scope.go:117] "RemoveContainer" containerID="481532bcf491ea63cf100f513077a0343354d58f27a3a671549501c0262258fe" Jul 30 00:50:02.370315 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:02.370286 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:50:02.370395 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:02.370376 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:50:02.370439 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:02.370422 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:50:03.380283 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:03.380239 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm" event={"ID":"9b2baa76-311c-457b-b557-c9e729111a45","Type":"ContainerStarted","Data":"351b0bba2cbc7e73ec0c6af7fb3a30dcc960d540f8ac8c064460b08dfa13f49c"} Jul 30 00:50:03.643581 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:03.643499 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm" Jul 30 00:50:03.643782 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:03.643757 2563 patch_prober.go:28] interesting pod/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 30 00:50:03.643838 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:03.643805 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm" podUID="9b2baa76-311c-457b-b557-c9e729111a45" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 30 00:50:04.643662 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:04.643627 2563 patch_prober.go:28] interesting pod/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 30 00:50:04.644059 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:04.643683 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm" podUID="9b2baa76-311c-457b-b557-c9e729111a45" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 30 00:50:05.644460 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:05.644425 2563 patch_prober.go:28] interesting pod/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 30 00:50:05.644823 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:05.644476 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm" podUID="9b2baa76-311c-457b-b557-c9e729111a45" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 30 00:50:06.643575 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:06.643534 2563 patch_prober.go:28] interesting pod/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 30 00:50:06.643789 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:06.643597 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm" podUID="9b2baa76-311c-457b-b557-c9e729111a45" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 30 00:50:07.647148 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:07.647116 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm" Jul 30 00:50:07.647577 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:07.647350 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm" Jul 30 00:50:07.647973 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:07.647952 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm" Jul 30 00:50:12.111070 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:12.110987 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:50:13.723775 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:13.723722 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:50:17.574474 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:17.574442 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:50:18.863430 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:18.863395 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:50:18.916639 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:18.916608 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:50:30.807628 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:30.807596 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:50:30.864061 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:30.864027 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:50:35.848221 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:35.848185 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:50:35.948514 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:35.948485 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:50:36.077565 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:36.077530 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm"] Jul 30 00:50:36.077834 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:36.077796 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm" podUID="9b2baa76-311c-457b-b557-c9e729111a45" containerName="istio-proxy" containerID="cri-o://351b0bba2cbc7e73ec0c6af7fb3a30dcc960d540f8ac8c064460b08dfa13f49c" gracePeriod=30 Jul 30 00:50:36.101227 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:36.101147 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:50:36.321907 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:36.321871 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:50:36.453372 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:36.453196 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:50:36.543137 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:36.543100 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:50:36.651437 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:36.651406 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:50:37.650378 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:37.650342 2563 patch_prober.go:28] interesting pod/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 30 00:50:37.650765 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:37.650410 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm" podUID="9b2baa76-311c-457b-b557-c9e729111a45" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 30 00:50:41.338206 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:41.338146 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm" Jul 30 00:50:41.353923 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:41.353897 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b2baa76-311c-457b-b557-c9e729111a45-workload-certs\") pod \"9b2baa76-311c-457b-b557-c9e729111a45\" (UID: \"9b2baa76-311c-457b-b557-c9e729111a45\") " Jul 30 00:50:41.353923 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:41.353925 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b2baa76-311c-457b-b557-c9e729111a45-istio-envoy\") pod \"9b2baa76-311c-457b-b557-c9e729111a45\" (UID: \"9b2baa76-311c-457b-b557-c9e729111a45\") " Jul 30 00:50:41.354078 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:41.353968 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b2baa76-311c-457b-b557-c9e729111a45-istio-podinfo\") pod \"9b2baa76-311c-457b-b557-c9e729111a45\" (UID: \"9b2baa76-311c-457b-b557-c9e729111a45\") " Jul 30 00:50:41.354078 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:41.353995 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tjs5b\" (UniqueName: \"kubernetes.io/projected/9b2baa76-311c-457b-b557-c9e729111a45-kube-api-access-tjs5b\") pod \"9b2baa76-311c-457b-b557-c9e729111a45\" (UID: \"9b2baa76-311c-457b-b557-c9e729111a45\") " Jul 30 00:50:41.354078 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:41.354034 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b2baa76-311c-457b-b557-c9e729111a45-workload-socket\") pod \"9b2baa76-311c-457b-b557-c9e729111a45\" (UID: \"9b2baa76-311c-457b-b557-c9e729111a45\") " Jul 30 00:50:41.354078 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:41.354058 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b2baa76-311c-457b-b557-c9e729111a45-istiod-ca-cert\") pod \"9b2baa76-311c-457b-b557-c9e729111a45\" (UID: \"9b2baa76-311c-457b-b557-c9e729111a45\") " Jul 30 00:50:41.354288 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:41.354144 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b2baa76-311c-457b-b557-c9e729111a45-credential-socket\") pod \"9b2baa76-311c-457b-b557-c9e729111a45\" (UID: \"9b2baa76-311c-457b-b557-c9e729111a45\") " Jul 30 00:50:41.354288 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:41.354172 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b2baa76-311c-457b-b557-c9e729111a45-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9b2baa76-311c-457b-b557-c9e729111a45" (UID: "9b2baa76-311c-457b-b557-c9e729111a45"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:50:41.354288 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:41.354180 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b2baa76-311c-457b-b557-c9e729111a45-istio-token\") pod \"9b2baa76-311c-457b-b557-c9e729111a45\" (UID: \"9b2baa76-311c-457b-b557-c9e729111a45\") " Jul 30 00:50:41.354288 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:41.354233 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b2baa76-311c-457b-b557-c9e729111a45-istio-data\") pod \"9b2baa76-311c-457b-b557-c9e729111a45\" (UID: \"9b2baa76-311c-457b-b557-c9e729111a45\") " Jul 30 00:50:41.354558 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:41.354526 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b2baa76-311c-457b-b557-c9e729111a45-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9b2baa76-311c-457b-b557-c9e729111a45" (UID: "9b2baa76-311c-457b-b557-c9e729111a45"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:50:41.354621 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:41.354594 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b2baa76-311c-457b-b557-c9e729111a45-workload-certs\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:50:41.354621 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:41.354600 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b2baa76-311c-457b-b557-c9e729111a45-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9b2baa76-311c-457b-b557-c9e729111a45" (UID: "9b2baa76-311c-457b-b557-c9e729111a45"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 00:50:41.354721 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:41.354659 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b2baa76-311c-457b-b557-c9e729111a45-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9b2baa76-311c-457b-b557-c9e729111a45" (UID: "9b2baa76-311c-457b-b557-c9e729111a45"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:50:41.355419 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:41.355380 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b2baa76-311c-457b-b557-c9e729111a45-istio-data" (OuterVolumeSpecName: "istio-data") pod "9b2baa76-311c-457b-b557-c9e729111a45" (UID: "9b2baa76-311c-457b-b557-c9e729111a45"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:50:41.356838 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:41.356801 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9b2baa76-311c-457b-b557-c9e729111a45-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9b2baa76-311c-457b-b557-c9e729111a45" (UID: "9b2baa76-311c-457b-b557-c9e729111a45"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 30 00:50:41.357051 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:41.357021 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b2baa76-311c-457b-b557-c9e729111a45-kube-api-access-tjs5b" (OuterVolumeSpecName: "kube-api-access-tjs5b") pod "9b2baa76-311c-457b-b557-c9e729111a45" (UID: "9b2baa76-311c-457b-b557-c9e729111a45"). InnerVolumeSpecName "kube-api-access-tjs5b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 00:50:41.357138 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:41.357114 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b2baa76-311c-457b-b557-c9e729111a45-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9b2baa76-311c-457b-b557-c9e729111a45" (UID: "9b2baa76-311c-457b-b557-c9e729111a45"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:50:41.358106 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:41.358040 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b2baa76-311c-457b-b557-c9e729111a45-istio-token" (OuterVolumeSpecName: "istio-token") pod "9b2baa76-311c-457b-b557-c9e729111a45" (UID: "9b2baa76-311c-457b-b557-c9e729111a45"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 00:50:41.456067 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:41.456029 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b2baa76-311c-457b-b557-c9e729111a45-istio-podinfo\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:50:41.456067 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:41.456057 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tjs5b\" (UniqueName: \"kubernetes.io/projected/9b2baa76-311c-457b-b557-c9e729111a45-kube-api-access-tjs5b\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:50:41.456067 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:41.456066 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b2baa76-311c-457b-b557-c9e729111a45-workload-socket\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:50:41.456067 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:41.456076 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b2baa76-311c-457b-b557-c9e729111a45-istiod-ca-cert\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:50:41.456398 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:41.456085 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b2baa76-311c-457b-b557-c9e729111a45-credential-socket\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:50:41.456398 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:41.456093 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b2baa76-311c-457b-b557-c9e729111a45-istio-token\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:50:41.456398 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:41.456100 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b2baa76-311c-457b-b557-c9e729111a45-istio-data\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:50:41.456398 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:41.456108 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b2baa76-311c-457b-b557-c9e729111a45-istio-envoy\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:50:41.574408 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:41.574378 2563 generic.go:356] "Generic (PLEG): container finished" podID="9b2baa76-311c-457b-b557-c9e729111a45" containerID="351b0bba2cbc7e73ec0c6af7fb3a30dcc960d540f8ac8c064460b08dfa13f49c" exitCode=0 Jul 30 00:50:41.574577 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:41.574440 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm" Jul 30 00:50:41.574577 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:41.574452 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm" event={"ID":"9b2baa76-311c-457b-b557-c9e729111a45","Type":"ContainerDied","Data":"351b0bba2cbc7e73ec0c6af7fb3a30dcc960d540f8ac8c064460b08dfa13f49c"} Jul 30 00:50:41.574577 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:41.574493 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm" event={"ID":"9b2baa76-311c-457b-b557-c9e729111a45","Type":"ContainerDied","Data":"2e395ce58ae980d01018165d0adc59001803f5109823e8ceea4a7f1a4ef00afb"} Jul 30 00:50:41.574577 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:41.574516 2563 scope.go:117] "RemoveContainer" containerID="351b0bba2cbc7e73ec0c6af7fb3a30dcc960d540f8ac8c064460b08dfa13f49c" Jul 30 00:50:41.583770 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:41.583720 2563 scope.go:117] "RemoveContainer" containerID="481532bcf491ea63cf100f513077a0343354d58f27a3a671549501c0262258fe" Jul 30 00:50:41.591945 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:41.591927 2563 scope.go:117] "RemoveContainer" containerID="351b0bba2cbc7e73ec0c6af7fb3a30dcc960d540f8ac8c064460b08dfa13f49c" Jul 30 00:50:41.592219 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:50:41.592199 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"351b0bba2cbc7e73ec0c6af7fb3a30dcc960d540f8ac8c064460b08dfa13f49c\": container with ID starting with 351b0bba2cbc7e73ec0c6af7fb3a30dcc960d540f8ac8c064460b08dfa13f49c not found: ID does not exist" containerID="351b0bba2cbc7e73ec0c6af7fb3a30dcc960d540f8ac8c064460b08dfa13f49c" Jul 30 00:50:41.592278 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:41.592226 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"351b0bba2cbc7e73ec0c6af7fb3a30dcc960d540f8ac8c064460b08dfa13f49c"} err="failed to get container status \"351b0bba2cbc7e73ec0c6af7fb3a30dcc960d540f8ac8c064460b08dfa13f49c\": rpc error: code = NotFound desc = could not find container \"351b0bba2cbc7e73ec0c6af7fb3a30dcc960d540f8ac8c064460b08dfa13f49c\": container with ID starting with 351b0bba2cbc7e73ec0c6af7fb3a30dcc960d540f8ac8c064460b08dfa13f49c not found: ID does not exist" Jul 30 00:50:41.592278 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:41.592245 2563 scope.go:117] "RemoveContainer" containerID="481532bcf491ea63cf100f513077a0343354d58f27a3a671549501c0262258fe" Jul 30 00:50:41.592501 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:50:41.592478 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"481532bcf491ea63cf100f513077a0343354d58f27a3a671549501c0262258fe\": container with ID starting with 481532bcf491ea63cf100f513077a0343354d58f27a3a671549501c0262258fe not found: ID does not exist" containerID="481532bcf491ea63cf100f513077a0343354d58f27a3a671549501c0262258fe" Jul 30 00:50:41.592561 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:41.592510 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"481532bcf491ea63cf100f513077a0343354d58f27a3a671549501c0262258fe"} err="failed to get container status \"481532bcf491ea63cf100f513077a0343354d58f27a3a671549501c0262258fe\": rpc error: code = NotFound desc = could not find container \"481532bcf491ea63cf100f513077a0343354d58f27a3a671549501c0262258fe\": container with ID starting with 481532bcf491ea63cf100f513077a0343354d58f27a3a671549501c0262258fe not found: ID does not exist" Jul 30 00:50:41.596216 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:41.596195 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm"] Jul 30 00:50:41.603721 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:41.603693 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-819bad55-openshift-default-577c6f5684-wlszm"] Jul 30 00:50:42.349335 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:42.349302 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:50:42.450671 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:42.450639 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:50:42.480705 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:42.480673 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b2baa76-311c-457b-b557-c9e729111a45" path="/var/lib/kubelet/pods/9b2baa76-311c-457b-b557-c9e729111a45/volumes" Jul 30 00:50:42.549913 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:42.549879 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:50:42.781759 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:42.781710 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:50:48.387086 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:48.387052 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:50:48.487344 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:48.487313 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:50:55.122473 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:55.122441 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:50:55.381263 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:55.381175 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z"] Jul 30 00:50:55.381762 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:55.381727 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b2baa76-311c-457b-b557-c9e729111a45" containerName="istio-proxy" Jul 30 00:50:55.381762 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:55.381762 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b2baa76-311c-457b-b557-c9e729111a45" containerName="istio-proxy" Jul 30 00:50:55.381918 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:55.381776 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b2baa76-311c-457b-b557-c9e729111a45" containerName="istio-proxy" Jul 30 00:50:55.381918 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:55.381781 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b2baa76-311c-457b-b557-c9e729111a45" containerName="istio-proxy" Jul 30 00:50:55.381918 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:55.381792 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f6736bc-86b8-4799-a5ac-cf21540272fa" containerName="istio-proxy" Jul 30 00:50:55.381918 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:55.381798 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f6736bc-86b8-4799-a5ac-cf21540272fa" containerName="istio-proxy" Jul 30 00:50:55.381918 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:55.381897 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b2baa76-311c-457b-b557-c9e729111a45" containerName="istio-proxy" Jul 30 00:50:55.381918 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:55.381908 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f6736bc-86b8-4799-a5ac-cf21540272fa" containerName="istio-proxy" Jul 30 00:50:55.381918 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:55.381914 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b2baa76-311c-457b-b557-c9e729111a45" containerName="istio-proxy" Jul 30 00:50:55.384844 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:55.384827 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z" Jul 30 00:50:55.387786 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:55.387758 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-80d5102b-openshift-default-dockercfg-99wzt\"" Jul 30 00:50:55.399679 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:55.399640 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z"] Jul 30 00:50:55.404219 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:55.404197 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:50:55.489486 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:55.489447 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04a78780-8c67-4097-ad2f-4300cbd28ec7-credential-socket\") pod \"e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z\" (UID: \"04a78780-8c67-4097-ad2f-4300cbd28ec7\") " pod="openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z" Jul 30 00:50:55.489683 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:55.489515 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04a78780-8c67-4097-ad2f-4300cbd28ec7-istio-envoy\") pod \"e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z\" (UID: \"04a78780-8c67-4097-ad2f-4300cbd28ec7\") " pod="openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z" Jul 30 00:50:55.489683 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:55.489543 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04a78780-8c67-4097-ad2f-4300cbd28ec7-workload-socket\") pod \"e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z\" (UID: \"04a78780-8c67-4097-ad2f-4300cbd28ec7\") " pod="openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z" Jul 30 00:50:55.489683 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:55.489605 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04a78780-8c67-4097-ad2f-4300cbd28ec7-istio-token\") pod \"e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z\" (UID: \"04a78780-8c67-4097-ad2f-4300cbd28ec7\") " pod="openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z" Jul 30 00:50:55.489683 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:55.489647 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04a78780-8c67-4097-ad2f-4300cbd28ec7-workload-certs\") pod \"e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z\" (UID: \"04a78780-8c67-4097-ad2f-4300cbd28ec7\") " pod="openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z" Jul 30 00:50:55.489683 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:55.489667 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04a78780-8c67-4097-ad2f-4300cbd28ec7-istio-podinfo\") pod \"e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z\" (UID: \"04a78780-8c67-4097-ad2f-4300cbd28ec7\") " pod="openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z" Jul 30 00:50:55.489929 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:55.489724 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04a78780-8c67-4097-ad2f-4300cbd28ec7-istio-data\") pod \"e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z\" (UID: \"04a78780-8c67-4097-ad2f-4300cbd28ec7\") " pod="openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z" Jul 30 00:50:55.489929 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:55.489799 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmtdf\" (UniqueName: \"kubernetes.io/projected/04a78780-8c67-4097-ad2f-4300cbd28ec7-kube-api-access-vmtdf\") pod \"e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z\" (UID: \"04a78780-8c67-4097-ad2f-4300cbd28ec7\") " pod="openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z" Jul 30 00:50:55.489929 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:55.489872 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04a78780-8c67-4097-ad2f-4300cbd28ec7-istiod-ca-cert\") pod \"e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z\" (UID: \"04a78780-8c67-4097-ad2f-4300cbd28ec7\") " pod="openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z" Jul 30 00:50:55.590720 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:55.590677 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04a78780-8c67-4097-ad2f-4300cbd28ec7-istio-token\") pod \"e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z\" (UID: \"04a78780-8c67-4097-ad2f-4300cbd28ec7\") " pod="openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z" Jul 30 00:50:55.590909 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:55.590759 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04a78780-8c67-4097-ad2f-4300cbd28ec7-workload-certs\") pod \"e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z\" (UID: \"04a78780-8c67-4097-ad2f-4300cbd28ec7\") " pod="openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z" Jul 30 00:50:55.590909 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:55.590786 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04a78780-8c67-4097-ad2f-4300cbd28ec7-istio-podinfo\") pod \"e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z\" (UID: \"04a78780-8c67-4097-ad2f-4300cbd28ec7\") " pod="openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z" Jul 30 00:50:55.590909 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:55.590815 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04a78780-8c67-4097-ad2f-4300cbd28ec7-istio-data\") pod \"e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z\" (UID: \"04a78780-8c67-4097-ad2f-4300cbd28ec7\") " pod="openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z" Jul 30 00:50:55.590909 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:55.590845 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmtdf\" (UniqueName: \"kubernetes.io/projected/04a78780-8c67-4097-ad2f-4300cbd28ec7-kube-api-access-vmtdf\") pod \"e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z\" (UID: \"04a78780-8c67-4097-ad2f-4300cbd28ec7\") " pod="openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z" Jul 30 00:50:55.591157 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:55.590993 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04a78780-8c67-4097-ad2f-4300cbd28ec7-istiod-ca-cert\") pod \"e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z\" (UID: \"04a78780-8c67-4097-ad2f-4300cbd28ec7\") " pod="openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z" Jul 30 00:50:55.591157 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:55.591047 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04a78780-8c67-4097-ad2f-4300cbd28ec7-credential-socket\") pod \"e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z\" (UID: \"04a78780-8c67-4097-ad2f-4300cbd28ec7\") " pod="openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z" Jul 30 00:50:55.591157 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:55.591105 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04a78780-8c67-4097-ad2f-4300cbd28ec7-istio-envoy\") pod \"e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z\" (UID: \"04a78780-8c67-4097-ad2f-4300cbd28ec7\") " pod="openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z" Jul 30 00:50:55.591157 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:55.591133 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04a78780-8c67-4097-ad2f-4300cbd28ec7-workload-socket\") pod \"e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z\" (UID: \"04a78780-8c67-4097-ad2f-4300cbd28ec7\") " pod="openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z" Jul 30 00:50:55.591364 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:55.591186 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04a78780-8c67-4097-ad2f-4300cbd28ec7-workload-certs\") pod \"e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z\" (UID: \"04a78780-8c67-4097-ad2f-4300cbd28ec7\") " pod="openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z" Jul 30 00:50:55.591511 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:55.591453 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04a78780-8c67-4097-ad2f-4300cbd28ec7-credential-socket\") pod \"e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z\" (UID: \"04a78780-8c67-4097-ad2f-4300cbd28ec7\") " pod="openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z" Jul 30 00:50:55.591669 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:55.591612 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04a78780-8c67-4097-ad2f-4300cbd28ec7-istio-data\") pod \"e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z\" (UID: \"04a78780-8c67-4097-ad2f-4300cbd28ec7\") " pod="openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z" Jul 30 00:50:55.591669 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:55.591638 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04a78780-8c67-4097-ad2f-4300cbd28ec7-workload-socket\") pod \"e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z\" (UID: \"04a78780-8c67-4097-ad2f-4300cbd28ec7\") " pod="openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z" Jul 30 00:50:55.591825 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:55.591716 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04a78780-8c67-4097-ad2f-4300cbd28ec7-istiod-ca-cert\") pod \"e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z\" (UID: \"04a78780-8c67-4097-ad2f-4300cbd28ec7\") " pod="openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z" Jul 30 00:50:55.593555 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:55.593527 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04a78780-8c67-4097-ad2f-4300cbd28ec7-istio-envoy\") pod \"e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z\" (UID: \"04a78780-8c67-4097-ad2f-4300cbd28ec7\") " pod="openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z" Jul 30 00:50:55.593680 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:55.593568 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04a78780-8c67-4097-ad2f-4300cbd28ec7-istio-podinfo\") pod \"e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z\" (UID: \"04a78780-8c67-4097-ad2f-4300cbd28ec7\") " pod="openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z" Jul 30 00:50:55.598947 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:55.598924 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04a78780-8c67-4097-ad2f-4300cbd28ec7-istio-token\") pod \"e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z\" (UID: \"04a78780-8c67-4097-ad2f-4300cbd28ec7\") " pod="openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z" Jul 30 00:50:55.599310 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:55.599287 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmtdf\" (UniqueName: \"kubernetes.io/projected/04a78780-8c67-4097-ad2f-4300cbd28ec7-kube-api-access-vmtdf\") pod \"e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z\" (UID: \"04a78780-8c67-4097-ad2f-4300cbd28ec7\") " pod="openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z" Jul 30 00:50:55.700916 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:55.700828 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z" Jul 30 00:50:55.779353 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:55.779312 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:50:55.839702 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:55.839668 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z"] Jul 30 00:50:55.841802 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:50:55.841718 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04a78780_8c67_4097_ad2f_4300cbd28ec7.slice/crio-4cae15484cea7dcd9f2106ba51862ab4dacf9fcfd4ecd497b30825f5056e4fa4 WatchSource:0}: Error finding container 4cae15484cea7dcd9f2106ba51862ab4dacf9fcfd4ecd497b30825f5056e4fa4: Status 404 returned error can't find the container with id 4cae15484cea7dcd9f2106ba51862ab4dacf9fcfd4ecd497b30825f5056e4fa4 Jul 30 00:50:55.843953 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:55.843923 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:50:55.844024 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:55.843995 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:50:55.844060 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:55.844026 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:50:56.654399 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:56.654363 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z" event={"ID":"04a78780-8c67-4097-ad2f-4300cbd28ec7","Type":"ContainerStarted","Data":"cb835c853147f43413f3dde8032f3475d4dc909d64a2ca3bb1a3f706ece32d6c"} Jul 30 00:50:56.654902 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:56.654424 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z" event={"ID":"04a78780-8c67-4097-ad2f-4300cbd28ec7","Type":"ContainerStarted","Data":"4cae15484cea7dcd9f2106ba51862ab4dacf9fcfd4ecd497b30825f5056e4fa4"} Jul 30 00:50:56.675442 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:56.675396 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z" podStartSLOduration=1.675381651 podStartE2EDuration="1.675381651s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:50:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:50:56.673440961 +0000 UTC m=+2020.732196933" watchObservedRunningTime="2026-07-30 00:50:56.675381651 +0000 UTC m=+2020.734137618" Jul 30 00:50:56.701469 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:56.701446 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z" Jul 30 00:50:56.706185 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:56.706162 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z" Jul 30 00:50:57.661187 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:57.661151 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z" Jul 30 00:50:57.662313 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:57.662289 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z" Jul 30 00:50:58.501303 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:58.501268 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:50:58.600239 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:58.600201 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:50:59.722858 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:59.722830 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:50:59.927500 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:50:59.927459 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:51:00.295830 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:51:00.295798 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:51:00.385790 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:51:00.385758 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:51:12.663153 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:51:12.663073 2563 patch_prober.go:28] interesting pod/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 30 00:51:12.663153 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:51:12.663127 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z" podUID="04a78780-8c67-4097-ad2f-4300cbd28ec7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 30 00:51:12.737275 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:51:12.737202 2563 generic.go:356] "Generic (PLEG): container finished" podID="04a78780-8c67-4097-ad2f-4300cbd28ec7" containerID="cb835c853147f43413f3dde8032f3475d4dc909d64a2ca3bb1a3f706ece32d6c" exitCode=0 Jul 30 00:51:12.737401 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:51:12.737277 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z" event={"ID":"04a78780-8c67-4097-ad2f-4300cbd28ec7","Type":"ContainerDied","Data":"cb835c853147f43413f3dde8032f3475d4dc909d64a2ca3bb1a3f706ece32d6c"} Jul 30 00:51:12.737671 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:51:12.737655 2563 scope.go:117] "RemoveContainer" containerID="cb835c853147f43413f3dde8032f3475d4dc909d64a2ca3bb1a3f706ece32d6c" Jul 30 00:51:12.738679 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:51:12.738650 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:51:12.738759 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:51:12.738720 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:51:12.738808 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:51:12.738777 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:51:13.749630 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:51:13.749593 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z" event={"ID":"04a78780-8c67-4097-ad2f-4300cbd28ec7","Type":"ContainerStarted","Data":"87add416922174c7f2a89935b4020f0c971ca97bf86cd19f9d6aa6ef27ba3572"} Jul 30 00:51:14.701102 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:51:14.701070 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z" Jul 30 00:51:14.701325 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:51:14.701300 2563 patch_prober.go:28] interesting pod/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 30 00:51:14.701410 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:51:14.701356 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z" podUID="04a78780-8c67-4097-ad2f-4300cbd28ec7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 30 00:51:15.701172 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:51:15.701138 2563 patch_prober.go:28] interesting pod/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 30 00:51:15.701559 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:51:15.701189 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z" podUID="04a78780-8c67-4097-ad2f-4300cbd28ec7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 30 00:51:16.701879 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:51:16.701839 2563 patch_prober.go:28] interesting pod/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 30 00:51:16.702250 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:51:16.701894 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z" podUID="04a78780-8c67-4097-ad2f-4300cbd28ec7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 30 00:51:17.701485 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:51:17.701448 2563 patch_prober.go:28] interesting pod/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 30 00:51:17.701677 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:51:17.701513 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z" podUID="04a78780-8c67-4097-ad2f-4300cbd28ec7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 30 00:51:18.702204 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:51:18.702161 2563 patch_prober.go:28] interesting pod/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 30 00:51:18.702625 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:51:18.702233 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z" podUID="04a78780-8c67-4097-ad2f-4300cbd28ec7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 30 00:51:19.706020 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:51:19.705988 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z" Jul 30 00:51:19.706771 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:51:19.706392 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z" Jul 30 00:51:19.707197 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:51:19.707180 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z" Jul 30 00:51:23.465225 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:51:23.465173 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:51:29.038092 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:51:29.038059 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:51:30.972267 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:51:30.972232 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:52:07.948779 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:52:07.948747 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:52:08.038249 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:52:08.038197 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:52:08.106832 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:52:08.106784 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:52:11.087697 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:52:11.087654 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:52:16.166234 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:52:16.166173 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:52:16.538605 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:52:16.538578 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-b92tw_0ef8f120-f712-4ed9-adbd-40f785d4aa9d/console-operator/1.log" Jul 30 00:52:16.541529 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:52:16.541490 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-b92tw_0ef8f120-f712-4ed9-adbd-40f785d4aa9d/console-operator/1.log" Jul 30 00:52:49.943296 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:52:49.943250 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:52:53.311862 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:52:53.311830 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:52:53.469953 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:52:53.469917 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:52:58.909992 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:52:58.909945 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:53:04.446638 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:04.446595 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:53:04.548082 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:04.548018 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:53:04.701345 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:04.701258 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z"] Jul 30 00:53:04.701790 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:04.701755 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z" podUID="04a78780-8c67-4097-ad2f-4300cbd28ec7" containerName="istio-proxy" containerID="cri-o://87add416922174c7f2a89935b4020f0c971ca97bf86cd19f9d6aa6ef27ba3572" gracePeriod=30 Jul 30 00:53:04.708650 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:04.708624 2563 patch_prober.go:28] interesting pod/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 30 00:53:04.708784 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:04.708666 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z" podUID="04a78780-8c67-4097-ad2f-4300cbd28ec7" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 30 00:53:04.722342 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:04.722317 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:53:04.936802 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:04.936763 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:53:05.968861 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:05.968827 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:53:06.073572 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:06.073528 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:53:06.165599 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:06.165560 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:53:06.846623 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:06.846589 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:53:06.943417 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:06.943385 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:53:07.037512 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:07.037474 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:53:08.334583 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:08.334542 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:53:09.965784 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:09.965699 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z" Jul 30 00:53:10.052935 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:10.052884 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04a78780-8c67-4097-ad2f-4300cbd28ec7-istiod-ca-cert\") pod \"04a78780-8c67-4097-ad2f-4300cbd28ec7\" (UID: \"04a78780-8c67-4097-ad2f-4300cbd28ec7\") " Jul 30 00:53:10.053109 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:10.052958 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vmtdf\" (UniqueName: \"kubernetes.io/projected/04a78780-8c67-4097-ad2f-4300cbd28ec7-kube-api-access-vmtdf\") pod \"04a78780-8c67-4097-ad2f-4300cbd28ec7\" (UID: \"04a78780-8c67-4097-ad2f-4300cbd28ec7\") " Jul 30 00:53:10.053109 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:10.053015 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04a78780-8c67-4097-ad2f-4300cbd28ec7-istio-envoy\") pod \"04a78780-8c67-4097-ad2f-4300cbd28ec7\" (UID: \"04a78780-8c67-4097-ad2f-4300cbd28ec7\") " Jul 30 00:53:10.053109 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:10.053049 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04a78780-8c67-4097-ad2f-4300cbd28ec7-workload-socket\") pod \"04a78780-8c67-4097-ad2f-4300cbd28ec7\" (UID: \"04a78780-8c67-4097-ad2f-4300cbd28ec7\") " Jul 30 00:53:10.053229 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:10.053116 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04a78780-8c67-4097-ad2f-4300cbd28ec7-istio-token\") pod \"04a78780-8c67-4097-ad2f-4300cbd28ec7\" (UID: \"04a78780-8c67-4097-ad2f-4300cbd28ec7\") " Jul 30 00:53:10.053229 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:10.053154 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04a78780-8c67-4097-ad2f-4300cbd28ec7-workload-certs\") pod \"04a78780-8c67-4097-ad2f-4300cbd28ec7\" (UID: \"04a78780-8c67-4097-ad2f-4300cbd28ec7\") " Jul 30 00:53:10.053229 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:10.053179 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04a78780-8c67-4097-ad2f-4300cbd28ec7-istio-podinfo\") pod \"04a78780-8c67-4097-ad2f-4300cbd28ec7\" (UID: \"04a78780-8c67-4097-ad2f-4300cbd28ec7\") " Jul 30 00:53:10.053392 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:10.053253 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04a78780-8c67-4097-ad2f-4300cbd28ec7-istio-data\") pod \"04a78780-8c67-4097-ad2f-4300cbd28ec7\" (UID: \"04a78780-8c67-4097-ad2f-4300cbd28ec7\") " Jul 30 00:53:10.053392 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:10.053282 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04a78780-8c67-4097-ad2f-4300cbd28ec7-credential-socket\") pod \"04a78780-8c67-4097-ad2f-4300cbd28ec7\" (UID: \"04a78780-8c67-4097-ad2f-4300cbd28ec7\") " Jul 30 00:53:10.053392 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:10.053363 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04a78780-8c67-4097-ad2f-4300cbd28ec7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "04a78780-8c67-4097-ad2f-4300cbd28ec7" (UID: "04a78780-8c67-4097-ad2f-4300cbd28ec7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:53:10.053579 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:10.053310 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04a78780-8c67-4097-ad2f-4300cbd28ec7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "04a78780-8c67-4097-ad2f-4300cbd28ec7" (UID: "04a78780-8c67-4097-ad2f-4300cbd28ec7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 00:53:10.053640 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:10.053574 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04a78780-8c67-4097-ad2f-4300cbd28ec7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "04a78780-8c67-4097-ad2f-4300cbd28ec7" (UID: "04a78780-8c67-4097-ad2f-4300cbd28ec7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:53:10.053699 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:10.053656 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04a78780-8c67-4097-ad2f-4300cbd28ec7-workload-socket\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:53:10.053699 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:10.053677 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04a78780-8c67-4097-ad2f-4300cbd28ec7-istiod-ca-cert\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:53:10.053841 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:10.053713 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04a78780-8c67-4097-ad2f-4300cbd28ec7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "04a78780-8c67-4097-ad2f-4300cbd28ec7" (UID: "04a78780-8c67-4097-ad2f-4300cbd28ec7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:53:10.054255 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:10.054211 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04a78780-8c67-4097-ad2f-4300cbd28ec7-istio-data" (OuterVolumeSpecName: "istio-data") pod "04a78780-8c67-4097-ad2f-4300cbd28ec7" (UID: "04a78780-8c67-4097-ad2f-4300cbd28ec7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:53:10.055568 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:10.055539 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04a78780-8c67-4097-ad2f-4300cbd28ec7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "04a78780-8c67-4097-ad2f-4300cbd28ec7" (UID: "04a78780-8c67-4097-ad2f-4300cbd28ec7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:53:10.055973 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:10.055940 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04a78780-8c67-4097-ad2f-4300cbd28ec7-istio-token" (OuterVolumeSpecName: "istio-token") pod "04a78780-8c67-4097-ad2f-4300cbd28ec7" (UID: "04a78780-8c67-4097-ad2f-4300cbd28ec7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 00:53:10.056062 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:10.055971 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/04a78780-8c67-4097-ad2f-4300cbd28ec7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "04a78780-8c67-4097-ad2f-4300cbd28ec7" (UID: "04a78780-8c67-4097-ad2f-4300cbd28ec7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 30 00:53:10.056062 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:10.055977 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04a78780-8c67-4097-ad2f-4300cbd28ec7-kube-api-access-vmtdf" (OuterVolumeSpecName: "kube-api-access-vmtdf") pod "04a78780-8c67-4097-ad2f-4300cbd28ec7" (UID: "04a78780-8c67-4097-ad2f-4300cbd28ec7"). InnerVolumeSpecName "kube-api-access-vmtdf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 00:53:10.154485 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:10.154444 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04a78780-8c67-4097-ad2f-4300cbd28ec7-istio-data\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:53:10.154485 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:10.154486 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04a78780-8c67-4097-ad2f-4300cbd28ec7-credential-socket\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:53:10.154692 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:10.154502 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vmtdf\" (UniqueName: \"kubernetes.io/projected/04a78780-8c67-4097-ad2f-4300cbd28ec7-kube-api-access-vmtdf\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:53:10.154692 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:10.154516 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04a78780-8c67-4097-ad2f-4300cbd28ec7-istio-envoy\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:53:10.154692 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:10.154529 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04a78780-8c67-4097-ad2f-4300cbd28ec7-istio-token\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:53:10.154692 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:10.154540 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04a78780-8c67-4097-ad2f-4300cbd28ec7-workload-certs\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:53:10.154692 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:10.154552 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04a78780-8c67-4097-ad2f-4300cbd28ec7-istio-podinfo\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:53:10.352899 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:10.352865 2563 generic.go:356] "Generic (PLEG): container finished" podID="04a78780-8c67-4097-ad2f-4300cbd28ec7" containerID="87add416922174c7f2a89935b4020f0c971ca97bf86cd19f9d6aa6ef27ba3572" exitCode=0 Jul 30 00:53:10.353065 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:10.352936 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z" Jul 30 00:53:10.353065 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:10.352955 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z" event={"ID":"04a78780-8c67-4097-ad2f-4300cbd28ec7","Type":"ContainerDied","Data":"87add416922174c7f2a89935b4020f0c971ca97bf86cd19f9d6aa6ef27ba3572"} Jul 30 00:53:10.353065 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:10.353050 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z" event={"ID":"04a78780-8c67-4097-ad2f-4300cbd28ec7","Type":"ContainerDied","Data":"4cae15484cea7dcd9f2106ba51862ab4dacf9fcfd4ecd497b30825f5056e4fa4"} Jul 30 00:53:10.353222 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:10.353080 2563 scope.go:117] "RemoveContainer" containerID="87add416922174c7f2a89935b4020f0c971ca97bf86cd19f9d6aa6ef27ba3572" Jul 30 00:53:10.363377 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:10.363342 2563 scope.go:117] "RemoveContainer" containerID="cb835c853147f43413f3dde8032f3475d4dc909d64a2ca3bb1a3f706ece32d6c" Jul 30 00:53:10.373768 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:10.373727 2563 scope.go:117] "RemoveContainer" containerID="87add416922174c7f2a89935b4020f0c971ca97bf86cd19f9d6aa6ef27ba3572" Jul 30 00:53:10.374062 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:53:10.374035 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87add416922174c7f2a89935b4020f0c971ca97bf86cd19f9d6aa6ef27ba3572\": container with ID starting with 87add416922174c7f2a89935b4020f0c971ca97bf86cd19f9d6aa6ef27ba3572 not found: ID does not exist" containerID="87add416922174c7f2a89935b4020f0c971ca97bf86cd19f9d6aa6ef27ba3572" Jul 30 00:53:10.374152 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:10.374069 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87add416922174c7f2a89935b4020f0c971ca97bf86cd19f9d6aa6ef27ba3572"} err="failed to get container status \"87add416922174c7f2a89935b4020f0c971ca97bf86cd19f9d6aa6ef27ba3572\": rpc error: code = NotFound desc = could not find container \"87add416922174c7f2a89935b4020f0c971ca97bf86cd19f9d6aa6ef27ba3572\": container with ID starting with 87add416922174c7f2a89935b4020f0c971ca97bf86cd19f9d6aa6ef27ba3572 not found: ID does not exist" Jul 30 00:53:10.374152 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:10.374093 2563 scope.go:117] "RemoveContainer" containerID="cb835c853147f43413f3dde8032f3475d4dc909d64a2ca3bb1a3f706ece32d6c" Jul 30 00:53:10.374368 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:53:10.374349 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb835c853147f43413f3dde8032f3475d4dc909d64a2ca3bb1a3f706ece32d6c\": container with ID starting with cb835c853147f43413f3dde8032f3475d4dc909d64a2ca3bb1a3f706ece32d6c not found: ID does not exist" containerID="cb835c853147f43413f3dde8032f3475d4dc909d64a2ca3bb1a3f706ece32d6c" Jul 30 00:53:10.374408 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:10.374384 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb835c853147f43413f3dde8032f3475d4dc909d64a2ca3bb1a3f706ece32d6c"} err="failed to get container status \"cb835c853147f43413f3dde8032f3475d4dc909d64a2ca3bb1a3f706ece32d6c\": rpc error: code = NotFound desc = could not find container \"cb835c853147f43413f3dde8032f3475d4dc909d64a2ca3bb1a3f706ece32d6c\": container with ID starting with cb835c853147f43413f3dde8032f3475d4dc909d64a2ca3bb1a3f706ece32d6c not found: ID does not exist" Jul 30 00:53:10.382055 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:10.382031 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z"] Jul 30 00:53:10.394827 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:10.394801 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-80d5102b-openshift-default-7cdbc97dd4-nm29z"] Jul 30 00:53:10.476016 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:10.475980 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04a78780-8c67-4097-ad2f-4300cbd28ec7" path="/var/lib/kubelet/pods/04a78780-8c67-4097-ad2f-4300cbd28ec7/volumes" Jul 30 00:53:14.083476 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:14.083442 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:53:14.184288 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:14.184257 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:53:14.737289 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:14.737255 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:53:20.474426 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:20.474380 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:53:20.575231 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:20.575177 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:53:20.905119 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:20.905087 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:53:31.150387 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:31.150349 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:53:31.473219 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:31.473121 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:53:52.377224 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:52.377186 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:53:57.112423 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:57.112382 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:53:57.212347 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:57.212303 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:53:58.576593 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:58.576546 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:53:58.676292 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:58.676259 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:53:58.898109 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:58.898018 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:53:59.173394 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:59.173302 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk"] Jul 30 00:53:59.173949 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:59.173922 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04a78780-8c67-4097-ad2f-4300cbd28ec7" containerName="istio-proxy" Jul 30 00:53:59.173949 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:59.173946 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="04a78780-8c67-4097-ad2f-4300cbd28ec7" containerName="istio-proxy" Jul 30 00:53:59.174151 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:59.174060 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="04a78780-8c67-4097-ad2f-4300cbd28ec7" containerName="istio-proxy" Jul 30 00:53:59.174151 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:59.174076 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="04a78780-8c67-4097-ad2f-4300cbd28ec7" containerName="istio-proxy" Jul 30 00:53:59.177301 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:59.177276 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk" Jul 30 00:53:59.179804 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:59.179784 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-2239f6-openshift-default-dockercfg-w24hj\"" Jul 30 00:53:59.187146 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:59.187121 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk"] Jul 30 00:53:59.197138 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:59.197110 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:53:59.285689 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:59.285655 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c65793d5-e47c-4045-b512-ce00ac69753c-workload-socket\") pod \"e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk\" (UID: \"c65793d5-e47c-4045-b512-ce00ac69753c\") " pod="openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk" Jul 30 00:53:59.285887 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:59.285700 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c65793d5-e47c-4045-b512-ce00ac69753c-istio-podinfo\") pod \"e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk\" (UID: \"c65793d5-e47c-4045-b512-ce00ac69753c\") " pod="openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk" Jul 30 00:53:59.285887 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:59.285821 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c65793d5-e47c-4045-b512-ce00ac69753c-istio-data\") pod \"e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk\" (UID: \"c65793d5-e47c-4045-b512-ce00ac69753c\") " pod="openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk" Jul 30 00:53:59.285887 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:59.285862 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c65793d5-e47c-4045-b512-ce00ac69753c-istio-envoy\") pod \"e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk\" (UID: \"c65793d5-e47c-4045-b512-ce00ac69753c\") " pod="openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk" Jul 30 00:53:59.286011 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:59.285891 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbzxz\" (UniqueName: \"kubernetes.io/projected/c65793d5-e47c-4045-b512-ce00ac69753c-kube-api-access-cbzxz\") pod \"e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk\" (UID: \"c65793d5-e47c-4045-b512-ce00ac69753c\") " pod="openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk" Jul 30 00:53:59.286011 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:59.285926 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c65793d5-e47c-4045-b512-ce00ac69753c-istiod-ca-cert\") pod \"e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk\" (UID: \"c65793d5-e47c-4045-b512-ce00ac69753c\") " pod="openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk" Jul 30 00:53:59.286011 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:59.285967 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c65793d5-e47c-4045-b512-ce00ac69753c-istio-token\") pod \"e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk\" (UID: \"c65793d5-e47c-4045-b512-ce00ac69753c\") " pod="openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk" Jul 30 00:53:59.286011 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:59.286004 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c65793d5-e47c-4045-b512-ce00ac69753c-credential-socket\") pod \"e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk\" (UID: \"c65793d5-e47c-4045-b512-ce00ac69753c\") " pod="openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk" Jul 30 00:53:59.286165 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:59.286151 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c65793d5-e47c-4045-b512-ce00ac69753c-workload-certs\") pod \"e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk\" (UID: \"c65793d5-e47c-4045-b512-ce00ac69753c\") " pod="openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk" Jul 30 00:53:59.387192 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:59.387150 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c65793d5-e47c-4045-b512-ce00ac69753c-workload-certs\") pod \"e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk\" (UID: \"c65793d5-e47c-4045-b512-ce00ac69753c\") " pod="openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk" Jul 30 00:53:59.387192 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:59.387192 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c65793d5-e47c-4045-b512-ce00ac69753c-workload-socket\") pod \"e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk\" (UID: \"c65793d5-e47c-4045-b512-ce00ac69753c\") " pod="openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk" Jul 30 00:53:59.387452 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:59.387215 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c65793d5-e47c-4045-b512-ce00ac69753c-istio-podinfo\") pod \"e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk\" (UID: \"c65793d5-e47c-4045-b512-ce00ac69753c\") " pod="openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk" Jul 30 00:53:59.387452 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:59.387345 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c65793d5-e47c-4045-b512-ce00ac69753c-istio-data\") pod \"e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk\" (UID: \"c65793d5-e47c-4045-b512-ce00ac69753c\") " pod="openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk" Jul 30 00:53:59.387452 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:59.387392 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c65793d5-e47c-4045-b512-ce00ac69753c-istio-envoy\") pod \"e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk\" (UID: \"c65793d5-e47c-4045-b512-ce00ac69753c\") " pod="openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk" Jul 30 00:53:59.387452 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:59.387420 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbzxz\" (UniqueName: \"kubernetes.io/projected/c65793d5-e47c-4045-b512-ce00ac69753c-kube-api-access-cbzxz\") pod \"e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk\" (UID: \"c65793d5-e47c-4045-b512-ce00ac69753c\") " pod="openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk" Jul 30 00:53:59.387452 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:59.387448 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c65793d5-e47c-4045-b512-ce00ac69753c-istiod-ca-cert\") pod \"e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk\" (UID: \"c65793d5-e47c-4045-b512-ce00ac69753c\") " pod="openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk" Jul 30 00:53:59.387707 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:59.387474 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c65793d5-e47c-4045-b512-ce00ac69753c-istio-token\") pod \"e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk\" (UID: \"c65793d5-e47c-4045-b512-ce00ac69753c\") " pod="openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk" Jul 30 00:53:59.387707 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:59.387513 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c65793d5-e47c-4045-b512-ce00ac69753c-credential-socket\") pod \"e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk\" (UID: \"c65793d5-e47c-4045-b512-ce00ac69753c\") " pod="openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk" Jul 30 00:53:59.387707 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:59.387588 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c65793d5-e47c-4045-b512-ce00ac69753c-workload-socket\") pod \"e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk\" (UID: \"c65793d5-e47c-4045-b512-ce00ac69753c\") " pod="openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk" Jul 30 00:53:59.387707 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:59.387626 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c65793d5-e47c-4045-b512-ce00ac69753c-workload-certs\") pod \"e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk\" (UID: \"c65793d5-e47c-4045-b512-ce00ac69753c\") " pod="openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk" Jul 30 00:53:59.387707 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:59.387671 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c65793d5-e47c-4045-b512-ce00ac69753c-istio-data\") pod \"e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk\" (UID: \"c65793d5-e47c-4045-b512-ce00ac69753c\") " pod="openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk" Jul 30 00:53:59.388091 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:59.388071 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c65793d5-e47c-4045-b512-ce00ac69753c-credential-socket\") pod \"e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk\" (UID: \"c65793d5-e47c-4045-b512-ce00ac69753c\") " pod="openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk" Jul 30 00:53:59.388263 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:59.388240 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c65793d5-e47c-4045-b512-ce00ac69753c-istiod-ca-cert\") pod \"e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk\" (UID: \"c65793d5-e47c-4045-b512-ce00ac69753c\") " pod="openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk" Jul 30 00:53:59.389805 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:59.389783 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c65793d5-e47c-4045-b512-ce00ac69753c-istio-podinfo\") pod \"e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk\" (UID: \"c65793d5-e47c-4045-b512-ce00ac69753c\") " pod="openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk" Jul 30 00:53:59.389895 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:59.389821 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c65793d5-e47c-4045-b512-ce00ac69753c-istio-envoy\") pod \"e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk\" (UID: \"c65793d5-e47c-4045-b512-ce00ac69753c\") " pod="openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk" Jul 30 00:53:59.394850 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:59.394826 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c65793d5-e47c-4045-b512-ce00ac69753c-istio-token\") pod \"e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk\" (UID: \"c65793d5-e47c-4045-b512-ce00ac69753c\") " pod="openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk" Jul 30 00:53:59.394941 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:59.394919 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbzxz\" (UniqueName: \"kubernetes.io/projected/c65793d5-e47c-4045-b512-ce00ac69753c-kube-api-access-cbzxz\") pod \"e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk\" (UID: \"c65793d5-e47c-4045-b512-ce00ac69753c\") " pod="openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk" Jul 30 00:53:59.488835 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:59.488724 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk" Jul 30 00:53:59.526374 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:59.526336 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:53:59.639852 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:59.639824 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk"] Jul 30 00:53:59.641848 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:53:59.641818 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc65793d5_e47c_4045_b512_ce00ac69753c.slice/crio-3e775b89925fe60668a5b3e7c52101164f7b626829fb3db0f85ba5f180f86078 WatchSource:0}: Error finding container 3e775b89925fe60668a5b3e7c52101164f7b626829fb3db0f85ba5f180f86078: Status 404 returned error can't find the container with id 3e775b89925fe60668a5b3e7c52101164f7b626829fb3db0f85ba5f180f86078 Jul 30 00:53:59.644000 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:59.643961 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:53:59.644102 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:59.644030 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:53:59.644102 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:53:59.644072 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:54:00.614589 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:00.614557 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk" event={"ID":"c65793d5-e47c-4045-b512-ce00ac69753c","Type":"ContainerStarted","Data":"b5f0a8f4bc4203cb16a7d789860c5665ce4cca56d0d341a8ce73ca5dedc77b1a"} Jul 30 00:54:00.614781 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:00.614612 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk" event={"ID":"c65793d5-e47c-4045-b512-ce00ac69753c","Type":"ContainerStarted","Data":"3e775b89925fe60668a5b3e7c52101164f7b626829fb3db0f85ba5f180f86078"} Jul 30 00:54:00.633729 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:00.633686 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk" podStartSLOduration=1.6336719899999999 podStartE2EDuration="1.63367199s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:53:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:54:00.63238044 +0000 UTC m=+2204.691136407" watchObservedRunningTime="2026-07-30 00:54:00.63367199 +0000 UTC m=+2204.692427958" Jul 30 00:54:01.489379 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:01.489340 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk" Jul 30 00:54:01.494089 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:01.494061 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk" Jul 30 00:54:01.620084 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:01.620055 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk" Jul 30 00:54:01.621036 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:01.621011 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk" Jul 30 00:54:02.523177 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:02.523138 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:54:02.628597 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:02.628565 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:54:03.729897 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:03.729863 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:54:04.153215 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:04.153177 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:54:04.256168 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:04.256134 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:54:04.355416 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:04.355377 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:54:16.689249 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:16.689139 2563 patch_prober.go:28] interesting pod/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": read tcp 10.132.0.2:50248->10.132.0.56:15021: read: connection reset by peer" start-of-body= Jul 30 00:54:16.689249 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:16.689228 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk" podUID="c65793d5-e47c-4045-b512-ce00ac69753c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": read tcp 10.132.0.2:50248->10.132.0.56:15021: read: connection reset by peer" Jul 30 00:54:17.707265 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:17.707231 2563 generic.go:356] "Generic (PLEG): container finished" podID="c65793d5-e47c-4045-b512-ce00ac69753c" containerID="b5f0a8f4bc4203cb16a7d789860c5665ce4cca56d0d341a8ce73ca5dedc77b1a" exitCode=0 Jul 30 00:54:17.707636 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:17.707302 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk" event={"ID":"c65793d5-e47c-4045-b512-ce00ac69753c","Type":"ContainerDied","Data":"b5f0a8f4bc4203cb16a7d789860c5665ce4cca56d0d341a8ce73ca5dedc77b1a"} Jul 30 00:54:17.707684 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:17.707664 2563 scope.go:117] "RemoveContainer" containerID="b5f0a8f4bc4203cb16a7d789860c5665ce4cca56d0d341a8ce73ca5dedc77b1a" Jul 30 00:54:17.708705 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:17.708677 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:54:17.708794 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:17.708775 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:54:17.708840 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:17.708823 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:54:18.714130 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:18.714094 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk" event={"ID":"c65793d5-e47c-4045-b512-ce00ac69753c","Type":"ContainerStarted","Data":"fa17916eb897dad7d65567fb0c771dc203d3164b4b80afd1334a92ef56dabbcd"} Jul 30 00:54:19.488937 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:19.488891 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk" Jul 30 00:54:19.489198 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:19.489172 2563 patch_prober.go:28] interesting pod/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 30 00:54:19.489292 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:19.489241 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk" podUID="c65793d5-e47c-4045-b512-ce00ac69753c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 30 00:54:20.489343 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:20.489308 2563 patch_prober.go:28] interesting pod/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 30 00:54:20.489781 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:20.489358 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk" podUID="c65793d5-e47c-4045-b512-ce00ac69753c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 30 00:54:21.489136 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:21.489100 2563 patch_prober.go:28] interesting pod/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 30 00:54:21.489317 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:21.489161 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk" podUID="c65793d5-e47c-4045-b512-ce00ac69753c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 30 00:54:22.489490 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:22.489449 2563 patch_prober.go:28] interesting pod/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 30 00:54:22.489932 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:22.489524 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk" podUID="c65793d5-e47c-4045-b512-ce00ac69753c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 30 00:54:23.492486 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:23.492453 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk" Jul 30 00:54:23.492961 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:23.492684 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk" Jul 30 00:54:23.493412 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:23.493396 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk" Jul 30 00:54:27.484958 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:27.484918 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:54:27.805019 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:27.804979 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:54:28.077325 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:28.077244 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn"] Jul 30 00:54:28.077834 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:28.077818 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04a78780-8c67-4097-ad2f-4300cbd28ec7" containerName="istio-proxy" Jul 30 00:54:28.077903 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:28.077836 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="04a78780-8c67-4097-ad2f-4300cbd28ec7" containerName="istio-proxy" Jul 30 00:54:28.080945 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:28.080925 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn" Jul 30 00:54:28.083235 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:28.083218 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-6898ea-openshift-default-dockercfg-fdrgk\"" Jul 30 00:54:28.092151 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:28.092130 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn"] Jul 30 00:54:28.135777 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:28.135749 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:54:28.139820 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:28.139801 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2-istio-podinfo\") pod \"e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn\" (UID: \"f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2\") " pod="openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn" Jul 30 00:54:28.139913 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:28.139841 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2-workload-socket\") pod \"e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn\" (UID: \"f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2\") " pod="openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn" Jul 30 00:54:28.139913 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:28.139861 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tf96t\" (UniqueName: \"kubernetes.io/projected/f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2-kube-api-access-tf96t\") pod \"e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn\" (UID: \"f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2\") " pod="openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn" Jul 30 00:54:28.139913 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:28.139884 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2-istio-data\") pod \"e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn\" (UID: \"f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2\") " pod="openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn" Jul 30 00:54:28.140093 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:28.139915 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2-credential-socket\") pod \"e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn\" (UID: \"f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2\") " pod="openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn" Jul 30 00:54:28.140093 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:28.139931 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2-istiod-ca-cert\") pod \"e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn\" (UID: \"f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2\") " pod="openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn" Jul 30 00:54:28.140093 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:28.139957 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2-istio-envoy\") pod \"e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn\" (UID: \"f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2\") " pod="openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn" Jul 30 00:54:28.140093 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:28.139984 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2-workload-certs\") pod \"e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn\" (UID: \"f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2\") " pod="openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn" Jul 30 00:54:28.140093 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:28.140001 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2-istio-token\") pod \"e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn\" (UID: \"f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2\") " pod="openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn" Jul 30 00:54:28.241084 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:28.241049 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2-workload-socket\") pod \"e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn\" (UID: \"f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2\") " pod="openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn" Jul 30 00:54:28.241084 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:28.241088 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tf96t\" (UniqueName: \"kubernetes.io/projected/f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2-kube-api-access-tf96t\") pod \"e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn\" (UID: \"f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2\") " pod="openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn" Jul 30 00:54:28.241332 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:28.241106 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2-istio-data\") pod \"e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn\" (UID: \"f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2\") " pod="openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn" Jul 30 00:54:28.241332 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:28.241136 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2-credential-socket\") pod \"e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn\" (UID: \"f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2\") " pod="openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn" Jul 30 00:54:28.241332 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:28.241153 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2-istiod-ca-cert\") pod \"e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn\" (UID: \"f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2\") " pod="openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn" Jul 30 00:54:28.241332 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:28.241180 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2-istio-envoy\") pod \"e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn\" (UID: \"f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2\") " pod="openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn" Jul 30 00:54:28.241332 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:28.241213 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2-workload-certs\") pod \"e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn\" (UID: \"f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2\") " pod="openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn" Jul 30 00:54:28.241332 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:28.241238 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2-istio-token\") pod \"e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn\" (UID: \"f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2\") " pod="openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn" Jul 30 00:54:28.241643 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:28.241345 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2-istio-podinfo\") pod \"e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn\" (UID: \"f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2\") " pod="openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn" Jul 30 00:54:28.241643 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:28.241461 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2-workload-socket\") pod \"e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn\" (UID: \"f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2\") " pod="openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn" Jul 30 00:54:28.241643 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:28.241510 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2-istio-data\") pod \"e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn\" (UID: \"f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2\") " pod="openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn" Jul 30 00:54:28.241643 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:28.241589 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2-credential-socket\") pod \"e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn\" (UID: \"f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2\") " pod="openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn" Jul 30 00:54:28.241977 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:28.241948 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2-workload-certs\") pod \"e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn\" (UID: \"f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2\") " pod="openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn" Jul 30 00:54:28.242306 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:28.242284 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2-istiod-ca-cert\") pod \"e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn\" (UID: \"f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2\") " pod="openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn" Jul 30 00:54:28.243554 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:28.243533 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2-istio-envoy\") pod \"e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn\" (UID: \"f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2\") " pod="openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn" Jul 30 00:54:28.244218 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:28.244197 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2-istio-podinfo\") pod \"e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn\" (UID: \"f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2\") " pod="openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn" Jul 30 00:54:28.248866 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:28.248842 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2-istio-token\") pod \"e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn\" (UID: \"f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2\") " pod="openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn" Jul 30 00:54:28.249240 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:28.249218 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tf96t\" (UniqueName: \"kubernetes.io/projected/f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2-kube-api-access-tf96t\") pod \"e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn\" (UID: \"f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2\") " pod="openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn" Jul 30 00:54:28.337769 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:28.337675 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:54:28.392746 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:28.392713 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn" Jul 30 00:54:28.568485 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:28.568461 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn"] Jul 30 00:54:28.570554 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:54:28.570527 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9ea2c30_3e64_49fc_8f4d_d03339b3f8c2.slice/crio-97443f780964cd6793ec46d59ec0160106ed22231dd382a54f2ab1370b4574ad WatchSource:0}: Error finding container 97443f780964cd6793ec46d59ec0160106ed22231dd382a54f2ab1370b4574ad: Status 404 returned error can't find the container with id 97443f780964cd6793ec46d59ec0160106ed22231dd382a54f2ab1370b4574ad Jul 30 00:54:28.572629 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:28.572597 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:54:28.572722 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:28.572657 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:54:28.572722 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:28.572688 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:54:28.765827 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:28.765790 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn" event={"ID":"f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2","Type":"ContainerStarted","Data":"f367bd939dae275b398dc6eb18ca76df1a9427fe69fa9e2630e2d794cdf92913"} Jul 30 00:54:28.766010 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:28.765854 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn" event={"ID":"f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2","Type":"ContainerStarted","Data":"97443f780964cd6793ec46d59ec0160106ed22231dd382a54f2ab1370b4574ad"} Jul 30 00:54:28.786957 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:28.786900 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn" podStartSLOduration=0.786886047 podStartE2EDuration="786.886047ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:54:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:54:28.785564995 +0000 UTC m=+2232.844320963" watchObservedRunningTime="2026-07-30 00:54:28.786886047 +0000 UTC m=+2232.845642014" Jul 30 00:54:29.393632 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:29.393595 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn" Jul 30 00:54:29.398166 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:29.398142 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn" Jul 30 00:54:29.771618 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:29.771586 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn" Jul 30 00:54:29.772509 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:29.772485 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn" Jul 30 00:54:30.646506 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:30.646468 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:54:30.745926 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:30.745898 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:54:31.930436 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:31.930401 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:54:32.348130 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:32.348090 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:54:32.447444 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:32.447411 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:54:32.518665 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:32.518621 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:54:45.727036 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:45.725861 2563 patch_prober.go:28] interesting pod/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": read tcp 10.132.0.2:58550->10.132.0.57:15021: read: connection reset by peer" start-of-body= Jul 30 00:54:45.727036 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:45.725925 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn" podUID="f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": read tcp 10.132.0.2:58550->10.132.0.57:15021: read: connection reset by peer" Jul 30 00:54:45.855942 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:45.855902 2563 generic.go:356] "Generic (PLEG): container finished" podID="f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2" containerID="f367bd939dae275b398dc6eb18ca76df1a9427fe69fa9e2630e2d794cdf92913" exitCode=0 Jul 30 00:54:45.856144 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:45.855980 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn" event={"ID":"f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2","Type":"ContainerDied","Data":"f367bd939dae275b398dc6eb18ca76df1a9427fe69fa9e2630e2d794cdf92913"} Jul 30 00:54:45.856390 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:45.856373 2563 scope.go:117] "RemoveContainer" containerID="f367bd939dae275b398dc6eb18ca76df1a9427fe69fa9e2630e2d794cdf92913" Jul 30 00:54:45.857443 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:45.857418 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:54:45.857517 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:45.857482 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:54:45.857574 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:45.857528 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:54:46.866457 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:46.866414 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn" event={"ID":"f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2","Type":"ContainerStarted","Data":"15f5b31d46c8a2487e57e09feda93a98a73e5946fd0a9888b7a2a3b501bd9f0d"} Jul 30 00:54:47.393471 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:47.393430 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn" Jul 30 00:54:47.393765 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:47.393635 2563 patch_prober.go:28] interesting pod/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 30 00:54:47.393765 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:47.393691 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn" podUID="f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 30 00:54:48.394004 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:48.393968 2563 patch_prober.go:28] interesting pod/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 30 00:54:48.394404 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:48.394036 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn" podUID="f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 30 00:54:49.393566 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:49.393526 2563 patch_prober.go:28] interesting pod/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 30 00:54:49.393793 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:49.393592 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn" podUID="f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 30 00:54:50.393927 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:50.393882 2563 patch_prober.go:28] interesting pod/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 30 00:54:50.394342 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:50.393957 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn" podUID="f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 30 00:54:51.396460 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:51.396430 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn" Jul 30 00:54:51.396837 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:51.396768 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn" Jul 30 00:54:51.397581 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:51.397562 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn" Jul 30 00:54:55.578201 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:54:55.578164 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:55:01.703129 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:55:01.703098 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:56:01.532194 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:56:01.532125 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:56:40.525882 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:56:40.525788 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:56:41.412229 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:56:41.412162 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:56:52.056302 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:56:52.056253 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:56:52.125254 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:56:52.125218 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:56:52.191079 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:56:52.191043 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:56:52.255784 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:56:52.255077 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:56:52.589257 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:56:52.589224 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:56:58.490198 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:56:58.490165 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:56:58.559332 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:56:58.559295 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:56:58.718943 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:56:58.718908 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk"] Jul 30 00:56:58.719166 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:56:58.719141 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk" podUID="c65793d5-e47c-4045-b512-ce00ac69753c" containerName="istio-proxy" containerID="cri-o://fa17916eb897dad7d65567fb0c771dc203d3164b4b80afd1334a92ef56dabbcd" gracePeriod=30 Jul 30 00:56:58.733501 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:56:58.733474 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:56:58.956781 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:56:58.956744 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:57:03.981933 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:03.981872 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk" Jul 30 00:57:04.128814 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:04.128780 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c65793d5-e47c-4045-b512-ce00ac69753c-workload-certs\") pod \"c65793d5-e47c-4045-b512-ce00ac69753c\" (UID: \"c65793d5-e47c-4045-b512-ce00ac69753c\") " Jul 30 00:57:04.129013 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:04.128838 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c65793d5-e47c-4045-b512-ce00ac69753c-istio-token\") pod \"c65793d5-e47c-4045-b512-ce00ac69753c\" (UID: \"c65793d5-e47c-4045-b512-ce00ac69753c\") " Jul 30 00:57:04.129013 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:04.128878 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c65793d5-e47c-4045-b512-ce00ac69753c-istio-data\") pod \"c65793d5-e47c-4045-b512-ce00ac69753c\" (UID: \"c65793d5-e47c-4045-b512-ce00ac69753c\") " Jul 30 00:57:04.129013 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:04.128925 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cbzxz\" (UniqueName: \"kubernetes.io/projected/c65793d5-e47c-4045-b512-ce00ac69753c-kube-api-access-cbzxz\") pod \"c65793d5-e47c-4045-b512-ce00ac69753c\" (UID: \"c65793d5-e47c-4045-b512-ce00ac69753c\") " Jul 30 00:57:04.129013 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:04.128957 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c65793d5-e47c-4045-b512-ce00ac69753c-istio-envoy\") pod \"c65793d5-e47c-4045-b512-ce00ac69753c\" (UID: \"c65793d5-e47c-4045-b512-ce00ac69753c\") " Jul 30 00:57:04.129245 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:04.129015 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c65793d5-e47c-4045-b512-ce00ac69753c-credential-socket\") pod \"c65793d5-e47c-4045-b512-ce00ac69753c\" (UID: \"c65793d5-e47c-4045-b512-ce00ac69753c\") " Jul 30 00:57:04.129245 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:04.129067 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c65793d5-e47c-4045-b512-ce00ac69753c-workload-socket\") pod \"c65793d5-e47c-4045-b512-ce00ac69753c\" (UID: \"c65793d5-e47c-4045-b512-ce00ac69753c\") " Jul 30 00:57:04.129245 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:04.129090 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c65793d5-e47c-4045-b512-ce00ac69753c-istiod-ca-cert\") pod \"c65793d5-e47c-4045-b512-ce00ac69753c\" (UID: \"c65793d5-e47c-4045-b512-ce00ac69753c\") " Jul 30 00:57:04.129245 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:04.129091 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c65793d5-e47c-4045-b512-ce00ac69753c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c65793d5-e47c-4045-b512-ce00ac69753c" (UID: "c65793d5-e47c-4045-b512-ce00ac69753c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:57:04.129245 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:04.129150 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c65793d5-e47c-4045-b512-ce00ac69753c-istio-podinfo\") pod \"c65793d5-e47c-4045-b512-ce00ac69753c\" (UID: \"c65793d5-e47c-4045-b512-ce00ac69753c\") " Jul 30 00:57:04.129522 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:04.129304 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c65793d5-e47c-4045-b512-ce00ac69753c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c65793d5-e47c-4045-b512-ce00ac69753c" (UID: "c65793d5-e47c-4045-b512-ce00ac69753c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:57:04.129522 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:04.129487 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c65793d5-e47c-4045-b512-ce00ac69753c-credential-socket\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:57:04.129522 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:04.129506 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c65793d5-e47c-4045-b512-ce00ac69753c-workload-certs\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:57:04.129796 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:04.129706 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c65793d5-e47c-4045-b512-ce00ac69753c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c65793d5-e47c-4045-b512-ce00ac69753c" (UID: "c65793d5-e47c-4045-b512-ce00ac69753c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:57:04.129989 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:04.129944 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c65793d5-e47c-4045-b512-ce00ac69753c-istio-data" (OuterVolumeSpecName: "istio-data") pod "c65793d5-e47c-4045-b512-ce00ac69753c" (UID: "c65793d5-e47c-4045-b512-ce00ac69753c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:57:04.129989 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:04.129954 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c65793d5-e47c-4045-b512-ce00ac69753c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c65793d5-e47c-4045-b512-ce00ac69753c" (UID: "c65793d5-e47c-4045-b512-ce00ac69753c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 00:57:04.131370 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:04.131346 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c65793d5-e47c-4045-b512-ce00ac69753c-istio-token" (OuterVolumeSpecName: "istio-token") pod "c65793d5-e47c-4045-b512-ce00ac69753c" (UID: "c65793d5-e47c-4045-b512-ce00ac69753c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 00:57:04.131617 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:04.131591 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c65793d5-e47c-4045-b512-ce00ac69753c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c65793d5-e47c-4045-b512-ce00ac69753c" (UID: "c65793d5-e47c-4045-b512-ce00ac69753c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:57:04.131700 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:04.131669 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c65793d5-e47c-4045-b512-ce00ac69753c-kube-api-access-cbzxz" (OuterVolumeSpecName: "kube-api-access-cbzxz") pod "c65793d5-e47c-4045-b512-ce00ac69753c" (UID: "c65793d5-e47c-4045-b512-ce00ac69753c"). InnerVolumeSpecName "kube-api-access-cbzxz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 00:57:04.131770 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:04.131701 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c65793d5-e47c-4045-b512-ce00ac69753c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c65793d5-e47c-4045-b512-ce00ac69753c" (UID: "c65793d5-e47c-4045-b512-ce00ac69753c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 30 00:57:04.230191 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:04.230157 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c65793d5-e47c-4045-b512-ce00ac69753c-workload-socket\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:57:04.230191 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:04.230185 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c65793d5-e47c-4045-b512-ce00ac69753c-istiod-ca-cert\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:57:04.230191 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:04.230196 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c65793d5-e47c-4045-b512-ce00ac69753c-istio-podinfo\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:57:04.230425 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:04.230204 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c65793d5-e47c-4045-b512-ce00ac69753c-istio-token\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:57:04.230425 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:04.230212 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c65793d5-e47c-4045-b512-ce00ac69753c-istio-data\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:57:04.230425 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:04.230219 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cbzxz\" (UniqueName: \"kubernetes.io/projected/c65793d5-e47c-4045-b512-ce00ac69753c-kube-api-access-cbzxz\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:57:04.230425 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:04.230229 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c65793d5-e47c-4045-b512-ce00ac69753c-istio-envoy\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:57:04.598650 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:04.598616 2563 generic.go:356] "Generic (PLEG): container finished" podID="c65793d5-e47c-4045-b512-ce00ac69753c" containerID="fa17916eb897dad7d65567fb0c771dc203d3164b4b80afd1334a92ef56dabbcd" exitCode=0 Jul 30 00:57:04.598830 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:04.598672 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk" Jul 30 00:57:04.598830 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:04.598689 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk" event={"ID":"c65793d5-e47c-4045-b512-ce00ac69753c","Type":"ContainerDied","Data":"fa17916eb897dad7d65567fb0c771dc203d3164b4b80afd1334a92ef56dabbcd"} Jul 30 00:57:04.598830 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:04.598761 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk" event={"ID":"c65793d5-e47c-4045-b512-ce00ac69753c","Type":"ContainerDied","Data":"3e775b89925fe60668a5b3e7c52101164f7b626829fb3db0f85ba5f180f86078"} Jul 30 00:57:04.598830 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:04.598783 2563 scope.go:117] "RemoveContainer" containerID="fa17916eb897dad7d65567fb0c771dc203d3164b4b80afd1334a92ef56dabbcd" Jul 30 00:57:04.609855 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:04.609830 2563 scope.go:117] "RemoveContainer" containerID="b5f0a8f4bc4203cb16a7d789860c5665ce4cca56d0d341a8ce73ca5dedc77b1a" Jul 30 00:57:04.615944 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:04.615922 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk"] Jul 30 00:57:04.619259 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:04.619241 2563 scope.go:117] "RemoveContainer" containerID="fa17916eb897dad7d65567fb0c771dc203d3164b4b80afd1334a92ef56dabbcd" Jul 30 00:57:04.619517 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:57:04.619494 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa17916eb897dad7d65567fb0c771dc203d3164b4b80afd1334a92ef56dabbcd\": container with ID starting with fa17916eb897dad7d65567fb0c771dc203d3164b4b80afd1334a92ef56dabbcd not found: ID does not exist" containerID="fa17916eb897dad7d65567fb0c771dc203d3164b4b80afd1334a92ef56dabbcd" Jul 30 00:57:04.619621 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:04.619530 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa17916eb897dad7d65567fb0c771dc203d3164b4b80afd1334a92ef56dabbcd"} err="failed to get container status \"fa17916eb897dad7d65567fb0c771dc203d3164b4b80afd1334a92ef56dabbcd\": rpc error: code = NotFound desc = could not find container \"fa17916eb897dad7d65567fb0c771dc203d3164b4b80afd1334a92ef56dabbcd\": container with ID starting with fa17916eb897dad7d65567fb0c771dc203d3164b4b80afd1334a92ef56dabbcd not found: ID does not exist" Jul 30 00:57:04.619621 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:04.619554 2563 scope.go:117] "RemoveContainer" containerID="b5f0a8f4bc4203cb16a7d789860c5665ce4cca56d0d341a8ce73ca5dedc77b1a" Jul 30 00:57:04.619863 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:57:04.619847 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5f0a8f4bc4203cb16a7d789860c5665ce4cca56d0d341a8ce73ca5dedc77b1a\": container with ID starting with b5f0a8f4bc4203cb16a7d789860c5665ce4cca56d0d341a8ce73ca5dedc77b1a not found: ID does not exist" containerID="b5f0a8f4bc4203cb16a7d789860c5665ce4cca56d0d341a8ce73ca5dedc77b1a" Jul 30 00:57:04.619921 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:04.619869 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5f0a8f4bc4203cb16a7d789860c5665ce4cca56d0d341a8ce73ca5dedc77b1a"} err="failed to get container status \"b5f0a8f4bc4203cb16a7d789860c5665ce4cca56d0d341a8ce73ca5dedc77b1a\": rpc error: code = NotFound desc = could not find container \"b5f0a8f4bc4203cb16a7d789860c5665ce4cca56d0d341a8ce73ca5dedc77b1a\": container with ID starting with b5f0a8f4bc4203cb16a7d789860c5665ce4cca56d0d341a8ce73ca5dedc77b1a not found: ID does not exist" Jul 30 00:57:04.620298 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:04.620279 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-2239f6-openshift-default-6c64557586-fddmk"] Jul 30 00:57:06.476742 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:06.476695 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c65793d5-e47c-4045-b512-ce00ac69753c" path="/var/lib/kubelet/pods/c65793d5-e47c-4045-b512-ce00ac69753c/volumes" Jul 30 00:57:09.557577 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:09.557536 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:57:09.628493 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:09.628454 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:57:09.691902 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:09.691857 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:57:09.746841 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:09.746803 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:57:10.076899 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:10.076858 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:57:15.827074 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:15.827040 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:57:15.927536 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:15.927499 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:57:15.986298 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:15.986263 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:57:16.056930 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:16.056897 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn"] Jul 30 00:57:16.057220 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:16.057170 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn" podUID="f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2" containerName="istio-proxy" containerID="cri-o://15f5b31d46c8a2487e57e09feda93a98a73e5946fd0a9888b7a2a3b501bd9f0d" gracePeriod=30 Jul 30 00:57:16.087012 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:16.086917 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:57:16.294467 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:16.294423 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:57:16.576450 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:16.576422 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-b92tw_0ef8f120-f712-4ed9-adbd-40f785d4aa9d/console-operator/1.log" Jul 30 00:57:16.579249 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:16.579227 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-b92tw_0ef8f120-f712-4ed9-adbd-40f785d4aa9d/console-operator/1.log" Jul 30 00:57:16.605407 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:16.605371 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:57:16.876885 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:16.876811 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6"] Jul 30 00:57:16.877219 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:16.877202 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c65793d5-e47c-4045-b512-ce00ac69753c" containerName="istio-proxy" Jul 30 00:57:16.877219 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:16.877213 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c65793d5-e47c-4045-b512-ce00ac69753c" containerName="istio-proxy" Jul 30 00:57:16.877293 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:16.877228 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c65793d5-e47c-4045-b512-ce00ac69753c" containerName="istio-proxy" Jul 30 00:57:16.877293 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:16.877236 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c65793d5-e47c-4045-b512-ce00ac69753c" containerName="istio-proxy" Jul 30 00:57:16.877352 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:16.877336 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c65793d5-e47c-4045-b512-ce00ac69753c" containerName="istio-proxy" Jul 30 00:57:16.877352 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:16.877344 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c65793d5-e47c-4045-b512-ce00ac69753c" containerName="istio-proxy" Jul 30 00:57:16.882042 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:16.882025 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6" Jul 30 00:57:16.884693 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:16.884670 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-772c30-openshift-default-dockercfg-rn9dd\"" Jul 30 00:57:16.893568 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:16.893541 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6"] Jul 30 00:57:16.902248 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:16.902215 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:57:16.946586 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:16.946546 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ee95f87-8205-45e8-be5d-d6a94ccc7992-istio-envoy\") pod \"e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6\" (UID: \"2ee95f87-8205-45e8-be5d-d6a94ccc7992\") " pod="openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6" Jul 30 00:57:16.946841 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:16.946608 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhd92\" (UniqueName: \"kubernetes.io/projected/2ee95f87-8205-45e8-be5d-d6a94ccc7992-kube-api-access-vhd92\") pod \"e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6\" (UID: \"2ee95f87-8205-45e8-be5d-d6a94ccc7992\") " pod="openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6" Jul 30 00:57:16.953917 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:16.953623 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ee95f87-8205-45e8-be5d-d6a94ccc7992-istio-podinfo\") pod \"e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6\" (UID: \"2ee95f87-8205-45e8-be5d-d6a94ccc7992\") " pod="openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6" Jul 30 00:57:16.953917 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:16.953674 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ee95f87-8205-45e8-be5d-d6a94ccc7992-istiod-ca-cert\") pod \"e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6\" (UID: \"2ee95f87-8205-45e8-be5d-d6a94ccc7992\") " pod="openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6" Jul 30 00:57:16.953917 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:16.953763 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ee95f87-8205-45e8-be5d-d6a94ccc7992-istio-token\") pod \"e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6\" (UID: \"2ee95f87-8205-45e8-be5d-d6a94ccc7992\") " pod="openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6" Jul 30 00:57:16.953917 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:16.953824 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ee95f87-8205-45e8-be5d-d6a94ccc7992-credential-socket\") pod \"e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6\" (UID: \"2ee95f87-8205-45e8-be5d-d6a94ccc7992\") " pod="openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6" Jul 30 00:57:16.953917 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:16.953878 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ee95f87-8205-45e8-be5d-d6a94ccc7992-workload-socket\") pod \"e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6\" (UID: \"2ee95f87-8205-45e8-be5d-d6a94ccc7992\") " pod="openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6" Jul 30 00:57:16.954272 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:16.953941 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ee95f87-8205-45e8-be5d-d6a94ccc7992-istio-data\") pod \"e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6\" (UID: \"2ee95f87-8205-45e8-be5d-d6a94ccc7992\") " pod="openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6" Jul 30 00:57:16.954272 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:16.954058 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ee95f87-8205-45e8-be5d-d6a94ccc7992-workload-certs\") pod \"e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6\" (UID: \"2ee95f87-8205-45e8-be5d-d6a94ccc7992\") " pod="openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6" Jul 30 00:57:17.055075 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:17.055032 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhd92\" (UniqueName: \"kubernetes.io/projected/2ee95f87-8205-45e8-be5d-d6a94ccc7992-kube-api-access-vhd92\") pod \"e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6\" (UID: \"2ee95f87-8205-45e8-be5d-d6a94ccc7992\") " pod="openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6" Jul 30 00:57:17.055282 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:17.055086 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ee95f87-8205-45e8-be5d-d6a94ccc7992-istio-podinfo\") pod \"e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6\" (UID: \"2ee95f87-8205-45e8-be5d-d6a94ccc7992\") " pod="openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6" Jul 30 00:57:17.055282 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:17.055114 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ee95f87-8205-45e8-be5d-d6a94ccc7992-istiod-ca-cert\") pod \"e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6\" (UID: \"2ee95f87-8205-45e8-be5d-d6a94ccc7992\") " pod="openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6" Jul 30 00:57:17.055282 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:17.055166 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ee95f87-8205-45e8-be5d-d6a94ccc7992-istio-token\") pod \"e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6\" (UID: \"2ee95f87-8205-45e8-be5d-d6a94ccc7992\") " pod="openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6" Jul 30 00:57:17.055282 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:17.055218 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ee95f87-8205-45e8-be5d-d6a94ccc7992-credential-socket\") pod \"e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6\" (UID: \"2ee95f87-8205-45e8-be5d-d6a94ccc7992\") " pod="openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6" Jul 30 00:57:17.055282 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:17.055261 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ee95f87-8205-45e8-be5d-d6a94ccc7992-workload-socket\") pod \"e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6\" (UID: \"2ee95f87-8205-45e8-be5d-d6a94ccc7992\") " pod="openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6" Jul 30 00:57:17.055569 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:17.055288 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ee95f87-8205-45e8-be5d-d6a94ccc7992-istio-data\") pod \"e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6\" (UID: \"2ee95f87-8205-45e8-be5d-d6a94ccc7992\") " pod="openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6" Jul 30 00:57:17.055569 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:17.055326 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ee95f87-8205-45e8-be5d-d6a94ccc7992-workload-certs\") pod \"e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6\" (UID: \"2ee95f87-8205-45e8-be5d-d6a94ccc7992\") " pod="openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6" Jul 30 00:57:17.055569 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:17.055379 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ee95f87-8205-45e8-be5d-d6a94ccc7992-istio-envoy\") pod \"e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6\" (UID: \"2ee95f87-8205-45e8-be5d-d6a94ccc7992\") " pod="openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6" Jul 30 00:57:17.055830 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:17.055803 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ee95f87-8205-45e8-be5d-d6a94ccc7992-workload-socket\") pod \"e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6\" (UID: \"2ee95f87-8205-45e8-be5d-d6a94ccc7992\") " pod="openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6" Jul 30 00:57:17.055927 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:17.055881 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ee95f87-8205-45e8-be5d-d6a94ccc7992-workload-certs\") pod \"e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6\" (UID: \"2ee95f87-8205-45e8-be5d-d6a94ccc7992\") " pod="openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6" Jul 30 00:57:17.056085 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:17.056063 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ee95f87-8205-45e8-be5d-d6a94ccc7992-credential-socket\") pod \"e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6\" (UID: \"2ee95f87-8205-45e8-be5d-d6a94ccc7992\") " pod="openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6" Jul 30 00:57:17.056168 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:17.056126 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ee95f87-8205-45e8-be5d-d6a94ccc7992-istio-data\") pod \"e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6\" (UID: \"2ee95f87-8205-45e8-be5d-d6a94ccc7992\") " pod="openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6" Jul 30 00:57:17.056393 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:17.056369 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ee95f87-8205-45e8-be5d-d6a94ccc7992-istiod-ca-cert\") pod \"e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6\" (UID: \"2ee95f87-8205-45e8-be5d-d6a94ccc7992\") " pod="openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6" Jul 30 00:57:17.058273 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:17.058247 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ee95f87-8205-45e8-be5d-d6a94ccc7992-istio-podinfo\") pod \"e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6\" (UID: \"2ee95f87-8205-45e8-be5d-d6a94ccc7992\") " pod="openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6" Jul 30 00:57:17.058386 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:17.058254 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ee95f87-8205-45e8-be5d-d6a94ccc7992-istio-envoy\") pod \"e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6\" (UID: \"2ee95f87-8205-45e8-be5d-d6a94ccc7992\") " pod="openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6" Jul 30 00:57:17.065868 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:17.065844 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhd92\" (UniqueName: \"kubernetes.io/projected/2ee95f87-8205-45e8-be5d-d6a94ccc7992-kube-api-access-vhd92\") pod \"e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6\" (UID: \"2ee95f87-8205-45e8-be5d-d6a94ccc7992\") " pod="openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6" Jul 30 00:57:17.065980 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:17.065845 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ee95f87-8205-45e8-be5d-d6a94ccc7992-istio-token\") pod \"e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6\" (UID: \"2ee95f87-8205-45e8-be5d-d6a94ccc7992\") " pod="openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6" Jul 30 00:57:17.195524 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:17.195427 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6" Jul 30 00:57:17.317253 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:17.317215 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:57:17.352046 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:17.352013 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6"] Jul 30 00:57:17.354794 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:57:17.354720 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ee95f87_8205_45e8_be5d_d6a94ccc7992.slice/crio-b8f307c224222698e4a33cb2265394407e9dd3b10424e352a2a94e41640dc06c WatchSource:0}: Error finding container b8f307c224222698e4a33cb2265394407e9dd3b10424e352a2a94e41640dc06c: Status 404 returned error can't find the container with id b8f307c224222698e4a33cb2265394407e9dd3b10424e352a2a94e41640dc06c Jul 30 00:57:17.357027 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:17.356993 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:57:17.357109 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:17.357075 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:57:17.357109 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:17.357105 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:57:17.664391 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:17.664345 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6" event={"ID":"2ee95f87-8205-45e8-be5d-d6a94ccc7992","Type":"ContainerStarted","Data":"f83e61466862f0dcfaac97b1387ab6c14e18e4ab32f732cf0d5a4eb2dbedf4c8"} Jul 30 00:57:17.664597 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:17.664405 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6" event={"ID":"2ee95f87-8205-45e8-be5d-d6a94ccc7992","Type":"ContainerStarted","Data":"b8f307c224222698e4a33cb2265394407e9dd3b10424e352a2a94e41640dc06c"} Jul 30 00:57:17.684581 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:17.684519 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6" podStartSLOduration=1.6845018980000002 podStartE2EDuration="1.684501898s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:57:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:57:17.682526956 +0000 UTC m=+2401.741282924" watchObservedRunningTime="2026-07-30 00:57:17.684501898 +0000 UTC m=+2401.743257865" Jul 30 00:57:18.195817 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:18.195778 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6" Jul 30 00:57:18.201174 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:18.201148 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6" Jul 30 00:57:18.669773 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:18.669721 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6" Jul 30 00:57:18.670622 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:18.670606 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6" Jul 30 00:57:19.535376 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:19.535346 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:57:19.637892 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:19.637852 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:57:20.658140 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:20.658107 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:57:20.881209 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:20.881172 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:57:21.134210 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:21.134175 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:57:21.221888 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:21.221815 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:57:21.310306 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:21.310280 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn" Jul 30 00:57:21.396349 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:21.396268 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tf96t\" (UniqueName: \"kubernetes.io/projected/f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2-kube-api-access-tf96t\") pod \"f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2\" (UID: \"f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2\") " Jul 30 00:57:21.396349 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:21.396298 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2-istiod-ca-cert\") pod \"f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2\" (UID: \"f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2\") " Jul 30 00:57:21.396349 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:21.396331 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2-istio-podinfo\") pod \"f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2\" (UID: \"f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2\") " Jul 30 00:57:21.396626 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:21.396395 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2-istio-token\") pod \"f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2\" (UID: \"f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2\") " Jul 30 00:57:21.396626 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:21.396425 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2-workload-socket\") pod \"f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2\" (UID: \"f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2\") " Jul 30 00:57:21.396626 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:21.396457 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2-istio-data\") pod \"f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2\" (UID: \"f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2\") " Jul 30 00:57:21.396626 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:21.396536 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2-istio-envoy\") pod \"f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2\" (UID: \"f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2\") " Jul 30 00:57:21.396626 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:21.396589 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2-workload-certs\") pod \"f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2\" (UID: \"f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2\") " Jul 30 00:57:21.396626 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:21.396621 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2-credential-socket\") pod \"f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2\" (UID: \"f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2\") " Jul 30 00:57:21.396967 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:21.396663 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2" (UID: "f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 00:57:21.396967 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:21.396761 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2" (UID: "f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:57:21.397077 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:21.396968 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2" (UID: "f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:57:21.397077 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:21.396995 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2-workload-socket\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:57:21.397077 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:21.397012 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2-istiod-ca-cert\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:57:21.397241 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:21.397154 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2" (UID: "f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:57:21.397847 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:21.397822 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2-istio-data" (OuterVolumeSpecName: "istio-data") pod "f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2" (UID: "f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:57:21.398910 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:21.398875 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2-kube-api-access-tf96t" (OuterVolumeSpecName: "kube-api-access-tf96t") pod "f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2" (UID: "f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2"). InnerVolumeSpecName "kube-api-access-tf96t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 00:57:21.399031 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:21.399012 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2" (UID: "f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 30 00:57:21.399215 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:21.399189 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2" (UID: "f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:57:21.399523 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:21.399504 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2-istio-token" (OuterVolumeSpecName: "istio-token") pod "f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2" (UID: "f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 00:57:21.498252 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:21.498225 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2-istio-envoy\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:57:21.498252 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:21.498247 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2-workload-certs\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:57:21.498252 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:21.498256 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2-credential-socket\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:57:21.498430 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:21.498266 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tf96t\" (UniqueName: \"kubernetes.io/projected/f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2-kube-api-access-tf96t\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:57:21.498430 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:21.498276 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2-istio-podinfo\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:57:21.498430 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:21.498284 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2-istio-token\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:57:21.498430 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:21.498291 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2-istio-data\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:57:21.686455 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:21.686378 2563 generic.go:356] "Generic (PLEG): container finished" podID="f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2" containerID="15f5b31d46c8a2487e57e09feda93a98a73e5946fd0a9888b7a2a3b501bd9f0d" exitCode=0 Jul 30 00:57:21.686455 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:21.686430 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn" Jul 30 00:57:21.686924 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:21.686462 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn" event={"ID":"f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2","Type":"ContainerDied","Data":"15f5b31d46c8a2487e57e09feda93a98a73e5946fd0a9888b7a2a3b501bd9f0d"} Jul 30 00:57:21.686924 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:21.686518 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn" event={"ID":"f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2","Type":"ContainerDied","Data":"97443f780964cd6793ec46d59ec0160106ed22231dd382a54f2ab1370b4574ad"} Jul 30 00:57:21.686924 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:21.686540 2563 scope.go:117] "RemoveContainer" containerID="15f5b31d46c8a2487e57e09feda93a98a73e5946fd0a9888b7a2a3b501bd9f0d" Jul 30 00:57:21.696197 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:21.696181 2563 scope.go:117] "RemoveContainer" containerID="f367bd939dae275b398dc6eb18ca76df1a9427fe69fa9e2630e2d794cdf92913" Jul 30 00:57:21.704184 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:21.704167 2563 scope.go:117] "RemoveContainer" containerID="15f5b31d46c8a2487e57e09feda93a98a73e5946fd0a9888b7a2a3b501bd9f0d" Jul 30 00:57:21.704438 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:57:21.704417 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15f5b31d46c8a2487e57e09feda93a98a73e5946fd0a9888b7a2a3b501bd9f0d\": container with ID starting with 15f5b31d46c8a2487e57e09feda93a98a73e5946fd0a9888b7a2a3b501bd9f0d not found: ID does not exist" containerID="15f5b31d46c8a2487e57e09feda93a98a73e5946fd0a9888b7a2a3b501bd9f0d" Jul 30 00:57:21.704484 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:21.704447 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15f5b31d46c8a2487e57e09feda93a98a73e5946fd0a9888b7a2a3b501bd9f0d"} err="failed to get container status \"15f5b31d46c8a2487e57e09feda93a98a73e5946fd0a9888b7a2a3b501bd9f0d\": rpc error: code = NotFound desc = could not find container \"15f5b31d46c8a2487e57e09feda93a98a73e5946fd0a9888b7a2a3b501bd9f0d\": container with ID starting with 15f5b31d46c8a2487e57e09feda93a98a73e5946fd0a9888b7a2a3b501bd9f0d not found: ID does not exist" Jul 30 00:57:21.704484 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:21.704464 2563 scope.go:117] "RemoveContainer" containerID="f367bd939dae275b398dc6eb18ca76df1a9427fe69fa9e2630e2d794cdf92913" Jul 30 00:57:21.704649 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:57:21.704632 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f367bd939dae275b398dc6eb18ca76df1a9427fe69fa9e2630e2d794cdf92913\": container with ID starting with f367bd939dae275b398dc6eb18ca76df1a9427fe69fa9e2630e2d794cdf92913 not found: ID does not exist" containerID="f367bd939dae275b398dc6eb18ca76df1a9427fe69fa9e2630e2d794cdf92913" Jul 30 00:57:21.704688 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:21.704653 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f367bd939dae275b398dc6eb18ca76df1a9427fe69fa9e2630e2d794cdf92913"} err="failed to get container status \"f367bd939dae275b398dc6eb18ca76df1a9427fe69fa9e2630e2d794cdf92913\": rpc error: code = NotFound desc = could not find container \"f367bd939dae275b398dc6eb18ca76df1a9427fe69fa9e2630e2d794cdf92913\": container with ID starting with f367bd939dae275b398dc6eb18ca76df1a9427fe69fa9e2630e2d794cdf92913 not found: ID does not exist" Jul 30 00:57:21.707921 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:21.707898 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn"] Jul 30 00:57:21.713647 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:21.713627 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-6898ea-openshift-default-665c46654b-fz6cn"] Jul 30 00:57:22.475565 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:22.475533 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2" path="/var/lib/kubelet/pods/f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2/volumes" Jul 30 00:57:33.671131 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:33.671091 2563 patch_prober.go:28] interesting pod/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 30 00:57:33.671563 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:33.671186 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6" podUID="2ee95f87-8205-45e8-be5d-d6a94ccc7992" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 30 00:57:33.749591 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:33.749558 2563 generic.go:356] "Generic (PLEG): container finished" podID="2ee95f87-8205-45e8-be5d-d6a94ccc7992" containerID="f83e61466862f0dcfaac97b1387ab6c14e18e4ab32f732cf0d5a4eb2dbedf4c8" exitCode=0 Jul 30 00:57:33.749786 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:33.749635 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6" event={"ID":"2ee95f87-8205-45e8-be5d-d6a94ccc7992","Type":"ContainerDied","Data":"f83e61466862f0dcfaac97b1387ab6c14e18e4ab32f732cf0d5a4eb2dbedf4c8"} Jul 30 00:57:33.750085 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:33.750071 2563 scope.go:117] "RemoveContainer" containerID="f83e61466862f0dcfaac97b1387ab6c14e18e4ab32f732cf0d5a4eb2dbedf4c8" Jul 30 00:57:33.751125 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:33.751101 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:57:33.751183 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:33.751161 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:57:33.751219 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:33.751204 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:57:34.757167 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:34.757126 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6" event={"ID":"2ee95f87-8205-45e8-be5d-d6a94ccc7992","Type":"ContainerStarted","Data":"9924ed6b5460d855b4b47d5dae83d2e1d9fb15ee31a3c590974be3614d7a09fa"} Jul 30 00:57:35.196526 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:35.196425 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6" Jul 30 00:57:35.196678 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:35.196650 2563 patch_prober.go:28] interesting pod/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 30 00:57:35.196777 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:35.196751 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6" podUID="2ee95f87-8205-45e8-be5d-d6a94ccc7992" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 30 00:57:36.196690 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:36.196651 2563 patch_prober.go:28] interesting pod/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 30 00:57:36.197177 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:36.196705 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6" podUID="2ee95f87-8205-45e8-be5d-d6a94ccc7992" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 30 00:57:37.196414 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:37.196372 2563 patch_prober.go:28] interesting pod/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 30 00:57:37.196623 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:37.196432 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6" podUID="2ee95f87-8205-45e8-be5d-d6a94ccc7992" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 30 00:57:38.196310 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:38.196268 2563 patch_prober.go:28] interesting pod/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 30 00:57:38.196813 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:38.196330 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6" podUID="2ee95f87-8205-45e8-be5d-d6a94ccc7992" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 30 00:57:39.200185 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:39.200104 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6" Jul 30 00:57:39.200642 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:39.200495 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6" Jul 30 00:57:39.201076 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:39.201057 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6" Jul 30 00:57:44.482975 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:44.482944 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:57:44.926963 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:44.926933 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:57:45.194082 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:45.193999 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln"] Jul 30 00:57:45.194441 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:45.194420 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2" containerName="istio-proxy" Jul 30 00:57:45.194441 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:45.194437 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2" containerName="istio-proxy" Jul 30 00:57:45.194644 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:45.194524 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2" containerName="istio-proxy" Jul 30 00:57:45.194644 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:45.194536 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2" containerName="istio-proxy" Jul 30 00:57:45.197046 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:45.197022 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln" Jul 30 00:57:45.199603 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:45.199582 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-68c25c-openshift-default-dockercfg-sjg22\"" Jul 30 00:57:45.209954 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:45.209931 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln"] Jul 30 00:57:45.224208 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:45.224176 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:57:45.242966 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:45.242937 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd030969-963a-4fe8-89b4-37508b74e303-workload-socket\") pod \"e2e-api-b-68c25c-openshift-default-6796c7488-k5tln\" (UID: \"fd030969-963a-4fe8-89b4-37508b74e303\") " pod="openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln" Jul 30 00:57:45.243125 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:45.242977 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd030969-963a-4fe8-89b4-37508b74e303-istiod-ca-cert\") pod \"e2e-api-b-68c25c-openshift-default-6796c7488-k5tln\" (UID: \"fd030969-963a-4fe8-89b4-37508b74e303\") " pod="openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln" Jul 30 00:57:45.243125 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:45.243035 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd030969-963a-4fe8-89b4-37508b74e303-istio-podinfo\") pod \"e2e-api-b-68c25c-openshift-default-6796c7488-k5tln\" (UID: \"fd030969-963a-4fe8-89b4-37508b74e303\") " pod="openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln" Jul 30 00:57:45.243125 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:45.243058 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd030969-963a-4fe8-89b4-37508b74e303-credential-socket\") pod \"e2e-api-b-68c25c-openshift-default-6796c7488-k5tln\" (UID: \"fd030969-963a-4fe8-89b4-37508b74e303\") " pod="openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln" Jul 30 00:57:45.243314 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:45.243146 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd030969-963a-4fe8-89b4-37508b74e303-istio-envoy\") pod \"e2e-api-b-68c25c-openshift-default-6796c7488-k5tln\" (UID: \"fd030969-963a-4fe8-89b4-37508b74e303\") " pod="openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln" Jul 30 00:57:45.243314 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:45.243191 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd030969-963a-4fe8-89b4-37508b74e303-istio-data\") pod \"e2e-api-b-68c25c-openshift-default-6796c7488-k5tln\" (UID: \"fd030969-963a-4fe8-89b4-37508b74e303\") " pod="openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln" Jul 30 00:57:45.243314 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:45.243242 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cd86z\" (UniqueName: \"kubernetes.io/projected/fd030969-963a-4fe8-89b4-37508b74e303-kube-api-access-cd86z\") pod \"e2e-api-b-68c25c-openshift-default-6796c7488-k5tln\" (UID: \"fd030969-963a-4fe8-89b4-37508b74e303\") " pod="openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln" Jul 30 00:57:45.243314 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:45.243271 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd030969-963a-4fe8-89b4-37508b74e303-workload-certs\") pod \"e2e-api-b-68c25c-openshift-default-6796c7488-k5tln\" (UID: \"fd030969-963a-4fe8-89b4-37508b74e303\") " pod="openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln" Jul 30 00:57:45.243476 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:45.243324 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd030969-963a-4fe8-89b4-37508b74e303-istio-token\") pod \"e2e-api-b-68c25c-openshift-default-6796c7488-k5tln\" (UID: \"fd030969-963a-4fe8-89b4-37508b74e303\") " pod="openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln" Jul 30 00:57:45.324543 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:45.324504 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:57:45.344532 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:45.344508 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd030969-963a-4fe8-89b4-37508b74e303-istio-data\") pod \"e2e-api-b-68c25c-openshift-default-6796c7488-k5tln\" (UID: \"fd030969-963a-4fe8-89b4-37508b74e303\") " pod="openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln" Jul 30 00:57:45.344686 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:45.344545 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cd86z\" (UniqueName: \"kubernetes.io/projected/fd030969-963a-4fe8-89b4-37508b74e303-kube-api-access-cd86z\") pod \"e2e-api-b-68c25c-openshift-default-6796c7488-k5tln\" (UID: \"fd030969-963a-4fe8-89b4-37508b74e303\") " pod="openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln" Jul 30 00:57:45.344686 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:45.344564 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd030969-963a-4fe8-89b4-37508b74e303-workload-certs\") pod \"e2e-api-b-68c25c-openshift-default-6796c7488-k5tln\" (UID: \"fd030969-963a-4fe8-89b4-37508b74e303\") " pod="openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln" Jul 30 00:57:45.344686 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:45.344611 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd030969-963a-4fe8-89b4-37508b74e303-istio-token\") pod \"e2e-api-b-68c25c-openshift-default-6796c7488-k5tln\" (UID: \"fd030969-963a-4fe8-89b4-37508b74e303\") " pod="openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln" Jul 30 00:57:45.344686 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:45.344664 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd030969-963a-4fe8-89b4-37508b74e303-workload-socket\") pod \"e2e-api-b-68c25c-openshift-default-6796c7488-k5tln\" (UID: \"fd030969-963a-4fe8-89b4-37508b74e303\") " pod="openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln" Jul 30 00:57:45.344973 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:45.344698 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd030969-963a-4fe8-89b4-37508b74e303-istiod-ca-cert\") pod \"e2e-api-b-68c25c-openshift-default-6796c7488-k5tln\" (UID: \"fd030969-963a-4fe8-89b4-37508b74e303\") " pod="openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln" Jul 30 00:57:45.344973 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:45.344777 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd030969-963a-4fe8-89b4-37508b74e303-istio-podinfo\") pod \"e2e-api-b-68c25c-openshift-default-6796c7488-k5tln\" (UID: \"fd030969-963a-4fe8-89b4-37508b74e303\") " pod="openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln" Jul 30 00:57:45.344973 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:45.344809 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd030969-963a-4fe8-89b4-37508b74e303-credential-socket\") pod \"e2e-api-b-68c25c-openshift-default-6796c7488-k5tln\" (UID: \"fd030969-963a-4fe8-89b4-37508b74e303\") " pod="openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln" Jul 30 00:57:45.344973 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:45.344854 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd030969-963a-4fe8-89b4-37508b74e303-istio-envoy\") pod \"e2e-api-b-68c25c-openshift-default-6796c7488-k5tln\" (UID: \"fd030969-963a-4fe8-89b4-37508b74e303\") " pod="openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln" Jul 30 00:57:45.345150 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:45.345028 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd030969-963a-4fe8-89b4-37508b74e303-istio-data\") pod \"e2e-api-b-68c25c-openshift-default-6796c7488-k5tln\" (UID: \"fd030969-963a-4fe8-89b4-37508b74e303\") " pod="openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln" Jul 30 00:57:45.345150 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:45.345119 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd030969-963a-4fe8-89b4-37508b74e303-workload-certs\") pod \"e2e-api-b-68c25c-openshift-default-6796c7488-k5tln\" (UID: \"fd030969-963a-4fe8-89b4-37508b74e303\") " pod="openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln" Jul 30 00:57:45.345231 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:45.345178 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd030969-963a-4fe8-89b4-37508b74e303-credential-socket\") pod \"e2e-api-b-68c25c-openshift-default-6796c7488-k5tln\" (UID: \"fd030969-963a-4fe8-89b4-37508b74e303\") " pod="openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln" Jul 30 00:57:45.345231 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:45.345208 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd030969-963a-4fe8-89b4-37508b74e303-workload-socket\") pod \"e2e-api-b-68c25c-openshift-default-6796c7488-k5tln\" (UID: \"fd030969-963a-4fe8-89b4-37508b74e303\") " pod="openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln" Jul 30 00:57:45.345430 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:45.345407 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd030969-963a-4fe8-89b4-37508b74e303-istiod-ca-cert\") pod \"e2e-api-b-68c25c-openshift-default-6796c7488-k5tln\" (UID: \"fd030969-963a-4fe8-89b4-37508b74e303\") " pod="openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln" Jul 30 00:57:45.347135 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:45.347113 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd030969-963a-4fe8-89b4-37508b74e303-istio-envoy\") pod \"e2e-api-b-68c25c-openshift-default-6796c7488-k5tln\" (UID: \"fd030969-963a-4fe8-89b4-37508b74e303\") " pod="openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln" Jul 30 00:57:45.347367 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:45.347337 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd030969-963a-4fe8-89b4-37508b74e303-istio-podinfo\") pod \"e2e-api-b-68c25c-openshift-default-6796c7488-k5tln\" (UID: \"fd030969-963a-4fe8-89b4-37508b74e303\") " pod="openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln" Jul 30 00:57:45.352367 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:45.352339 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd030969-963a-4fe8-89b4-37508b74e303-istio-token\") pod \"e2e-api-b-68c25c-openshift-default-6796c7488-k5tln\" (UID: \"fd030969-963a-4fe8-89b4-37508b74e303\") " pod="openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln" Jul 30 00:57:45.352548 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:45.352532 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cd86z\" (UniqueName: \"kubernetes.io/projected/fd030969-963a-4fe8-89b4-37508b74e303-kube-api-access-cd86z\") pod \"e2e-api-b-68c25c-openshift-default-6796c7488-k5tln\" (UID: \"fd030969-963a-4fe8-89b4-37508b74e303\") " pod="openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln" Jul 30 00:57:45.508437 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:45.508404 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln" Jul 30 00:57:45.681550 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:45.681523 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln"] Jul 30 00:57:45.683839 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:57:45.683803 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd030969_963a_4fe8_89b4_37508b74e303.slice/crio-9eacd008d614cc6685da2ebae011690623aa2443cfaecfc1f6ffcc76db6f6ffb WatchSource:0}: Error finding container 9eacd008d614cc6685da2ebae011690623aa2443cfaecfc1f6ffcc76db6f6ffb: Status 404 returned error can't find the container with id 9eacd008d614cc6685da2ebae011690623aa2443cfaecfc1f6ffcc76db6f6ffb Jul 30 00:57:45.686236 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:45.686204 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:57:45.686334 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:45.686268 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:57:45.686334 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:45.686325 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:57:45.825585 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:45.825484 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln" event={"ID":"fd030969-963a-4fe8-89b4-37508b74e303","Type":"ContainerStarted","Data":"b58c59a7029323dbc6c1416532765ae3f31a91795a0b047103f04486fef2610b"} Jul 30 00:57:45.825585 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:45.825541 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln" event={"ID":"fd030969-963a-4fe8-89b4-37508b74e303","Type":"ContainerStarted","Data":"9eacd008d614cc6685da2ebae011690623aa2443cfaecfc1f6ffcc76db6f6ffb"} Jul 30 00:57:45.848670 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:45.848622 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln" podStartSLOduration=0.848599137 podStartE2EDuration="848.599137ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:57:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:57:45.846952285 +0000 UTC m=+2429.905708255" watchObservedRunningTime="2026-07-30 00:57:45.848599137 +0000 UTC m=+2429.907355106" Jul 30 00:57:46.508987 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:46.508951 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln" Jul 30 00:57:46.514296 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:46.514264 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln" Jul 30 00:57:46.831932 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:46.831843 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln" Jul 30 00:57:46.832782 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:46.832759 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln" Jul 30 00:57:47.702641 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:47.702591 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:57:47.811542 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:47.811505 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:57:48.959975 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:48.959930 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:57:49.112020 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:49.111980 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:57:49.414099 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:49.414060 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:57:49.503298 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:57:49.503264 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:58:02.874864 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:02.874826 2563 patch_prober.go:28] interesting pod/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 30 00:58:02.875357 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:02.874896 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln" podUID="fd030969-963a-4fe8-89b4-37508b74e303" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 30 00:58:03.927179 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:03.927142 2563 generic.go:356] "Generic (PLEG): container finished" podID="fd030969-963a-4fe8-89b4-37508b74e303" containerID="b58c59a7029323dbc6c1416532765ae3f31a91795a0b047103f04486fef2610b" exitCode=0 Jul 30 00:58:03.927694 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:03.927219 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln" event={"ID":"fd030969-963a-4fe8-89b4-37508b74e303","Type":"ContainerDied","Data":"b58c59a7029323dbc6c1416532765ae3f31a91795a0b047103f04486fef2610b"} Jul 30 00:58:03.927801 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:03.927761 2563 scope.go:117] "RemoveContainer" containerID="b58c59a7029323dbc6c1416532765ae3f31a91795a0b047103f04486fef2610b" Jul 30 00:58:03.928809 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:03.928784 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:58:03.928916 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:03.928883 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:58:03.928980 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:03.928932 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:58:04.933765 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:04.933718 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln" event={"ID":"fd030969-963a-4fe8-89b4-37508b74e303","Type":"ContainerStarted","Data":"8a336d907849f2dc16530b16b1ce69b0c1d8a6d9f63fcb0792f9149908421bd1"} Jul 30 00:58:05.509047 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:05.509007 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln" Jul 30 00:58:05.509261 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:05.509240 2563 patch_prober.go:28] interesting pod/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 30 00:58:05.509308 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:05.509287 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln" podUID="fd030969-963a-4fe8-89b4-37508b74e303" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 30 00:58:06.509197 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:06.509162 2563 patch_prober.go:28] interesting pod/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 30 00:58:06.509657 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:06.509233 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln" podUID="fd030969-963a-4fe8-89b4-37508b74e303" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 30 00:58:07.508859 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:07.508824 2563 patch_prober.go:28] interesting pod/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 30 00:58:07.509042 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:07.508880 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln" podUID="fd030969-963a-4fe8-89b4-37508b74e303" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 30 00:58:08.509530 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:08.509493 2563 patch_prober.go:28] interesting pod/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 30 00:58:08.509984 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:08.509557 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln" podUID="fd030969-963a-4fe8-89b4-37508b74e303" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 30 00:58:09.512918 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:09.512883 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln" Jul 30 00:58:09.513333 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:09.513132 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln" Jul 30 00:58:09.513729 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:09.513711 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln" Jul 30 00:58:12.675869 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:12.675833 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:58:31.037186 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:31.037153 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:58:31.108328 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:31.108285 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:58:31.202546 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:31.202513 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:58:31.253762 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:31.253690 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:58:31.563173 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:31.563141 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:58:33.262296 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:33.262250 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:58:37.190519 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:37.190481 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:58:37.290875 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:37.290841 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:58:37.425690 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:37.425658 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6"] Jul 30 00:58:37.425890 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:37.425867 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6" podUID="2ee95f87-8205-45e8-be5d-d6a94ccc7992" containerName="istio-proxy" containerID="cri-o://9924ed6b5460d855b4b47d5dae83d2e1d9fb15ee31a3c590974be3614d7a09fa" gracePeriod=30 Jul 30 00:58:37.443064 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:37.442982 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:58:37.648502 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:37.648464 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:58:39.203142 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:39.203097 2563 patch_prober.go:28] interesting pod/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 30 00:58:39.203518 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:39.203169 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6" podUID="2ee95f87-8205-45e8-be5d-d6a94ccc7992" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 30 00:58:42.686938 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:42.686878 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6" Jul 30 00:58:42.804118 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:42.804082 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ee95f87-8205-45e8-be5d-d6a94ccc7992-workload-socket\") pod \"2ee95f87-8205-45e8-be5d-d6a94ccc7992\" (UID: \"2ee95f87-8205-45e8-be5d-d6a94ccc7992\") " Jul 30 00:58:42.804290 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:42.804157 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ee95f87-8205-45e8-be5d-d6a94ccc7992-credential-socket\") pod \"2ee95f87-8205-45e8-be5d-d6a94ccc7992\" (UID: \"2ee95f87-8205-45e8-be5d-d6a94ccc7992\") " Jul 30 00:58:42.804290 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:42.804193 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ee95f87-8205-45e8-be5d-d6a94ccc7992-workload-certs\") pod \"2ee95f87-8205-45e8-be5d-d6a94ccc7992\" (UID: \"2ee95f87-8205-45e8-be5d-d6a94ccc7992\") " Jul 30 00:58:42.804290 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:42.804214 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ee95f87-8205-45e8-be5d-d6a94ccc7992-istio-token\") pod \"2ee95f87-8205-45e8-be5d-d6a94ccc7992\" (UID: \"2ee95f87-8205-45e8-be5d-d6a94ccc7992\") " Jul 30 00:58:42.804290 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:42.804244 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ee95f87-8205-45e8-be5d-d6a94ccc7992-istio-envoy\") pod \"2ee95f87-8205-45e8-be5d-d6a94ccc7992\" (UID: \"2ee95f87-8205-45e8-be5d-d6a94ccc7992\") " Jul 30 00:58:42.804502 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:42.804310 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ee95f87-8205-45e8-be5d-d6a94ccc7992-istio-data\") pod \"2ee95f87-8205-45e8-be5d-d6a94ccc7992\" (UID: \"2ee95f87-8205-45e8-be5d-d6a94ccc7992\") " Jul 30 00:58:42.804502 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:42.804353 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vhd92\" (UniqueName: \"kubernetes.io/projected/2ee95f87-8205-45e8-be5d-d6a94ccc7992-kube-api-access-vhd92\") pod \"2ee95f87-8205-45e8-be5d-d6a94ccc7992\" (UID: \"2ee95f87-8205-45e8-be5d-d6a94ccc7992\") " Jul 30 00:58:42.804502 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:42.804395 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ee95f87-8205-45e8-be5d-d6a94ccc7992-istio-podinfo\") pod \"2ee95f87-8205-45e8-be5d-d6a94ccc7992\" (UID: \"2ee95f87-8205-45e8-be5d-d6a94ccc7992\") " Jul 30 00:58:42.804502 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:42.804390 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ee95f87-8205-45e8-be5d-d6a94ccc7992-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2ee95f87-8205-45e8-be5d-d6a94ccc7992" (UID: "2ee95f87-8205-45e8-be5d-d6a94ccc7992"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:58:42.804502 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:42.804430 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ee95f87-8205-45e8-be5d-d6a94ccc7992-istiod-ca-cert\") pod \"2ee95f87-8205-45e8-be5d-d6a94ccc7992\" (UID: \"2ee95f87-8205-45e8-be5d-d6a94ccc7992\") " Jul 30 00:58:42.804820 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:42.804505 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ee95f87-8205-45e8-be5d-d6a94ccc7992-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2ee95f87-8205-45e8-be5d-d6a94ccc7992" (UID: "2ee95f87-8205-45e8-be5d-d6a94ccc7992"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:58:42.804820 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:42.804522 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ee95f87-8205-45e8-be5d-d6a94ccc7992-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2ee95f87-8205-45e8-be5d-d6a94ccc7992" (UID: "2ee95f87-8205-45e8-be5d-d6a94ccc7992"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:58:42.804820 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:42.804799 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ee95f87-8205-45e8-be5d-d6a94ccc7992-workload-socket\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:58:42.804820 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:42.804820 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ee95f87-8205-45e8-be5d-d6a94ccc7992-credential-socket\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:58:42.805035 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:42.804836 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ee95f87-8205-45e8-be5d-d6a94ccc7992-workload-certs\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:58:42.805035 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:42.804853 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ee95f87-8205-45e8-be5d-d6a94ccc7992-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2ee95f87-8205-45e8-be5d-d6a94ccc7992" (UID: "2ee95f87-8205-45e8-be5d-d6a94ccc7992"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 00:58:42.805273 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:42.805251 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ee95f87-8205-45e8-be5d-d6a94ccc7992-istio-data" (OuterVolumeSpecName: "istio-data") pod "2ee95f87-8205-45e8-be5d-d6a94ccc7992" (UID: "2ee95f87-8205-45e8-be5d-d6a94ccc7992"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:58:42.806597 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:42.806569 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ee95f87-8205-45e8-be5d-d6a94ccc7992-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2ee95f87-8205-45e8-be5d-d6a94ccc7992" (UID: "2ee95f87-8205-45e8-be5d-d6a94ccc7992"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:58:42.806912 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:42.806883 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ee95f87-8205-45e8-be5d-d6a94ccc7992-kube-api-access-vhd92" (OuterVolumeSpecName: "kube-api-access-vhd92") pod "2ee95f87-8205-45e8-be5d-d6a94ccc7992" (UID: "2ee95f87-8205-45e8-be5d-d6a94ccc7992"). InnerVolumeSpecName "kube-api-access-vhd92". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 00:58:42.806912 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:42.806891 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ee95f87-8205-45e8-be5d-d6a94ccc7992-istio-token" (OuterVolumeSpecName: "istio-token") pod "2ee95f87-8205-45e8-be5d-d6a94ccc7992" (UID: "2ee95f87-8205-45e8-be5d-d6a94ccc7992"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 00:58:42.807039 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:42.806894 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2ee95f87-8205-45e8-be5d-d6a94ccc7992-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2ee95f87-8205-45e8-be5d-d6a94ccc7992" (UID: "2ee95f87-8205-45e8-be5d-d6a94ccc7992"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 30 00:58:42.905905 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:42.905866 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ee95f87-8205-45e8-be5d-d6a94ccc7992-istio-data\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:58:42.905905 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:42.905897 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vhd92\" (UniqueName: \"kubernetes.io/projected/2ee95f87-8205-45e8-be5d-d6a94ccc7992-kube-api-access-vhd92\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:58:42.905905 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:42.905909 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ee95f87-8205-45e8-be5d-d6a94ccc7992-istio-podinfo\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:58:42.905905 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:42.905918 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ee95f87-8205-45e8-be5d-d6a94ccc7992-istiod-ca-cert\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:58:42.906174 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:42.905926 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ee95f87-8205-45e8-be5d-d6a94ccc7992-istio-token\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:58:42.906174 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:42.905933 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ee95f87-8205-45e8-be5d-d6a94ccc7992-istio-envoy\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:58:43.138331 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:43.138295 2563 generic.go:356] "Generic (PLEG): container finished" podID="2ee95f87-8205-45e8-be5d-d6a94ccc7992" containerID="9924ed6b5460d855b4b47d5dae83d2e1d9fb15ee31a3c590974be3614d7a09fa" exitCode=0 Jul 30 00:58:43.138504 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:43.138343 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6" Jul 30 00:58:43.138504 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:43.138378 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6" event={"ID":"2ee95f87-8205-45e8-be5d-d6a94ccc7992","Type":"ContainerDied","Data":"9924ed6b5460d855b4b47d5dae83d2e1d9fb15ee31a3c590974be3614d7a09fa"} Jul 30 00:58:43.138504 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:43.138427 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6" event={"ID":"2ee95f87-8205-45e8-be5d-d6a94ccc7992","Type":"ContainerDied","Data":"b8f307c224222698e4a33cb2265394407e9dd3b10424e352a2a94e41640dc06c"} Jul 30 00:58:43.138504 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:43.138448 2563 scope.go:117] "RemoveContainer" containerID="9924ed6b5460d855b4b47d5dae83d2e1d9fb15ee31a3c590974be3614d7a09fa" Jul 30 00:58:43.148839 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:43.148819 2563 scope.go:117] "RemoveContainer" containerID="f83e61466862f0dcfaac97b1387ab6c14e18e4ab32f732cf0d5a4eb2dbedf4c8" Jul 30 00:58:43.157577 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:43.157529 2563 scope.go:117] "RemoveContainer" containerID="9924ed6b5460d855b4b47d5dae83d2e1d9fb15ee31a3c590974be3614d7a09fa" Jul 30 00:58:43.158297 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:58:43.158269 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9924ed6b5460d855b4b47d5dae83d2e1d9fb15ee31a3c590974be3614d7a09fa\": container with ID starting with 9924ed6b5460d855b4b47d5dae83d2e1d9fb15ee31a3c590974be3614d7a09fa not found: ID does not exist" containerID="9924ed6b5460d855b4b47d5dae83d2e1d9fb15ee31a3c590974be3614d7a09fa" Jul 30 00:58:43.158401 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:43.158308 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9924ed6b5460d855b4b47d5dae83d2e1d9fb15ee31a3c590974be3614d7a09fa"} err="failed to get container status \"9924ed6b5460d855b4b47d5dae83d2e1d9fb15ee31a3c590974be3614d7a09fa\": rpc error: code = NotFound desc = could not find container \"9924ed6b5460d855b4b47d5dae83d2e1d9fb15ee31a3c590974be3614d7a09fa\": container with ID starting with 9924ed6b5460d855b4b47d5dae83d2e1d9fb15ee31a3c590974be3614d7a09fa not found: ID does not exist" Jul 30 00:58:43.158401 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:43.158331 2563 scope.go:117] "RemoveContainer" containerID="f83e61466862f0dcfaac97b1387ab6c14e18e4ab32f732cf0d5a4eb2dbedf4c8" Jul 30 00:58:43.158614 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:58:43.158594 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f83e61466862f0dcfaac97b1387ab6c14e18e4ab32f732cf0d5a4eb2dbedf4c8\": container with ID starting with f83e61466862f0dcfaac97b1387ab6c14e18e4ab32f732cf0d5a4eb2dbedf4c8 not found: ID does not exist" containerID="f83e61466862f0dcfaac97b1387ab6c14e18e4ab32f732cf0d5a4eb2dbedf4c8" Jul 30 00:58:43.158677 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:43.158619 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f83e61466862f0dcfaac97b1387ab6c14e18e4ab32f732cf0d5a4eb2dbedf4c8"} err="failed to get container status \"f83e61466862f0dcfaac97b1387ab6c14e18e4ab32f732cf0d5a4eb2dbedf4c8\": rpc error: code = NotFound desc = could not find container \"f83e61466862f0dcfaac97b1387ab6c14e18e4ab32f732cf0d5a4eb2dbedf4c8\": container with ID starting with f83e61466862f0dcfaac97b1387ab6c14e18e4ab32f732cf0d5a4eb2dbedf4c8 not found: ID does not exist" Jul 30 00:58:43.160298 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:43.160273 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6"] Jul 30 00:58:43.164753 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:43.164712 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-772c30-openshift-default-678fd7cd88-gmdm6"] Jul 30 00:58:44.476057 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:44.476017 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ee95f87-8205-45e8-be5d-d6a94ccc7992" path="/var/lib/kubelet/pods/2ee95f87-8205-45e8-be5d-d6a94ccc7992/volumes" Jul 30 00:58:53.185146 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:53.185111 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:58:53.250279 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:53.250232 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:58:53.308819 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:53.308781 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:58:53.366424 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:53.366386 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:58:53.699021 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:53.698988 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:58:59.624049 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:59.624009 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln"] Jul 30 00:58:59.624480 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:58:59.624240 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln" podUID="fd030969-963a-4fe8-89b4-37508b74e303" containerName="istio-proxy" containerID="cri-o://8a336d907849f2dc16530b16b1ce69b0c1d8a6d9f63fcb0792f9149908421bd1" gracePeriod=30 Jul 30 00:59:00.440593 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:00.440551 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8"] Jul 30 00:59:00.441169 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:00.441153 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2" containerName="istio-proxy" Jul 30 00:59:00.441268 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:00.441171 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9ea2c30-3e64-49fc-8f4d-d03339b3f8c2" containerName="istio-proxy" Jul 30 00:59:00.441268 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:00.441185 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ee95f87-8205-45e8-be5d-d6a94ccc7992" containerName="istio-proxy" Jul 30 00:59:00.441268 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:00.441193 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ee95f87-8205-45e8-be5d-d6a94ccc7992" containerName="istio-proxy" Jul 30 00:59:00.441367 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:00.441300 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ee95f87-8205-45e8-be5d-d6a94ccc7992" containerName="istio-proxy" Jul 30 00:59:00.441367 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:00.441317 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ee95f87-8205-45e8-be5d-d6a94ccc7992" containerName="istio-proxy" Jul 30 00:59:00.444883 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:00.444866 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8" Jul 30 00:59:00.447578 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:00.447554 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-a47fe8-openshift-default-dockercfg-kxdk7\"" Jul 30 00:59:00.457048 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:00.457026 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8"] Jul 30 00:59:00.545890 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:00.545777 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:59:00.577978 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:00.577942 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7fe7def-87fe-4774-9f6a-c47701fd6096-istiod-ca-cert\") pod \"e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8\" (UID: \"f7fe7def-87fe-4774-9f6a-c47701fd6096\") " pod="openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8" Jul 30 00:59:00.578167 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:00.577994 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7fe7def-87fe-4774-9f6a-c47701fd6096-credential-socket\") pod \"e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8\" (UID: \"f7fe7def-87fe-4774-9f6a-c47701fd6096\") " pod="openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8" Jul 30 00:59:00.578167 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:00.578050 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7fe7def-87fe-4774-9f6a-c47701fd6096-istio-data\") pod \"e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8\" (UID: \"f7fe7def-87fe-4774-9f6a-c47701fd6096\") " pod="openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8" Jul 30 00:59:00.578167 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:00.578076 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7fe7def-87fe-4774-9f6a-c47701fd6096-istio-token\") pod \"e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8\" (UID: \"f7fe7def-87fe-4774-9f6a-c47701fd6096\") " pod="openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8" Jul 30 00:59:00.578167 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:00.578137 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kv9q6\" (UniqueName: \"kubernetes.io/projected/f7fe7def-87fe-4774-9f6a-c47701fd6096-kube-api-access-kv9q6\") pod \"e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8\" (UID: \"f7fe7def-87fe-4774-9f6a-c47701fd6096\") " pod="openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8" Jul 30 00:59:00.578167 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:00.578158 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7fe7def-87fe-4774-9f6a-c47701fd6096-workload-socket\") pod \"e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8\" (UID: \"f7fe7def-87fe-4774-9f6a-c47701fd6096\") " pod="openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8" Jul 30 00:59:00.578350 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:00.578200 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7fe7def-87fe-4774-9f6a-c47701fd6096-workload-certs\") pod \"e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8\" (UID: \"f7fe7def-87fe-4774-9f6a-c47701fd6096\") " pod="openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8" Jul 30 00:59:00.578350 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:00.578227 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7fe7def-87fe-4774-9f6a-c47701fd6096-istio-podinfo\") pod \"e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8\" (UID: \"f7fe7def-87fe-4774-9f6a-c47701fd6096\") " pod="openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8" Jul 30 00:59:00.578350 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:00.578250 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7fe7def-87fe-4774-9f6a-c47701fd6096-istio-envoy\") pod \"e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8\" (UID: \"f7fe7def-87fe-4774-9f6a-c47701fd6096\") " pod="openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8" Jul 30 00:59:00.679294 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:00.679257 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kv9q6\" (UniqueName: \"kubernetes.io/projected/f7fe7def-87fe-4774-9f6a-c47701fd6096-kube-api-access-kv9q6\") pod \"e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8\" (UID: \"f7fe7def-87fe-4774-9f6a-c47701fd6096\") " pod="openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8" Jul 30 00:59:00.679294 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:00.679296 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7fe7def-87fe-4774-9f6a-c47701fd6096-workload-socket\") pod \"e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8\" (UID: \"f7fe7def-87fe-4774-9f6a-c47701fd6096\") " pod="openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8" Jul 30 00:59:00.679855 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:00.679342 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7fe7def-87fe-4774-9f6a-c47701fd6096-workload-certs\") pod \"e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8\" (UID: \"f7fe7def-87fe-4774-9f6a-c47701fd6096\") " pod="openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8" Jul 30 00:59:00.679855 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:00.679359 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7fe7def-87fe-4774-9f6a-c47701fd6096-istio-podinfo\") pod \"e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8\" (UID: \"f7fe7def-87fe-4774-9f6a-c47701fd6096\") " pod="openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8" Jul 30 00:59:00.679855 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:00.679385 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7fe7def-87fe-4774-9f6a-c47701fd6096-istio-envoy\") pod \"e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8\" (UID: \"f7fe7def-87fe-4774-9f6a-c47701fd6096\") " pod="openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8" Jul 30 00:59:00.679855 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:00.679463 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7fe7def-87fe-4774-9f6a-c47701fd6096-istiod-ca-cert\") pod \"e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8\" (UID: \"f7fe7def-87fe-4774-9f6a-c47701fd6096\") " pod="openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8" Jul 30 00:59:00.679855 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:00.679514 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7fe7def-87fe-4774-9f6a-c47701fd6096-credential-socket\") pod \"e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8\" (UID: \"f7fe7def-87fe-4774-9f6a-c47701fd6096\") " pod="openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8" Jul 30 00:59:00.679855 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:00.679540 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7fe7def-87fe-4774-9f6a-c47701fd6096-istio-data\") pod \"e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8\" (UID: \"f7fe7def-87fe-4774-9f6a-c47701fd6096\") " pod="openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8" Jul 30 00:59:00.679855 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:00.679563 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7fe7def-87fe-4774-9f6a-c47701fd6096-istio-token\") pod \"e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8\" (UID: \"f7fe7def-87fe-4774-9f6a-c47701fd6096\") " pod="openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8" Jul 30 00:59:00.680228 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:00.679959 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7fe7def-87fe-4774-9f6a-c47701fd6096-istio-data\") pod \"e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8\" (UID: \"f7fe7def-87fe-4774-9f6a-c47701fd6096\") " pod="openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8" Jul 30 00:59:00.680228 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:00.679976 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7fe7def-87fe-4774-9f6a-c47701fd6096-credential-socket\") pod \"e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8\" (UID: \"f7fe7def-87fe-4774-9f6a-c47701fd6096\") " pod="openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8" Jul 30 00:59:00.680228 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:00.680074 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7fe7def-87fe-4774-9f6a-c47701fd6096-workload-socket\") pod \"e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8\" (UID: \"f7fe7def-87fe-4774-9f6a-c47701fd6096\") " pod="openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8" Jul 30 00:59:00.680228 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:00.680141 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7fe7def-87fe-4774-9f6a-c47701fd6096-workload-certs\") pod \"e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8\" (UID: \"f7fe7def-87fe-4774-9f6a-c47701fd6096\") " pod="openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8" Jul 30 00:59:00.680469 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:00.680448 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7fe7def-87fe-4774-9f6a-c47701fd6096-istiod-ca-cert\") pod \"e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8\" (UID: \"f7fe7def-87fe-4774-9f6a-c47701fd6096\") " pod="openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8" Jul 30 00:59:00.681970 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:00.681943 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7fe7def-87fe-4774-9f6a-c47701fd6096-istio-podinfo\") pod \"e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8\" (UID: \"f7fe7def-87fe-4774-9f6a-c47701fd6096\") " pod="openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8" Jul 30 00:59:00.682091 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:00.682069 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7fe7def-87fe-4774-9f6a-c47701fd6096-istio-envoy\") pod \"e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8\" (UID: \"f7fe7def-87fe-4774-9f6a-c47701fd6096\") " pod="openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8" Jul 30 00:59:00.687146 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:00.687120 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kv9q6\" (UniqueName: \"kubernetes.io/projected/f7fe7def-87fe-4774-9f6a-c47701fd6096-kube-api-access-kv9q6\") pod \"e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8\" (UID: \"f7fe7def-87fe-4774-9f6a-c47701fd6096\") " pod="openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8" Jul 30 00:59:00.687441 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:00.687420 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7fe7def-87fe-4774-9f6a-c47701fd6096-istio-token\") pod \"e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8\" (UID: \"f7fe7def-87fe-4774-9f6a-c47701fd6096\") " pod="openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8" Jul 30 00:59:00.757890 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:00.757859 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8" Jul 30 00:59:00.908760 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:00.906614 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8"] Jul 30 00:59:00.913998 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:59:00.913967 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7fe7def_87fe_4774_9f6a_c47701fd6096.slice/crio-b77a088fd706d12980af5651987f1e89b85d3eada0e2ada04836e6fd4131094d WatchSource:0}: Error finding container b77a088fd706d12980af5651987f1e89b85d3eada0e2ada04836e6fd4131094d: Status 404 returned error can't find the container with id b77a088fd706d12980af5651987f1e89b85d3eada0e2ada04836e6fd4131094d Jul 30 00:59:00.916230 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:00.916196 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:59:00.916335 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:00.916273 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:59:00.916335 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:00.916312 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:59:01.235925 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:01.235841 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8" event={"ID":"f7fe7def-87fe-4774-9f6a-c47701fd6096","Type":"ContainerStarted","Data":"37f7ffef82d5157f0cad043b61ec19f4b05cd0efe2cca4d840ea6c2a4442b7eb"} Jul 30 00:59:01.235925 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:01.235894 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8" event={"ID":"f7fe7def-87fe-4774-9f6a-c47701fd6096","Type":"ContainerStarted","Data":"b77a088fd706d12980af5651987f1e89b85d3eada0e2ada04836e6fd4131094d"} Jul 30 00:59:01.253267 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:01.253212 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8" podStartSLOduration=1.253194117 podStartE2EDuration="1.253194117s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:59:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:59:01.252578683 +0000 UTC m=+2505.311334663" watchObservedRunningTime="2026-07-30 00:59:01.253194117 +0000 UTC m=+2505.311950085" Jul 30 00:59:01.758819 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:01.758786 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8" Jul 30 00:59:01.763678 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:01.763652 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8" Jul 30 00:59:01.882625 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:01.882595 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:59:01.937019 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:01.936992 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:59:02.241211 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:02.241123 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8" Jul 30 00:59:02.242077 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:02.242057 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8" Jul 30 00:59:03.134788 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:03.134729 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:59:03.231183 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:03.231147 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:59:04.385446 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:04.385412 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:59:04.505615 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:04.505572 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:59:04.819300 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:04.819267 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:59:04.898176 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:04.898121 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln" Jul 30 00:59:04.924644 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:04.924610 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:59:05.016492 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:05.016452 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd030969-963a-4fe8-89b4-37508b74e303-credential-socket\") pod \"fd030969-963a-4fe8-89b4-37508b74e303\" (UID: \"fd030969-963a-4fe8-89b4-37508b74e303\") " Jul 30 00:59:05.016675 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:05.016503 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd030969-963a-4fe8-89b4-37508b74e303-istiod-ca-cert\") pod \"fd030969-963a-4fe8-89b4-37508b74e303\" (UID: \"fd030969-963a-4fe8-89b4-37508b74e303\") " Jul 30 00:59:05.016675 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:05.016523 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd030969-963a-4fe8-89b4-37508b74e303-workload-certs\") pod \"fd030969-963a-4fe8-89b4-37508b74e303\" (UID: \"fd030969-963a-4fe8-89b4-37508b74e303\") " Jul 30 00:59:05.016675 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:05.016560 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd030969-963a-4fe8-89b4-37508b74e303-istio-envoy\") pod \"fd030969-963a-4fe8-89b4-37508b74e303\" (UID: \"fd030969-963a-4fe8-89b4-37508b74e303\") " Jul 30 00:59:05.016675 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:05.016584 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd030969-963a-4fe8-89b4-37508b74e303-workload-socket\") pod \"fd030969-963a-4fe8-89b4-37508b74e303\" (UID: \"fd030969-963a-4fe8-89b4-37508b74e303\") " Jul 30 00:59:05.016675 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:05.016606 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd030969-963a-4fe8-89b4-37508b74e303-istio-data\") pod \"fd030969-963a-4fe8-89b4-37508b74e303\" (UID: \"fd030969-963a-4fe8-89b4-37508b74e303\") " Jul 30 00:59:05.016675 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:05.016638 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cd86z\" (UniqueName: \"kubernetes.io/projected/fd030969-963a-4fe8-89b4-37508b74e303-kube-api-access-cd86z\") pod \"fd030969-963a-4fe8-89b4-37508b74e303\" (UID: \"fd030969-963a-4fe8-89b4-37508b74e303\") " Jul 30 00:59:05.017025 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:05.016689 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd030969-963a-4fe8-89b4-37508b74e303-istio-podinfo\") pod \"fd030969-963a-4fe8-89b4-37508b74e303\" (UID: \"fd030969-963a-4fe8-89b4-37508b74e303\") " Jul 30 00:59:05.017025 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:05.016730 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd030969-963a-4fe8-89b4-37508b74e303-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fd030969-963a-4fe8-89b4-37508b74e303" (UID: "fd030969-963a-4fe8-89b4-37508b74e303"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:59:05.017025 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:05.016772 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd030969-963a-4fe8-89b4-37508b74e303-istio-token\") pod \"fd030969-963a-4fe8-89b4-37508b74e303\" (UID: \"fd030969-963a-4fe8-89b4-37508b74e303\") " Jul 30 00:59:05.017025 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:05.016954 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd030969-963a-4fe8-89b4-37508b74e303-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fd030969-963a-4fe8-89b4-37508b74e303" (UID: "fd030969-963a-4fe8-89b4-37508b74e303"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 00:59:05.017025 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:05.016974 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd030969-963a-4fe8-89b4-37508b74e303-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fd030969-963a-4fe8-89b4-37508b74e303" (UID: "fd030969-963a-4fe8-89b4-37508b74e303"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:59:05.017268 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:05.017143 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd030969-963a-4fe8-89b4-37508b74e303-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fd030969-963a-4fe8-89b4-37508b74e303" (UID: "fd030969-963a-4fe8-89b4-37508b74e303"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:59:05.017268 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:05.017161 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd030969-963a-4fe8-89b4-37508b74e303-credential-socket\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:59:05.017268 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:05.017182 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd030969-963a-4fe8-89b4-37508b74e303-istiod-ca-cert\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:59:05.017268 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:05.017197 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd030969-963a-4fe8-89b4-37508b74e303-workload-certs\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:59:05.017669 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:05.017641 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd030969-963a-4fe8-89b4-37508b74e303-istio-data" (OuterVolumeSpecName: "istio-data") pod "fd030969-963a-4fe8-89b4-37508b74e303" (UID: "fd030969-963a-4fe8-89b4-37508b74e303"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:59:05.019007 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:05.018973 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd030969-963a-4fe8-89b4-37508b74e303-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fd030969-963a-4fe8-89b4-37508b74e303" (UID: "fd030969-963a-4fe8-89b4-37508b74e303"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 00:59:05.019007 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:05.018982 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd030969-963a-4fe8-89b4-37508b74e303-istio-token" (OuterVolumeSpecName: "istio-token") pod "fd030969-963a-4fe8-89b4-37508b74e303" (UID: "fd030969-963a-4fe8-89b4-37508b74e303"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 00:59:05.019313 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:05.019285 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fd030969-963a-4fe8-89b4-37508b74e303-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fd030969-963a-4fe8-89b4-37508b74e303" (UID: "fd030969-963a-4fe8-89b4-37508b74e303"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 30 00:59:05.019313 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:05.019305 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd030969-963a-4fe8-89b4-37508b74e303-kube-api-access-cd86z" (OuterVolumeSpecName: "kube-api-access-cd86z") pod "fd030969-963a-4fe8-89b4-37508b74e303" (UID: "fd030969-963a-4fe8-89b4-37508b74e303"). InnerVolumeSpecName "kube-api-access-cd86z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 00:59:05.118680 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:05.118637 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd030969-963a-4fe8-89b4-37508b74e303-istio-envoy\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:59:05.118680 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:05.118677 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd030969-963a-4fe8-89b4-37508b74e303-workload-socket\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:59:05.118680 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:05.118687 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd030969-963a-4fe8-89b4-37508b74e303-istio-data\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:59:05.118680 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:05.118695 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cd86z\" (UniqueName: \"kubernetes.io/projected/fd030969-963a-4fe8-89b4-37508b74e303-kube-api-access-cd86z\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:59:05.118985 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:05.118704 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd030969-963a-4fe8-89b4-37508b74e303-istio-podinfo\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:59:05.118985 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:05.118712 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd030969-963a-4fe8-89b4-37508b74e303-istio-token\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 00:59:05.267469 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:05.267433 2563 generic.go:356] "Generic (PLEG): container finished" podID="fd030969-963a-4fe8-89b4-37508b74e303" containerID="8a336d907849f2dc16530b16b1ce69b0c1d8a6d9f63fcb0792f9149908421bd1" exitCode=0 Jul 30 00:59:05.267656 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:05.267496 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln" Jul 30 00:59:05.267656 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:05.267509 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln" event={"ID":"fd030969-963a-4fe8-89b4-37508b74e303","Type":"ContainerDied","Data":"8a336d907849f2dc16530b16b1ce69b0c1d8a6d9f63fcb0792f9149908421bd1"} Jul 30 00:59:05.267656 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:05.267574 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln" event={"ID":"fd030969-963a-4fe8-89b4-37508b74e303","Type":"ContainerDied","Data":"9eacd008d614cc6685da2ebae011690623aa2443cfaecfc1f6ffcc76db6f6ffb"} Jul 30 00:59:05.267656 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:05.267595 2563 scope.go:117] "RemoveContainer" containerID="8a336d907849f2dc16530b16b1ce69b0c1d8a6d9f63fcb0792f9149908421bd1" Jul 30 00:59:05.279376 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:05.279348 2563 scope.go:117] "RemoveContainer" containerID="b58c59a7029323dbc6c1416532765ae3f31a91795a0b047103f04486fef2610b" Jul 30 00:59:05.287432 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:05.287414 2563 scope.go:117] "RemoveContainer" containerID="8a336d907849f2dc16530b16b1ce69b0c1d8a6d9f63fcb0792f9149908421bd1" Jul 30 00:59:05.287678 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:59:05.287656 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a336d907849f2dc16530b16b1ce69b0c1d8a6d9f63fcb0792f9149908421bd1\": container with ID starting with 8a336d907849f2dc16530b16b1ce69b0c1d8a6d9f63fcb0792f9149908421bd1 not found: ID does not exist" containerID="8a336d907849f2dc16530b16b1ce69b0c1d8a6d9f63fcb0792f9149908421bd1" Jul 30 00:59:05.287765 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:05.287685 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a336d907849f2dc16530b16b1ce69b0c1d8a6d9f63fcb0792f9149908421bd1"} err="failed to get container status \"8a336d907849f2dc16530b16b1ce69b0c1d8a6d9f63fcb0792f9149908421bd1\": rpc error: code = NotFound desc = could not find container \"8a336d907849f2dc16530b16b1ce69b0c1d8a6d9f63fcb0792f9149908421bd1\": container with ID starting with 8a336d907849f2dc16530b16b1ce69b0c1d8a6d9f63fcb0792f9149908421bd1 not found: ID does not exist" Jul 30 00:59:05.287765 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:05.287701 2563 scope.go:117] "RemoveContainer" containerID="b58c59a7029323dbc6c1416532765ae3f31a91795a0b047103f04486fef2610b" Jul 30 00:59:05.287952 ip-10-0-133-10 kubenswrapper[2563]: E0730 00:59:05.287933 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b58c59a7029323dbc6c1416532765ae3f31a91795a0b047103f04486fef2610b\": container with ID starting with b58c59a7029323dbc6c1416532765ae3f31a91795a0b047103f04486fef2610b not found: ID does not exist" containerID="b58c59a7029323dbc6c1416532765ae3f31a91795a0b047103f04486fef2610b" Jul 30 00:59:05.287993 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:05.287958 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b58c59a7029323dbc6c1416532765ae3f31a91795a0b047103f04486fef2610b"} err="failed to get container status \"b58c59a7029323dbc6c1416532765ae3f31a91795a0b047103f04486fef2610b\": rpc error: code = NotFound desc = could not find container \"b58c59a7029323dbc6c1416532765ae3f31a91795a0b047103f04486fef2610b\": container with ID starting with b58c59a7029323dbc6c1416532765ae3f31a91795a0b047103f04486fef2610b not found: ID does not exist" Jul 30 00:59:05.293241 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:05.293219 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln"] Jul 30 00:59:05.296728 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:05.296707 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-68c25c-openshift-default-6796c7488-k5tln"] Jul 30 00:59:06.476182 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:06.476146 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd030969-963a-4fe8-89b4-37508b74e303" path="/var/lib/kubelet/pods/fd030969-963a-4fe8-89b4-37508b74e303/volumes" Jul 30 00:59:17.271698 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:17.271531 2563 patch_prober.go:28] interesting pod/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": read tcp 10.132.0.2:39946->10.132.0.60:15021: read: connection reset by peer" start-of-body= Jul 30 00:59:17.271698 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:17.271614 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8" podUID="f7fe7def-87fe-4774-9f6a-c47701fd6096" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": read tcp 10.132.0.2:39946->10.132.0.60:15021: read: connection reset by peer" Jul 30 00:59:17.328273 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:17.328233 2563 generic.go:356] "Generic (PLEG): container finished" podID="f7fe7def-87fe-4774-9f6a-c47701fd6096" containerID="37f7ffef82d5157f0cad043b61ec19f4b05cd0efe2cca4d840ea6c2a4442b7eb" exitCode=0 Jul 30 00:59:17.328417 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:17.328302 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8" event={"ID":"f7fe7def-87fe-4774-9f6a-c47701fd6096","Type":"ContainerDied","Data":"37f7ffef82d5157f0cad043b61ec19f4b05cd0efe2cca4d840ea6c2a4442b7eb"} Jul 30 00:59:17.328712 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:17.328697 2563 scope.go:117] "RemoveContainer" containerID="37f7ffef82d5157f0cad043b61ec19f4b05cd0efe2cca4d840ea6c2a4442b7eb" Jul 30 00:59:17.329690 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:17.329663 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:59:17.329779 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:17.329728 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:59:17.329832 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:17.329788 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:59:18.335054 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:18.335016 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8" event={"ID":"f7fe7def-87fe-4774-9f6a-c47701fd6096","Type":"ContainerStarted","Data":"7bfad3da532cd6a3e3c6168fba62ac6d01d6d1468159eaaadbff4d52dadc193c"} Jul 30 00:59:18.759016 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:18.758985 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8" Jul 30 00:59:18.759401 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:18.759286 2563 patch_prober.go:28] interesting pod/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 30 00:59:18.759586 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:18.759558 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8" podUID="f7fe7def-87fe-4774-9f6a-c47701fd6096" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 30 00:59:19.758307 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:19.758269 2563 patch_prober.go:28] interesting pod/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 30 00:59:19.758948 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:19.758914 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8" podUID="f7fe7def-87fe-4774-9f6a-c47701fd6096" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 30 00:59:20.758802 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:20.758758 2563 patch_prober.go:28] interesting pod/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 30 00:59:20.759304 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:20.758834 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8" podUID="f7fe7def-87fe-4774-9f6a-c47701fd6096" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 30 00:59:21.758915 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:21.758871 2563 patch_prober.go:28] interesting pod/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 30 00:59:21.759413 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:21.758944 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8" podUID="f7fe7def-87fe-4774-9f6a-c47701fd6096" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 30 00:59:22.761978 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:22.761940 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8" Jul 30 00:59:22.762376 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:22.762203 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8" Jul 30 00:59:22.762827 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:22.762810 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8" Jul 30 00:59:28.054972 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:28.054932 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:59:28.362160 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:28.362063 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:59:28.631190 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:28.631110 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb"] Jul 30 00:59:28.631540 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:28.631526 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ee95f87-8205-45e8-be5d-d6a94ccc7992" containerName="istio-proxy" Jul 30 00:59:28.631599 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:28.631541 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ee95f87-8205-45e8-be5d-d6a94ccc7992" containerName="istio-proxy" Jul 30 00:59:28.631599 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:28.631551 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd030969-963a-4fe8-89b4-37508b74e303" containerName="istio-proxy" Jul 30 00:59:28.631599 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:28.631556 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd030969-963a-4fe8-89b4-37508b74e303" containerName="istio-proxy" Jul 30 00:59:28.631599 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:28.631564 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd030969-963a-4fe8-89b4-37508b74e303" containerName="istio-proxy" Jul 30 00:59:28.631599 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:28.631570 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd030969-963a-4fe8-89b4-37508b74e303" containerName="istio-proxy" Jul 30 00:59:28.631796 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:28.631632 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd030969-963a-4fe8-89b4-37508b74e303" containerName="istio-proxy" Jul 30 00:59:28.631796 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:28.631641 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd030969-963a-4fe8-89b4-37508b74e303" containerName="istio-proxy" Jul 30 00:59:28.635067 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:28.635046 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb" Jul 30 00:59:28.637990 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:28.637970 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-c4c410-openshift-default-dockercfg-gc7kv\"" Jul 30 00:59:28.647699 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:28.647677 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb"] Jul 30 00:59:28.657307 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:28.657277 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:59:28.747067 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:28.747027 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85b6e832-ee6a-494c-a8e0-57b37f5abfe8-istio-podinfo\") pod \"e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb\" (UID: \"85b6e832-ee6a-494c-a8e0-57b37f5abfe8\") " pod="openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb" Jul 30 00:59:28.747239 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:28.747115 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jd2xm\" (UniqueName: \"kubernetes.io/projected/85b6e832-ee6a-494c-a8e0-57b37f5abfe8-kube-api-access-jd2xm\") pod \"e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb\" (UID: \"85b6e832-ee6a-494c-a8e0-57b37f5abfe8\") " pod="openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb" Jul 30 00:59:28.747239 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:28.747194 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85b6e832-ee6a-494c-a8e0-57b37f5abfe8-workload-certs\") pod \"e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb\" (UID: \"85b6e832-ee6a-494c-a8e0-57b37f5abfe8\") " pod="openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb" Jul 30 00:59:28.747239 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:28.747230 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85b6e832-ee6a-494c-a8e0-57b37f5abfe8-istio-token\") pod \"e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb\" (UID: \"85b6e832-ee6a-494c-a8e0-57b37f5abfe8\") " pod="openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb" Jul 30 00:59:28.747605 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:28.747255 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85b6e832-ee6a-494c-a8e0-57b37f5abfe8-istiod-ca-cert\") pod \"e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb\" (UID: \"85b6e832-ee6a-494c-a8e0-57b37f5abfe8\") " pod="openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb" Jul 30 00:59:28.747605 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:28.747280 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85b6e832-ee6a-494c-a8e0-57b37f5abfe8-istio-envoy\") pod \"e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb\" (UID: \"85b6e832-ee6a-494c-a8e0-57b37f5abfe8\") " pod="openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb" Jul 30 00:59:28.747605 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:28.747317 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85b6e832-ee6a-494c-a8e0-57b37f5abfe8-credential-socket\") pod \"e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb\" (UID: \"85b6e832-ee6a-494c-a8e0-57b37f5abfe8\") " pod="openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb" Jul 30 00:59:28.747605 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:28.747422 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85b6e832-ee6a-494c-a8e0-57b37f5abfe8-workload-socket\") pod \"e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb\" (UID: \"85b6e832-ee6a-494c-a8e0-57b37f5abfe8\") " pod="openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb" Jul 30 00:59:28.747605 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:28.747454 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85b6e832-ee6a-494c-a8e0-57b37f5abfe8-istio-data\") pod \"e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb\" (UID: \"85b6e832-ee6a-494c-a8e0-57b37f5abfe8\") " pod="openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb" Jul 30 00:59:28.848378 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:28.848344 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85b6e832-ee6a-494c-a8e0-57b37f5abfe8-istio-podinfo\") pod \"e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb\" (UID: \"85b6e832-ee6a-494c-a8e0-57b37f5abfe8\") " pod="openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb" Jul 30 00:59:28.848378 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:28.848382 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jd2xm\" (UniqueName: \"kubernetes.io/projected/85b6e832-ee6a-494c-a8e0-57b37f5abfe8-kube-api-access-jd2xm\") pod \"e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb\" (UID: \"85b6e832-ee6a-494c-a8e0-57b37f5abfe8\") " pod="openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb" Jul 30 00:59:28.848643 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:28.848569 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85b6e832-ee6a-494c-a8e0-57b37f5abfe8-workload-certs\") pod \"e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb\" (UID: \"85b6e832-ee6a-494c-a8e0-57b37f5abfe8\") " pod="openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb" Jul 30 00:59:28.848643 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:28.848604 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85b6e832-ee6a-494c-a8e0-57b37f5abfe8-istio-token\") pod \"e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb\" (UID: \"85b6e832-ee6a-494c-a8e0-57b37f5abfe8\") " pod="openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb" Jul 30 00:59:28.848643 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:28.848633 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85b6e832-ee6a-494c-a8e0-57b37f5abfe8-istiod-ca-cert\") pod \"e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb\" (UID: \"85b6e832-ee6a-494c-a8e0-57b37f5abfe8\") " pod="openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb" Jul 30 00:59:28.848857 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:28.848662 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85b6e832-ee6a-494c-a8e0-57b37f5abfe8-istio-envoy\") pod \"e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb\" (UID: \"85b6e832-ee6a-494c-a8e0-57b37f5abfe8\") " pod="openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb" Jul 30 00:59:28.848857 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:28.848715 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85b6e832-ee6a-494c-a8e0-57b37f5abfe8-credential-socket\") pod \"e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb\" (UID: \"85b6e832-ee6a-494c-a8e0-57b37f5abfe8\") " pod="openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb" Jul 30 00:59:28.848974 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:28.848860 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85b6e832-ee6a-494c-a8e0-57b37f5abfe8-workload-socket\") pod \"e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb\" (UID: \"85b6e832-ee6a-494c-a8e0-57b37f5abfe8\") " pod="openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb" Jul 30 00:59:28.848974 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:28.848894 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85b6e832-ee6a-494c-a8e0-57b37f5abfe8-istio-data\") pod \"e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb\" (UID: \"85b6e832-ee6a-494c-a8e0-57b37f5abfe8\") " pod="openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb" Jul 30 00:59:28.849084 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:28.849005 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85b6e832-ee6a-494c-a8e0-57b37f5abfe8-workload-certs\") pod \"e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb\" (UID: \"85b6e832-ee6a-494c-a8e0-57b37f5abfe8\") " pod="openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb" Jul 30 00:59:28.849261 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:28.849236 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85b6e832-ee6a-494c-a8e0-57b37f5abfe8-workload-socket\") pod \"e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb\" (UID: \"85b6e832-ee6a-494c-a8e0-57b37f5abfe8\") " pod="openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb" Jul 30 00:59:28.849335 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:28.849249 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85b6e832-ee6a-494c-a8e0-57b37f5abfe8-istio-data\") pod \"e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb\" (UID: \"85b6e832-ee6a-494c-a8e0-57b37f5abfe8\") " pod="openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb" Jul 30 00:59:28.849335 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:28.849279 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85b6e832-ee6a-494c-a8e0-57b37f5abfe8-credential-socket\") pod \"e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb\" (UID: \"85b6e832-ee6a-494c-a8e0-57b37f5abfe8\") " pod="openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb" Jul 30 00:59:28.849475 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:28.849451 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85b6e832-ee6a-494c-a8e0-57b37f5abfe8-istiod-ca-cert\") pod \"e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb\" (UID: \"85b6e832-ee6a-494c-a8e0-57b37f5abfe8\") " pod="openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb" Jul 30 00:59:28.851003 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:28.850972 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85b6e832-ee6a-494c-a8e0-57b37f5abfe8-istio-envoy\") pod \"e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb\" (UID: \"85b6e832-ee6a-494c-a8e0-57b37f5abfe8\") " pod="openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb" Jul 30 00:59:28.851121 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:28.851105 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85b6e832-ee6a-494c-a8e0-57b37f5abfe8-istio-podinfo\") pod \"e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb\" (UID: \"85b6e832-ee6a-494c-a8e0-57b37f5abfe8\") " pod="openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb" Jul 30 00:59:28.857162 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:28.857127 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jd2xm\" (UniqueName: \"kubernetes.io/projected/85b6e832-ee6a-494c-a8e0-57b37f5abfe8-kube-api-access-jd2xm\") pod \"e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb\" (UID: \"85b6e832-ee6a-494c-a8e0-57b37f5abfe8\") " pod="openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb" Jul 30 00:59:28.857952 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:28.857931 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85b6e832-ee6a-494c-a8e0-57b37f5abfe8-istio-token\") pod \"e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb\" (UID: \"85b6e832-ee6a-494c-a8e0-57b37f5abfe8\") " pod="openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb" Jul 30 00:59:28.946164 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:28.946065 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb" Jul 30 00:59:28.977001 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:28.976971 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:59:29.093021 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:29.092984 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb"] Jul 30 00:59:29.100214 ip-10-0-133-10 kubenswrapper[2563]: W0730 00:59:29.100186 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85b6e832_ee6a_494c_a8e0_57b37f5abfe8.slice/crio-adf12e4c738b202f5323826ae0881f09f456359c1d47a62b3e13f8a06e61971b WatchSource:0}: Error finding container adf12e4c738b202f5323826ae0881f09f456359c1d47a62b3e13f8a06e61971b: Status 404 returned error can't find the container with id adf12e4c738b202f5323826ae0881f09f456359c1d47a62b3e13f8a06e61971b Jul 30 00:59:29.102287 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:29.102256 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:59:29.102368 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:29.102320 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:59:29.102368 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:29.102346 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:59:29.390898 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:29.390866 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb" event={"ID":"85b6e832-ee6a-494c-a8e0-57b37f5abfe8","Type":"ContainerStarted","Data":"8bf65bf0b62f04687f999fcf15b2f6d6b8d7ed9cc6954770c7ec0e5337e7852c"} Jul 30 00:59:29.391207 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:29.390918 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb" event={"ID":"85b6e832-ee6a-494c-a8e0-57b37f5abfe8","Type":"ContainerStarted","Data":"adf12e4c738b202f5323826ae0881f09f456359c1d47a62b3e13f8a06e61971b"} Jul 30 00:59:29.410643 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:29.410593 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb" podStartSLOduration=1.410575659 podStartE2EDuration="1.410575659s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:59:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:59:29.41001305 +0000 UTC m=+2533.468769054" watchObservedRunningTime="2026-07-30 00:59:29.410575659 +0000 UTC m=+2533.469331628" Jul 30 00:59:29.611090 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:29.611056 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:59:29.946504 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:29.946460 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb" Jul 30 00:59:30.951956 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:30.951924 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb" Jul 30 00:59:31.251699 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:31.251662 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:59:31.354261 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:31.354228 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:59:31.401872 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:31.401845 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb" Jul 30 00:59:31.402827 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:31.402810 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb" Jul 30 00:59:32.378425 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:32.378385 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:59:32.522268 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:32.522235 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:59:32.870369 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:32.870325 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:59:34.609309 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:34.609269 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 00:59:46.403266 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:46.403238 2563 patch_prober.go:28] interesting pod/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 30 00:59:46.403581 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:46.403293 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb" podUID="85b6e832-ee6a-494c-a8e0-57b37f5abfe8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 30 00:59:46.488016 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:46.487985 2563 generic.go:356] "Generic (PLEG): container finished" podID="85b6e832-ee6a-494c-a8e0-57b37f5abfe8" containerID="8bf65bf0b62f04687f999fcf15b2f6d6b8d7ed9cc6954770c7ec0e5337e7852c" exitCode=0 Jul 30 00:59:46.488194 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:46.488069 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb" event={"ID":"85b6e832-ee6a-494c-a8e0-57b37f5abfe8","Type":"ContainerDied","Data":"8bf65bf0b62f04687f999fcf15b2f6d6b8d7ed9cc6954770c7ec0e5337e7852c"} Jul 30 00:59:46.488503 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:46.488488 2563 scope.go:117] "RemoveContainer" containerID="8bf65bf0b62f04687f999fcf15b2f6d6b8d7ed9cc6954770c7ec0e5337e7852c" Jul 30 00:59:46.489507 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:46.489480 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:59:46.489575 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:46.489551 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:59:46.489618 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:46.489582 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 00:59:47.494995 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:47.494960 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb" event={"ID":"85b6e832-ee6a-494c-a8e0-57b37f5abfe8","Type":"ContainerStarted","Data":"8a48bf8dd772636bae484392b24cdf08ea1ac6902b31ed91b6c32db60bbd56c0"} Jul 30 00:59:47.947260 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:47.947231 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb" Jul 30 00:59:47.947504 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:47.947476 2563 patch_prober.go:28] interesting pod/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 30 00:59:47.947628 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:47.947535 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb" podUID="85b6e832-ee6a-494c-a8e0-57b37f5abfe8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 30 00:59:48.947113 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:48.947076 2563 patch_prober.go:28] interesting pod/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 30 00:59:48.947525 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:48.947130 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb" podUID="85b6e832-ee6a-494c-a8e0-57b37f5abfe8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 30 00:59:49.947040 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:49.947007 2563 patch_prober.go:28] interesting pod/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 30 00:59:49.947220 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:49.947061 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb" podUID="85b6e832-ee6a-494c-a8e0-57b37f5abfe8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 30 00:59:50.946483 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:50.946444 2563 patch_prober.go:28] interesting pod/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 30 00:59:50.946669 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:50.946502 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb" podUID="85b6e832-ee6a-494c-a8e0-57b37f5abfe8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 30 00:59:51.950087 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:51.950054 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb" Jul 30 00:59:51.950464 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:51.950315 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb" Jul 30 00:59:51.950871 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:51.950855 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb" Jul 30 00:59:56.205094 ip-10-0-133-10 kubenswrapper[2563]: I0730 00:59:56.205057 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:00:03.352865 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:00:03.352830 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:00:12.080905 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:00:12.080867 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:00:38.053014 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:00:38.052966 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:00:44.818586 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:00:44.818505 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:00:44.879228 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:00:44.879190 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:01:19.714501 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:01:19.714463 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:01:23.778147 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:01:23.778114 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:02:16.613064 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:02:16.612993 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-b92tw_0ef8f120-f712-4ed9-adbd-40f785d4aa9d/console-operator/1.log" Jul 30 01:02:16.615199 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:02:16.615178 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-b92tw_0ef8f120-f712-4ed9-adbd-40f785d4aa9d/console-operator/1.log" Jul 30 01:04:52.169819 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:04:52.169782 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:04:53.040874 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:04:53.040841 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:05:13.551784 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:05:13.551560 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:06:00.025471 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:06:00.025434 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:06:43.394291 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:06:43.394203 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:06:56.714023 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:06:56.713994 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:06:57.725907 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:06:57.725870 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:07:14.745950 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:07:14.745914 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:07:16.644198 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:07:16.644166 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-b92tw_0ef8f120-f712-4ed9-adbd-40f785d4aa9d/console-operator/1.log" Jul 30 01:07:16.648133 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:07:16.648112 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-b92tw_0ef8f120-f712-4ed9-adbd-40f785d4aa9d/console-operator/1.log" Jul 30 01:07:33.050696 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:07:33.050651 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:08:37.449648 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:08:37.449570 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:08:38.356930 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:08:38.356895 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:08:57.366298 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:08:57.366258 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:08:57.662970 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:08:57.662873 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:08:57.859182 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:08:57.859141 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:09:04.651187 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:09:04.651137 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:09:04.749210 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:09:04.749155 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:09:05.772179 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:09:05.772143 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:09:05.875292 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:09:05.875235 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:09:06.287279 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:09:06.287245 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:09:06.365327 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:09:06.365289 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:09:07.026854 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:09:07.026814 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:09:29.593349 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:09:29.593317 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:09:30.233416 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:09:30.233381 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:09:30.511217 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:09:30.511179 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:09:30.744459 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:09:30.744421 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:09:34.036523 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:09:34.036487 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:09:34.141063 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:09:34.141027 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:09:35.637346 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:09:35.637309 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:09:36.201115 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:09:36.201079 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:09:36.303425 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:09:36.303389 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:09:40.886612 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:09:40.886526 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:09:40.943283 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:09:40.943251 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:09:48.619730 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:09:48.619695 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:09:53.184493 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:09:53.184454 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:09:53.560159 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:09:53.560130 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:09:54.544687 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:09:54.544653 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:09:54.645000 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:09:54.644970 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:09:59.260220 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:09:59.260187 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-29cmx"] Jul 30 01:09:59.263759 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:09:59.263724 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-29cmx" Jul 30 01:09:59.272292 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:09:59.272268 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-29cmx"] Jul 30 01:09:59.310160 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:09:59.310123 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2c78q\" (UniqueName: \"kubernetes.io/projected/8f5ba62e-4c38-46a0-8707-c5946927cb31-kube-api-access-2c78q\") pod \"kuadrant-operator-catalog-29cmx\" (UID: \"8f5ba62e-4c38-46a0-8707-c5946927cb31\") " pod="kuadrant-system/kuadrant-operator-catalog-29cmx" Jul 30 01:09:59.411511 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:09:59.411475 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2c78q\" (UniqueName: \"kubernetes.io/projected/8f5ba62e-4c38-46a0-8707-c5946927cb31-kube-api-access-2c78q\") pod \"kuadrant-operator-catalog-29cmx\" (UID: \"8f5ba62e-4c38-46a0-8707-c5946927cb31\") " pod="kuadrant-system/kuadrant-operator-catalog-29cmx" Jul 30 01:09:59.419526 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:09:59.419501 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2c78q\" (UniqueName: \"kubernetes.io/projected/8f5ba62e-4c38-46a0-8707-c5946927cb31-kube-api-access-2c78q\") pod \"kuadrant-operator-catalog-29cmx\" (UID: \"8f5ba62e-4c38-46a0-8707-c5946927cb31\") " pod="kuadrant-system/kuadrant-operator-catalog-29cmx" Jul 30 01:09:59.574398 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:09:59.574312 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-29cmx" Jul 30 01:09:59.713998 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:09:59.713974 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-29cmx"] Jul 30 01:09:59.715623 ip-10-0-133-10 kubenswrapper[2563]: W0730 01:09:59.715597 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f5ba62e_4c38_46a0_8707_c5946927cb31.slice/crio-b8680ecdc02b98fb4f9ef35a52e7dceb661a84dd2e4575d369b742593891e49e WatchSource:0}: Error finding container b8680ecdc02b98fb4f9ef35a52e7dceb661a84dd2e4575d369b742593891e49e: Status 404 returned error can't find the container with id b8680ecdc02b98fb4f9ef35a52e7dceb661a84dd2e4575d369b742593891e49e Jul 30 01:09:59.717071 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:09:59.717043 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 30 01:10:00.652972 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:00.652935 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-29cmx" event={"ID":"8f5ba62e-4c38-46a0-8707-c5946927cb31","Type":"ContainerStarted","Data":"fed49f6e06a4755bcf15be307594b1a34ab301ab6142047d22b657483307a35e"} Jul 30 01:10:00.653351 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:00.652989 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-29cmx" event={"ID":"8f5ba62e-4c38-46a0-8707-c5946927cb31","Type":"ContainerStarted","Data":"b8680ecdc02b98fb4f9ef35a52e7dceb661a84dd2e4575d369b742593891e49e"} Jul 30 01:10:00.669206 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:00.669155 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-29cmx" podStartSLOduration=1.337323343 podStartE2EDuration="1.66913769s" totalImagesPullingTime="331.814347ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:09:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:10:00.667243275 +0000 UTC m=+3164.725999242" watchObservedRunningTime="2026-07-30 01:10:00.66913769 +0000 UTC m=+3164.727893685" Jul 30 01:10:03.709378 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:03.709345 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-5912cb/ipp-route-5912cb-kserve-646b75fc79-w5m6w"] Jul 30 01:10:03.713292 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:03.713268 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-5912cb/ipp-route-5912cb-kserve-646b75fc79-w5m6w" Jul 30 01:10:03.716908 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:03.716882 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-5912cb\"/\"kube-root-ca.crt\"" Jul 30 01:10:03.717040 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:03.716883 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-5912cb\"/\"openshift-service-ca.crt\"" Jul 30 01:10:03.717040 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:03.716918 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-5912cb\"/\"ipp-route-5912cb-kserve-self-signed-certs\"" Jul 30 01:10:03.717040 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:03.716920 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-5912cb\"/\"default-dockercfg-mtqvz\"" Jul 30 01:10:03.723449 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:03.723427 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-5912cb/ipp-route-5912cb-kserve-646b75fc79-w5m6w"] Jul 30 01:10:03.752925 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:03.752898 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1066810c-b530-4597-be77-4135a4cc34b2-tmp-dir\") pod \"ipp-route-5912cb-kserve-646b75fc79-w5m6w\" (UID: \"1066810c-b530-4597-be77-4135a4cc34b2\") " pod="ai-tenant-e2e-ipp-a-5912cb/ipp-route-5912cb-kserve-646b75fc79-w5m6w" Jul 30 01:10:03.753085 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:03.752961 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1066810c-b530-4597-be77-4135a4cc34b2-model-cache\") pod \"ipp-route-5912cb-kserve-646b75fc79-w5m6w\" (UID: \"1066810c-b530-4597-be77-4135a4cc34b2\") " pod="ai-tenant-e2e-ipp-a-5912cb/ipp-route-5912cb-kserve-646b75fc79-w5m6w" Jul 30 01:10:03.753085 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:03.753009 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjsx4\" (UniqueName: \"kubernetes.io/projected/1066810c-b530-4597-be77-4135a4cc34b2-kube-api-access-xjsx4\") pod \"ipp-route-5912cb-kserve-646b75fc79-w5m6w\" (UID: \"1066810c-b530-4597-be77-4135a4cc34b2\") " pod="ai-tenant-e2e-ipp-a-5912cb/ipp-route-5912cb-kserve-646b75fc79-w5m6w" Jul 30 01:10:03.753177 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:03.753090 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1066810c-b530-4597-be77-4135a4cc34b2-tls-certs\") pod \"ipp-route-5912cb-kserve-646b75fc79-w5m6w\" (UID: \"1066810c-b530-4597-be77-4135a4cc34b2\") " pod="ai-tenant-e2e-ipp-a-5912cb/ipp-route-5912cb-kserve-646b75fc79-w5m6w" Jul 30 01:10:03.753177 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:03.753164 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1066810c-b530-4597-be77-4135a4cc34b2-dshm\") pod \"ipp-route-5912cb-kserve-646b75fc79-w5m6w\" (UID: \"1066810c-b530-4597-be77-4135a4cc34b2\") " pod="ai-tenant-e2e-ipp-a-5912cb/ipp-route-5912cb-kserve-646b75fc79-w5m6w" Jul 30 01:10:03.753243 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:03.753201 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1066810c-b530-4597-be77-4135a4cc34b2-home\") pod \"ipp-route-5912cb-kserve-646b75fc79-w5m6w\" (UID: \"1066810c-b530-4597-be77-4135a4cc34b2\") " pod="ai-tenant-e2e-ipp-a-5912cb/ipp-route-5912cb-kserve-646b75fc79-w5m6w" Jul 30 01:10:03.854169 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:03.854134 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1066810c-b530-4597-be77-4135a4cc34b2-tls-certs\") pod \"ipp-route-5912cb-kserve-646b75fc79-w5m6w\" (UID: \"1066810c-b530-4597-be77-4135a4cc34b2\") " pod="ai-tenant-e2e-ipp-a-5912cb/ipp-route-5912cb-kserve-646b75fc79-w5m6w" Jul 30 01:10:03.854383 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:03.854189 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1066810c-b530-4597-be77-4135a4cc34b2-dshm\") pod \"ipp-route-5912cb-kserve-646b75fc79-w5m6w\" (UID: \"1066810c-b530-4597-be77-4135a4cc34b2\") " pod="ai-tenant-e2e-ipp-a-5912cb/ipp-route-5912cb-kserve-646b75fc79-w5m6w" Jul 30 01:10:03.854383 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:03.854225 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1066810c-b530-4597-be77-4135a4cc34b2-home\") pod \"ipp-route-5912cb-kserve-646b75fc79-w5m6w\" (UID: \"1066810c-b530-4597-be77-4135a4cc34b2\") " pod="ai-tenant-e2e-ipp-a-5912cb/ipp-route-5912cb-kserve-646b75fc79-w5m6w" Jul 30 01:10:03.854383 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:03.854270 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1066810c-b530-4597-be77-4135a4cc34b2-tmp-dir\") pod \"ipp-route-5912cb-kserve-646b75fc79-w5m6w\" (UID: \"1066810c-b530-4597-be77-4135a4cc34b2\") " pod="ai-tenant-e2e-ipp-a-5912cb/ipp-route-5912cb-kserve-646b75fc79-w5m6w" Jul 30 01:10:03.854383 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:03.854325 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1066810c-b530-4597-be77-4135a4cc34b2-model-cache\") pod \"ipp-route-5912cb-kserve-646b75fc79-w5m6w\" (UID: \"1066810c-b530-4597-be77-4135a4cc34b2\") " pod="ai-tenant-e2e-ipp-a-5912cb/ipp-route-5912cb-kserve-646b75fc79-w5m6w" Jul 30 01:10:03.854383 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:03.854371 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjsx4\" (UniqueName: \"kubernetes.io/projected/1066810c-b530-4597-be77-4135a4cc34b2-kube-api-access-xjsx4\") pod \"ipp-route-5912cb-kserve-646b75fc79-w5m6w\" (UID: \"1066810c-b530-4597-be77-4135a4cc34b2\") " pod="ai-tenant-e2e-ipp-a-5912cb/ipp-route-5912cb-kserve-646b75fc79-w5m6w" Jul 30 01:10:03.854759 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:03.854713 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1066810c-b530-4597-be77-4135a4cc34b2-tmp-dir\") pod \"ipp-route-5912cb-kserve-646b75fc79-w5m6w\" (UID: \"1066810c-b530-4597-be77-4135a4cc34b2\") " pod="ai-tenant-e2e-ipp-a-5912cb/ipp-route-5912cb-kserve-646b75fc79-w5m6w" Jul 30 01:10:03.854759 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:03.854729 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1066810c-b530-4597-be77-4135a4cc34b2-home\") pod \"ipp-route-5912cb-kserve-646b75fc79-w5m6w\" (UID: \"1066810c-b530-4597-be77-4135a4cc34b2\") " pod="ai-tenant-e2e-ipp-a-5912cb/ipp-route-5912cb-kserve-646b75fc79-w5m6w" Jul 30 01:10:03.854997 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:03.854969 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1066810c-b530-4597-be77-4135a4cc34b2-model-cache\") pod \"ipp-route-5912cb-kserve-646b75fc79-w5m6w\" (UID: \"1066810c-b530-4597-be77-4135a4cc34b2\") " pod="ai-tenant-e2e-ipp-a-5912cb/ipp-route-5912cb-kserve-646b75fc79-w5m6w" Jul 30 01:10:03.856779 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:03.856759 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1066810c-b530-4597-be77-4135a4cc34b2-tls-certs\") pod \"ipp-route-5912cb-kserve-646b75fc79-w5m6w\" (UID: \"1066810c-b530-4597-be77-4135a4cc34b2\") " pod="ai-tenant-e2e-ipp-a-5912cb/ipp-route-5912cb-kserve-646b75fc79-w5m6w" Jul 30 01:10:03.856876 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:03.856804 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1066810c-b530-4597-be77-4135a4cc34b2-dshm\") pod \"ipp-route-5912cb-kserve-646b75fc79-w5m6w\" (UID: \"1066810c-b530-4597-be77-4135a4cc34b2\") " pod="ai-tenant-e2e-ipp-a-5912cb/ipp-route-5912cb-kserve-646b75fc79-w5m6w" Jul 30 01:10:03.861840 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:03.861814 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjsx4\" (UniqueName: \"kubernetes.io/projected/1066810c-b530-4597-be77-4135a4cc34b2-kube-api-access-xjsx4\") pod \"ipp-route-5912cb-kserve-646b75fc79-w5m6w\" (UID: \"1066810c-b530-4597-be77-4135a4cc34b2\") " pod="ai-tenant-e2e-ipp-a-5912cb/ipp-route-5912cb-kserve-646b75fc79-w5m6w" Jul 30 01:10:04.006084 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:04.006042 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:10:04.023683 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:04.023651 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-5912cb/ipp-route-5912cb-kserve-646b75fc79-w5m6w" Jul 30 01:10:04.170844 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:04.170813 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-5912cb/ipp-route-5912cb-kserve-646b75fc79-w5m6w"] Jul 30 01:10:04.172333 ip-10-0-133-10 kubenswrapper[2563]: W0730 01:10:04.172306 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1066810c_b530_4597_be77_4135a4cc34b2.slice/crio-c671c28baa7e19d44fea6b09ee937c4414b48e99be0d93eb7ad630f5cfab75f2 WatchSource:0}: Error finding container c671c28baa7e19d44fea6b09ee937c4414b48e99be0d93eb7ad630f5cfab75f2: Status 404 returned error can't find the container with id c671c28baa7e19d44fea6b09ee937c4414b48e99be0d93eb7ad630f5cfab75f2 Jul 30 01:10:04.676924 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:04.676885 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-5912cb/ipp-route-5912cb-kserve-646b75fc79-w5m6w" event={"ID":"1066810c-b530-4597-be77-4135a4cc34b2","Type":"ContainerStarted","Data":"c427d24acb5cc9843b84bcfe22b53b2006b392f48087d87ea9f0e0db2290fae4"} Jul 30 01:10:04.677099 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:04.676934 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-5912cb/ipp-route-5912cb-kserve-646b75fc79-w5m6w" event={"ID":"1066810c-b530-4597-be77-4135a4cc34b2","Type":"ContainerStarted","Data":"c671c28baa7e19d44fea6b09ee937c4414b48e99be0d93eb7ad630f5cfab75f2"} Jul 30 01:10:04.694851 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:04.694807 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-5912cb/ipp-route-5912cb-kserve-646b75fc79-w5m6w" podStartSLOduration=1.694793238 podStartE2EDuration="1.694793238s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:10:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:10:04.693696295 +0000 UTC m=+3168.752452264" watchObservedRunningTime="2026-07-30 01:10:04.694793238 +0000 UTC m=+3168.753549232" Jul 30 01:10:09.574678 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:09.574642 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-29cmx" Jul 30 01:10:09.575196 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:09.574722 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-29cmx" Jul 30 01:10:09.596511 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:09.596486 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-29cmx" Jul 30 01:10:09.726153 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:09.726126 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-29cmx" Jul 30 01:10:09.823093 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:09.823060 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-29cmx"] Jul 30 01:10:11.715795 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:11.715752 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-29cmx" podUID="8f5ba62e-4c38-46a0-8707-c5946927cb31" containerName="registry-server" containerID="cri-o://fed49f6e06a4755bcf15be307594b1a34ab301ab6142047d22b657483307a35e" gracePeriod=2 Jul 30 01:10:11.972321 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:11.972261 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-29cmx" Jul 30 01:10:12.033016 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:12.032985 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2c78q\" (UniqueName: \"kubernetes.io/projected/8f5ba62e-4c38-46a0-8707-c5946927cb31-kube-api-access-2c78q\") pod \"8f5ba62e-4c38-46a0-8707-c5946927cb31\" (UID: \"8f5ba62e-4c38-46a0-8707-c5946927cb31\") " Jul 30 01:10:12.035039 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:12.035000 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f5ba62e-4c38-46a0-8707-c5946927cb31-kube-api-access-2c78q" (OuterVolumeSpecName: "kube-api-access-2c78q") pod "8f5ba62e-4c38-46a0-8707-c5946927cb31" (UID: "8f5ba62e-4c38-46a0-8707-c5946927cb31"). InnerVolumeSpecName "kube-api-access-2c78q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 01:10:12.134210 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:12.134180 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2c78q\" (UniqueName: \"kubernetes.io/projected/8f5ba62e-4c38-46a0-8707-c5946927cb31-kube-api-access-2c78q\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 01:10:12.722414 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:12.722381 2563 generic.go:356] "Generic (PLEG): container finished" podID="8f5ba62e-4c38-46a0-8707-c5946927cb31" containerID="fed49f6e06a4755bcf15be307594b1a34ab301ab6142047d22b657483307a35e" exitCode=0 Jul 30 01:10:12.722937 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:12.722424 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-29cmx" Jul 30 01:10:12.722937 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:12.722468 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-29cmx" event={"ID":"8f5ba62e-4c38-46a0-8707-c5946927cb31","Type":"ContainerDied","Data":"fed49f6e06a4755bcf15be307594b1a34ab301ab6142047d22b657483307a35e"} Jul 30 01:10:12.722937 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:12.722516 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-29cmx" event={"ID":"8f5ba62e-4c38-46a0-8707-c5946927cb31","Type":"ContainerDied","Data":"b8680ecdc02b98fb4f9ef35a52e7dceb661a84dd2e4575d369b742593891e49e"} Jul 30 01:10:12.722937 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:12.722542 2563 scope.go:117] "RemoveContainer" containerID="fed49f6e06a4755bcf15be307594b1a34ab301ab6142047d22b657483307a35e" Jul 30 01:10:12.731859 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:12.731843 2563 scope.go:117] "RemoveContainer" containerID="fed49f6e06a4755bcf15be307594b1a34ab301ab6142047d22b657483307a35e" Jul 30 01:10:12.732128 ip-10-0-133-10 kubenswrapper[2563]: E0730 01:10:12.732107 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fed49f6e06a4755bcf15be307594b1a34ab301ab6142047d22b657483307a35e\": container with ID starting with fed49f6e06a4755bcf15be307594b1a34ab301ab6142047d22b657483307a35e not found: ID does not exist" containerID="fed49f6e06a4755bcf15be307594b1a34ab301ab6142047d22b657483307a35e" Jul 30 01:10:12.732177 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:12.732135 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fed49f6e06a4755bcf15be307594b1a34ab301ab6142047d22b657483307a35e"} err="failed to get container status \"fed49f6e06a4755bcf15be307594b1a34ab301ab6142047d22b657483307a35e\": rpc error: code = NotFound desc = could not find container \"fed49f6e06a4755bcf15be307594b1a34ab301ab6142047d22b657483307a35e\": container with ID starting with fed49f6e06a4755bcf15be307594b1a34ab301ab6142047d22b657483307a35e not found: ID does not exist" Jul 30 01:10:12.739313 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:12.739264 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-29cmx"] Jul 30 01:10:12.743517 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:12.743497 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-29cmx"] Jul 30 01:10:14.024556 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:14.024523 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-5912cb/ipp-route-5912cb-kserve-646b75fc79-w5m6w" Jul 30 01:10:14.024957 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:14.024570 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-5912cb/ipp-route-5912cb-kserve-646b75fc79-w5m6w" Jul 30 01:10:14.036689 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:14.036660 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-5912cb/ipp-route-5912cb-kserve-646b75fc79-w5m6w" Jul 30 01:10:14.475701 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:14.475618 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f5ba62e-4c38-46a0-8707-c5946927cb31" path="/var/lib/kubelet/pods/8f5ba62e-4c38-46a0-8707-c5946927cb31/volumes" Jul 30 01:10:14.744719 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:14.744635 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-5912cb/ipp-route-5912cb-kserve-646b75fc79-w5m6w" Jul 30 01:10:55.849160 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:55.849126 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:10:56.148683 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:56.148575 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:10:56.268730 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:56.268692 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:10:59.358004 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:59.357968 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:10:59.455815 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:10:59.455778 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:11:02.758996 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:11:02.758961 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:11:02.816237 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:11:02.816205 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:11:03.717531 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:11:03.717496 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:11:03.817914 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:11:03.817861 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:11:13.990966 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:11:13.990849 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:11:27.389338 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:11:27.389282 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:11:27.463797 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:11:27.463691 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:11:27.522798 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:11:27.522755 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:11:27.584425 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:11:27.584394 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:11:27.921698 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:11:27.921653 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:11:33.877275 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:11:33.877228 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:11:33.941937 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:11:33.941906 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:11:34.126948 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:11:34.126911 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:11:34.353869 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:11:34.353830 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:11:51.704861 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:11:51.704824 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:11:55.941806 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:11:55.941773 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:11:56.510941 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:11:56.510902 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-5912cb/ipp-route-5912cb-kserve-646b75fc79-w5m6w"] Jul 30 01:11:56.606486 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:11:56.606446 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:12:11.530585 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:12:11.530544 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-5912cb/ipp-route-5912cb-kserve-646b75fc79-w5m6w" podUID="1066810c-b530-4597-be77-4135a4cc34b2" containerName="main" containerID="cri-o://c427d24acb5cc9843b84bcfe22b53b2006b392f48087d87ea9f0e0db2290fae4" gracePeriod=45 Jul 30 01:12:11.790205 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:12:11.790133 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-5912cb/ipp-route-5912cb-kserve-646b75fc79-w5m6w" Jul 30 01:12:11.868868 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:12:11.868833 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1066810c-b530-4597-be77-4135a4cc34b2-tmp-dir\") pod \"1066810c-b530-4597-be77-4135a4cc34b2\" (UID: \"1066810c-b530-4597-be77-4135a4cc34b2\") " Jul 30 01:12:11.869040 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:12:11.868892 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xjsx4\" (UniqueName: \"kubernetes.io/projected/1066810c-b530-4597-be77-4135a4cc34b2-kube-api-access-xjsx4\") pod \"1066810c-b530-4597-be77-4135a4cc34b2\" (UID: \"1066810c-b530-4597-be77-4135a4cc34b2\") " Jul 30 01:12:11.869040 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:12:11.868914 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1066810c-b530-4597-be77-4135a4cc34b2-home\") pod \"1066810c-b530-4597-be77-4135a4cc34b2\" (UID: \"1066810c-b530-4597-be77-4135a4cc34b2\") " Jul 30 01:12:11.869040 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:12:11.868982 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1066810c-b530-4597-be77-4135a4cc34b2-tls-certs\") pod \"1066810c-b530-4597-be77-4135a4cc34b2\" (UID: \"1066810c-b530-4597-be77-4135a4cc34b2\") " Jul 30 01:12:11.869040 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:12:11.869035 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1066810c-b530-4597-be77-4135a4cc34b2-dshm\") pod \"1066810c-b530-4597-be77-4135a4cc34b2\" (UID: \"1066810c-b530-4597-be77-4135a4cc34b2\") " Jul 30 01:12:11.869263 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:12:11.869107 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1066810c-b530-4597-be77-4135a4cc34b2-model-cache\") pod \"1066810c-b530-4597-be77-4135a4cc34b2\" (UID: \"1066810c-b530-4597-be77-4135a4cc34b2\") " Jul 30 01:12:11.869263 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:12:11.869122 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1066810c-b530-4597-be77-4135a4cc34b2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1066810c-b530-4597-be77-4135a4cc34b2" (UID: "1066810c-b530-4597-be77-4135a4cc34b2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 01:12:11.869263 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:12:11.869148 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1066810c-b530-4597-be77-4135a4cc34b2-home" (OuterVolumeSpecName: "home") pod "1066810c-b530-4597-be77-4135a4cc34b2" (UID: "1066810c-b530-4597-be77-4135a4cc34b2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 01:12:11.869409 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:12:11.869376 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1066810c-b530-4597-be77-4135a4cc34b2-model-cache" (OuterVolumeSpecName: "model-cache") pod "1066810c-b530-4597-be77-4135a4cc34b2" (UID: "1066810c-b530-4597-be77-4135a4cc34b2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 01:12:11.869468 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:12:11.869428 2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1066810c-b530-4597-be77-4135a4cc34b2-tmp-dir\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 01:12:11.869468 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:12:11.869457 2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1066810c-b530-4597-be77-4135a4cc34b2-home\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 01:12:11.871233 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:12:11.871206 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1066810c-b530-4597-be77-4135a4cc34b2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1066810c-b530-4597-be77-4135a4cc34b2" (UID: "1066810c-b530-4597-be77-4135a4cc34b2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 01:12:11.871361 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:12:11.871270 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1066810c-b530-4597-be77-4135a4cc34b2-dshm" (OuterVolumeSpecName: "dshm") pod "1066810c-b530-4597-be77-4135a4cc34b2" (UID: "1066810c-b530-4597-be77-4135a4cc34b2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 01:12:11.871361 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:12:11.871275 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1066810c-b530-4597-be77-4135a4cc34b2-kube-api-access-xjsx4" (OuterVolumeSpecName: "kube-api-access-xjsx4") pod "1066810c-b530-4597-be77-4135a4cc34b2" (UID: "1066810c-b530-4597-be77-4135a4cc34b2"). InnerVolumeSpecName "kube-api-access-xjsx4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 01:12:11.970233 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:12:11.970194 2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1066810c-b530-4597-be77-4135a4cc34b2-dshm\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 01:12:11.970233 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:12:11.970225 2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1066810c-b530-4597-be77-4135a4cc34b2-model-cache\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 01:12:11.970443 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:12:11.970247 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xjsx4\" (UniqueName: \"kubernetes.io/projected/1066810c-b530-4597-be77-4135a4cc34b2-kube-api-access-xjsx4\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 01:12:11.970443 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:12:11.970256 2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1066810c-b530-4597-be77-4135a4cc34b2-tls-certs\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 01:12:12.367195 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:12:12.367159 2563 generic.go:356] "Generic (PLEG): container finished" podID="1066810c-b530-4597-be77-4135a4cc34b2" containerID="c427d24acb5cc9843b84bcfe22b53b2006b392f48087d87ea9f0e0db2290fae4" exitCode=0 Jul 30 01:12:12.367393 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:12:12.367208 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-5912cb/ipp-route-5912cb-kserve-646b75fc79-w5m6w" Jul 30 01:12:12.367393 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:12:12.367244 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-5912cb/ipp-route-5912cb-kserve-646b75fc79-w5m6w" event={"ID":"1066810c-b530-4597-be77-4135a4cc34b2","Type":"ContainerDied","Data":"c427d24acb5cc9843b84bcfe22b53b2006b392f48087d87ea9f0e0db2290fae4"} Jul 30 01:12:12.367393 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:12:12.367291 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-5912cb/ipp-route-5912cb-kserve-646b75fc79-w5m6w" event={"ID":"1066810c-b530-4597-be77-4135a4cc34b2","Type":"ContainerDied","Data":"c671c28baa7e19d44fea6b09ee937c4414b48e99be0d93eb7ad630f5cfab75f2"} Jul 30 01:12:12.367393 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:12:12.367310 2563 scope.go:117] "RemoveContainer" containerID="c427d24acb5cc9843b84bcfe22b53b2006b392f48087d87ea9f0e0db2290fae4" Jul 30 01:12:12.381138 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:12:12.380941 2563 scope.go:117] "RemoveContainer" containerID="c427d24acb5cc9843b84bcfe22b53b2006b392f48087d87ea9f0e0db2290fae4" Jul 30 01:12:12.381334 ip-10-0-133-10 kubenswrapper[2563]: E0730 01:12:12.381306 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c427d24acb5cc9843b84bcfe22b53b2006b392f48087d87ea9f0e0db2290fae4\": container with ID starting with c427d24acb5cc9843b84bcfe22b53b2006b392f48087d87ea9f0e0db2290fae4 not found: ID does not exist" containerID="c427d24acb5cc9843b84bcfe22b53b2006b392f48087d87ea9f0e0db2290fae4" Jul 30 01:12:12.381502 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:12:12.381475 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c427d24acb5cc9843b84bcfe22b53b2006b392f48087d87ea9f0e0db2290fae4"} err="failed to get container status \"c427d24acb5cc9843b84bcfe22b53b2006b392f48087d87ea9f0e0db2290fae4\": rpc error: code = NotFound desc = could not find container \"c427d24acb5cc9843b84bcfe22b53b2006b392f48087d87ea9f0e0db2290fae4\": container with ID starting with c427d24acb5cc9843b84bcfe22b53b2006b392f48087d87ea9f0e0db2290fae4 not found: ID does not exist" Jul 30 01:12:12.394464 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:12:12.394441 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-5912cb/ipp-route-5912cb-kserve-646b75fc79-w5m6w"] Jul 30 01:12:12.398267 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:12:12.398232 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-5912cb/ipp-route-5912cb-kserve-646b75fc79-w5m6w"] Jul 30 01:12:12.475603 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:12:12.475570 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1066810c-b530-4597-be77-4135a4cc34b2" path="/var/lib/kubelet/pods/1066810c-b530-4597-be77-4135a4cc34b2/volumes" Jul 30 01:12:16.675399 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:12:16.675367 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-b92tw_0ef8f120-f712-4ed9-adbd-40f785d4aa9d/console-operator/1.log" Jul 30 01:12:16.684169 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:12:16.684148 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-b92tw_0ef8f120-f712-4ed9-adbd-40f785d4aa9d/console-operator/1.log" Jul 30 01:12:27.803322 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:12:27.803286 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:12:33.072348 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:12:33.072306 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:12:34.369615 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:12:34.369584 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:12:40.501496 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:12:40.501375 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:12:46.336621 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:12:46.336546 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:12:46.438387 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:12:46.438356 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:12:47.081368 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:12:47.081315 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:12:47.182316 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:12:47.182280 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:12:53.159037 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:12:53.158987 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:12:53.262993 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:12:53.262957 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:12:53.362230 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:12:53.362177 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:12:53.422572 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:12:53.422472 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:12:53.819878 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:12:53.819843 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:12:59.815976 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:12:59.815922 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:12:59.871696 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:12:59.871654 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:13:00.074997 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:00.074913 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:13:00.269315 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:00.269276 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:13:04.857536 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:04.857499 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:13:04.929830 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:04.929799 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:13:05.030442 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:05.030411 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:13:05.130879 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:05.130788 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:13:05.505411 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:05.505379 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:13:11.725232 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:11.725167 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:13:11.825669 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:11.825627 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:13:11.936409 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:11.936377 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8"] Jul 30 01:13:11.936674 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:11.936647 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8" podUID="f7fe7def-87fe-4774-9f6a-c47701fd6096" containerName="istio-proxy" containerID="cri-o://7bfad3da532cd6a3e3c6168fba62ac6d01d6d1468159eaaadbff4d52dadc193c" gracePeriod=30 Jul 30 01:13:11.966274 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:11.966230 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:13:12.164257 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:12.164209 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:13:17.196640 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:17.196615 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8" Jul 30 01:13:17.281093 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:17.281060 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7fe7def-87fe-4774-9f6a-c47701fd6096-workload-socket\") pod \"f7fe7def-87fe-4774-9f6a-c47701fd6096\" (UID: \"f7fe7def-87fe-4774-9f6a-c47701fd6096\") " Jul 30 01:13:17.281253 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:17.281159 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7fe7def-87fe-4774-9f6a-c47701fd6096-istio-data\") pod \"f7fe7def-87fe-4774-9f6a-c47701fd6096\" (UID: \"f7fe7def-87fe-4774-9f6a-c47701fd6096\") " Jul 30 01:13:17.281253 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:17.281182 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7fe7def-87fe-4774-9f6a-c47701fd6096-istiod-ca-cert\") pod \"f7fe7def-87fe-4774-9f6a-c47701fd6096\" (UID: \"f7fe7def-87fe-4774-9f6a-c47701fd6096\") " Jul 30 01:13:17.281253 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:17.281204 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7fe7def-87fe-4774-9f6a-c47701fd6096-istio-envoy\") pod \"f7fe7def-87fe-4774-9f6a-c47701fd6096\" (UID: \"f7fe7def-87fe-4774-9f6a-c47701fd6096\") " Jul 30 01:13:17.281425 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:17.281251 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kv9q6\" (UniqueName: \"kubernetes.io/projected/f7fe7def-87fe-4774-9f6a-c47701fd6096-kube-api-access-kv9q6\") pod \"f7fe7def-87fe-4774-9f6a-c47701fd6096\" (UID: \"f7fe7def-87fe-4774-9f6a-c47701fd6096\") " Jul 30 01:13:17.281425 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:17.281354 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7fe7def-87fe-4774-9f6a-c47701fd6096-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f7fe7def-87fe-4774-9f6a-c47701fd6096" (UID: "f7fe7def-87fe-4774-9f6a-c47701fd6096"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 01:13:17.281425 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:17.281371 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7fe7def-87fe-4774-9f6a-c47701fd6096-istio-podinfo\") pod \"f7fe7def-87fe-4774-9f6a-c47701fd6096\" (UID: \"f7fe7def-87fe-4774-9f6a-c47701fd6096\") " Jul 30 01:13:17.281425 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:17.281410 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7fe7def-87fe-4774-9f6a-c47701fd6096-credential-socket\") pod \"f7fe7def-87fe-4774-9f6a-c47701fd6096\" (UID: \"f7fe7def-87fe-4774-9f6a-c47701fd6096\") " Jul 30 01:13:17.281551 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:17.281447 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7fe7def-87fe-4774-9f6a-c47701fd6096-istio-token\") pod \"f7fe7def-87fe-4774-9f6a-c47701fd6096\" (UID: \"f7fe7def-87fe-4774-9f6a-c47701fd6096\") " Jul 30 01:13:17.281551 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:17.281518 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7fe7def-87fe-4774-9f6a-c47701fd6096-workload-certs\") pod \"f7fe7def-87fe-4774-9f6a-c47701fd6096\" (UID: \"f7fe7def-87fe-4774-9f6a-c47701fd6096\") " Jul 30 01:13:17.281879 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:17.281849 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7fe7def-87fe-4774-9f6a-c47701fd6096-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f7fe7def-87fe-4774-9f6a-c47701fd6096" (UID: "f7fe7def-87fe-4774-9f6a-c47701fd6096"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 01:13:17.281995 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:17.281906 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7fe7def-87fe-4774-9f6a-c47701fd6096-workload-socket\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 01:13:17.281995 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:17.281970 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7fe7def-87fe-4774-9f6a-c47701fd6096-istio-data" (OuterVolumeSpecName: "istio-data") pod "f7fe7def-87fe-4774-9f6a-c47701fd6096" (UID: "f7fe7def-87fe-4774-9f6a-c47701fd6096"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 01:13:17.282113 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:17.281998 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7fe7def-87fe-4774-9f6a-c47701fd6096-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f7fe7def-87fe-4774-9f6a-c47701fd6096" (UID: "f7fe7def-87fe-4774-9f6a-c47701fd6096"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 01:13:17.282113 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:17.282096 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7fe7def-87fe-4774-9f6a-c47701fd6096-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f7fe7def-87fe-4774-9f6a-c47701fd6096" (UID: "f7fe7def-87fe-4774-9f6a-c47701fd6096"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 01:13:17.283525 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:17.283497 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7fe7def-87fe-4774-9f6a-c47701fd6096-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f7fe7def-87fe-4774-9f6a-c47701fd6096" (UID: "f7fe7def-87fe-4774-9f6a-c47701fd6096"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 01:13:17.283619 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:17.283522 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7fe7def-87fe-4774-9f6a-c47701fd6096-kube-api-access-kv9q6" (OuterVolumeSpecName: "kube-api-access-kv9q6") pod "f7fe7def-87fe-4774-9f6a-c47701fd6096" (UID: "f7fe7def-87fe-4774-9f6a-c47701fd6096"). InnerVolumeSpecName "kube-api-access-kv9q6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 01:13:17.283889 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:17.283867 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f7fe7def-87fe-4774-9f6a-c47701fd6096-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f7fe7def-87fe-4774-9f6a-c47701fd6096" (UID: "f7fe7def-87fe-4774-9f6a-c47701fd6096"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 30 01:13:17.283978 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:17.283886 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7fe7def-87fe-4774-9f6a-c47701fd6096-istio-token" (OuterVolumeSpecName: "istio-token") pod "f7fe7def-87fe-4774-9f6a-c47701fd6096" (UID: "f7fe7def-87fe-4774-9f6a-c47701fd6096"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 01:13:17.383171 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:17.383138 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7fe7def-87fe-4774-9f6a-c47701fd6096-istio-data\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 01:13:17.383171 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:17.383162 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7fe7def-87fe-4774-9f6a-c47701fd6096-istiod-ca-cert\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 01:13:17.383171 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:17.383172 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7fe7def-87fe-4774-9f6a-c47701fd6096-istio-envoy\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 01:13:17.383171 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:17.383180 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kv9q6\" (UniqueName: \"kubernetes.io/projected/f7fe7def-87fe-4774-9f6a-c47701fd6096-kube-api-access-kv9q6\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 01:13:17.383429 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:17.383189 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7fe7def-87fe-4774-9f6a-c47701fd6096-istio-podinfo\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 01:13:17.383429 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:17.383199 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7fe7def-87fe-4774-9f6a-c47701fd6096-credential-socket\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 01:13:17.383429 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:17.383207 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7fe7def-87fe-4774-9f6a-c47701fd6096-istio-token\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 01:13:17.383429 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:17.383216 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7fe7def-87fe-4774-9f6a-c47701fd6096-workload-certs\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 01:13:17.519351 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:17.519317 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:13:17.600439 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:17.600362 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:13:17.702020 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:17.701987 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:13:17.703398 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:17.703372 2563 generic.go:356] "Generic (PLEG): container finished" podID="f7fe7def-87fe-4774-9f6a-c47701fd6096" containerID="7bfad3da532cd6a3e3c6168fba62ac6d01d6d1468159eaaadbff4d52dadc193c" exitCode=0 Jul 30 01:13:17.703531 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:17.703419 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8" Jul 30 01:13:17.703531 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:17.703451 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8" event={"ID":"f7fe7def-87fe-4774-9f6a-c47701fd6096","Type":"ContainerDied","Data":"7bfad3da532cd6a3e3c6168fba62ac6d01d6d1468159eaaadbff4d52dadc193c"} Jul 30 01:13:17.703531 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:17.703503 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8" event={"ID":"f7fe7def-87fe-4774-9f6a-c47701fd6096","Type":"ContainerDied","Data":"b77a088fd706d12980af5651987f1e89b85d3eada0e2ada04836e6fd4131094d"} Jul 30 01:13:17.703531 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:17.703527 2563 scope.go:117] "RemoveContainer" containerID="7bfad3da532cd6a3e3c6168fba62ac6d01d6d1468159eaaadbff4d52dadc193c" Jul 30 01:13:17.713055 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:17.713040 2563 scope.go:117] "RemoveContainer" containerID="37f7ffef82d5157f0cad043b61ec19f4b05cd0efe2cca4d840ea6c2a4442b7eb" Jul 30 01:13:17.721213 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:17.721193 2563 scope.go:117] "RemoveContainer" containerID="7bfad3da532cd6a3e3c6168fba62ac6d01d6d1468159eaaadbff4d52dadc193c" Jul 30 01:13:17.721459 ip-10-0-133-10 kubenswrapper[2563]: E0730 01:13:17.721440 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7bfad3da532cd6a3e3c6168fba62ac6d01d6d1468159eaaadbff4d52dadc193c\": container with ID starting with 7bfad3da532cd6a3e3c6168fba62ac6d01d6d1468159eaaadbff4d52dadc193c not found: ID does not exist" containerID="7bfad3da532cd6a3e3c6168fba62ac6d01d6d1468159eaaadbff4d52dadc193c" Jul 30 01:13:17.721531 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:17.721473 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7bfad3da532cd6a3e3c6168fba62ac6d01d6d1468159eaaadbff4d52dadc193c"} err="failed to get container status \"7bfad3da532cd6a3e3c6168fba62ac6d01d6d1468159eaaadbff4d52dadc193c\": rpc error: code = NotFound desc = could not find container \"7bfad3da532cd6a3e3c6168fba62ac6d01d6d1468159eaaadbff4d52dadc193c\": container with ID starting with 7bfad3da532cd6a3e3c6168fba62ac6d01d6d1468159eaaadbff4d52dadc193c not found: ID does not exist" Jul 30 01:13:17.721531 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:17.721497 2563 scope.go:117] "RemoveContainer" containerID="37f7ffef82d5157f0cad043b61ec19f4b05cd0efe2cca4d840ea6c2a4442b7eb" Jul 30 01:13:17.721773 ip-10-0-133-10 kubenswrapper[2563]: E0730 01:13:17.721755 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37f7ffef82d5157f0cad043b61ec19f4b05cd0efe2cca4d840ea6c2a4442b7eb\": container with ID starting with 37f7ffef82d5157f0cad043b61ec19f4b05cd0efe2cca4d840ea6c2a4442b7eb not found: ID does not exist" containerID="37f7ffef82d5157f0cad043b61ec19f4b05cd0efe2cca4d840ea6c2a4442b7eb" Jul 30 01:13:17.721819 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:17.721780 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37f7ffef82d5157f0cad043b61ec19f4b05cd0efe2cca4d840ea6c2a4442b7eb"} err="failed to get container status \"37f7ffef82d5157f0cad043b61ec19f4b05cd0efe2cca4d840ea6c2a4442b7eb\": rpc error: code = NotFound desc = could not find container \"37f7ffef82d5157f0cad043b61ec19f4b05cd0efe2cca4d840ea6c2a4442b7eb\": container with ID starting with 37f7ffef82d5157f0cad043b61ec19f4b05cd0efe2cca4d840ea6c2a4442b7eb not found: ID does not exist" Jul 30 01:13:17.726096 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:17.726074 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8"] Jul 30 01:13:17.731955 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:17.731929 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-a47fe8-openshift-default-77878545d4-jj5j8"] Jul 30 01:13:17.811994 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:17.811964 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:13:18.092360 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:18.092327 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:13:18.475841 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:18.475766 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7fe7def-87fe-4774-9f6a-c47701fd6096" path="/var/lib/kubelet/pods/f7fe7def-87fe-4774-9f6a-c47701fd6096/volumes" Jul 30 01:13:19.572003 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:19.571941 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:13:19.639752 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:19.639715 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:13:24.330195 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:24.330161 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:13:24.429905 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:24.429869 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:13:24.593579 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:24.593475 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb"] Jul 30 01:13:24.593825 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:24.593691 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb" podUID="85b6e832-ee6a-494c-a8e0-57b37f5abfe8" containerName="istio-proxy" containerID="cri-o://8a48bf8dd772636bae484392b24cdf08ea1ac6902b31ed91b6c32db60bbd56c0" gracePeriod=30 Jul 30 01:13:24.624130 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:24.624099 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:13:24.815766 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:24.815723 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:13:25.228879 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:25.228842 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-54cd44d85-rjqq4_9e19c7ad-fe70-44a0-a535-cb2889a76b52/authorino/0.log" Jul 30 01:13:29.768298 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:29.768261 2563 generic.go:356] "Generic (PLEG): container finished" podID="85b6e832-ee6a-494c-a8e0-57b37f5abfe8" containerID="8a48bf8dd772636bae484392b24cdf08ea1ac6902b31ed91b6c32db60bbd56c0" exitCode=0 Jul 30 01:13:29.768693 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:29.768361 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb" event={"ID":"85b6e832-ee6a-494c-a8e0-57b37f5abfe8","Type":"ContainerDied","Data":"8a48bf8dd772636bae484392b24cdf08ea1ac6902b31ed91b6c32db60bbd56c0"} Jul 30 01:13:29.768693 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:29.768441 2563 scope.go:117] "RemoveContainer" containerID="8bf65bf0b62f04687f999fcf15b2f6d6b8d7ed9cc6954770c7ec0e5337e7852c" Jul 30 01:13:29.875665 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:29.875639 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb" Jul 30 01:13:29.998370 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:29.998334 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85b6e832-ee6a-494c-a8e0-57b37f5abfe8-istiod-ca-cert\") pod \"85b6e832-ee6a-494c-a8e0-57b37f5abfe8\" (UID: \"85b6e832-ee6a-494c-a8e0-57b37f5abfe8\") " Jul 30 01:13:29.998370 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:29.998374 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85b6e832-ee6a-494c-a8e0-57b37f5abfe8-istio-podinfo\") pod \"85b6e832-ee6a-494c-a8e0-57b37f5abfe8\" (UID: \"85b6e832-ee6a-494c-a8e0-57b37f5abfe8\") " Jul 30 01:13:29.998646 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:29.998403 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85b6e832-ee6a-494c-a8e0-57b37f5abfe8-istio-data\") pod \"85b6e832-ee6a-494c-a8e0-57b37f5abfe8\" (UID: \"85b6e832-ee6a-494c-a8e0-57b37f5abfe8\") " Jul 30 01:13:29.998646 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:29.998437 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85b6e832-ee6a-494c-a8e0-57b37f5abfe8-istio-envoy\") pod \"85b6e832-ee6a-494c-a8e0-57b37f5abfe8\" (UID: \"85b6e832-ee6a-494c-a8e0-57b37f5abfe8\") " Jul 30 01:13:29.998646 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:29.998474 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85b6e832-ee6a-494c-a8e0-57b37f5abfe8-workload-certs\") pod \"85b6e832-ee6a-494c-a8e0-57b37f5abfe8\" (UID: \"85b6e832-ee6a-494c-a8e0-57b37f5abfe8\") " Jul 30 01:13:29.998646 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:29.998592 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85b6e832-ee6a-494c-a8e0-57b37f5abfe8-istio-token\") pod \"85b6e832-ee6a-494c-a8e0-57b37f5abfe8\" (UID: \"85b6e832-ee6a-494c-a8e0-57b37f5abfe8\") " Jul 30 01:13:29.998646 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:29.998642 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85b6e832-ee6a-494c-a8e0-57b37f5abfe8-workload-socket\") pod \"85b6e832-ee6a-494c-a8e0-57b37f5abfe8\" (UID: \"85b6e832-ee6a-494c-a8e0-57b37f5abfe8\") " Jul 30 01:13:29.998950 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:29.998664 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85b6e832-ee6a-494c-a8e0-57b37f5abfe8-credential-socket\") pod \"85b6e832-ee6a-494c-a8e0-57b37f5abfe8\" (UID: \"85b6e832-ee6a-494c-a8e0-57b37f5abfe8\") " Jul 30 01:13:29.998950 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:29.998689 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jd2xm\" (UniqueName: \"kubernetes.io/projected/85b6e832-ee6a-494c-a8e0-57b37f5abfe8-kube-api-access-jd2xm\") pod \"85b6e832-ee6a-494c-a8e0-57b37f5abfe8\" (UID: \"85b6e832-ee6a-494c-a8e0-57b37f5abfe8\") " Jul 30 01:13:29.998950 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:29.998914 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85b6e832-ee6a-494c-a8e0-57b37f5abfe8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "85b6e832-ee6a-494c-a8e0-57b37f5abfe8" (UID: "85b6e832-ee6a-494c-a8e0-57b37f5abfe8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 01:13:29.999111 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:29.999075 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85b6e832-ee6a-494c-a8e0-57b37f5abfe8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "85b6e832-ee6a-494c-a8e0-57b37f5abfe8" (UID: "85b6e832-ee6a-494c-a8e0-57b37f5abfe8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 01:13:29.999165 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:29.999082 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85b6e832-ee6a-494c-a8e0-57b37f5abfe8-istiod-ca-cert\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 01:13:29.999165 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:29.999147 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85b6e832-ee6a-494c-a8e0-57b37f5abfe8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "85b6e832-ee6a-494c-a8e0-57b37f5abfe8" (UID: "85b6e832-ee6a-494c-a8e0-57b37f5abfe8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 01:13:29.999317 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:29.999295 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85b6e832-ee6a-494c-a8e0-57b37f5abfe8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "85b6e832-ee6a-494c-a8e0-57b37f5abfe8" (UID: "85b6e832-ee6a-494c-a8e0-57b37f5abfe8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 01:13:29.999498 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:29.999475 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85b6e832-ee6a-494c-a8e0-57b37f5abfe8-istio-data" (OuterVolumeSpecName: "istio-data") pod "85b6e832-ee6a-494c-a8e0-57b37f5abfe8" (UID: "85b6e832-ee6a-494c-a8e0-57b37f5abfe8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 01:13:30.001069 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:30.001040 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/85b6e832-ee6a-494c-a8e0-57b37f5abfe8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "85b6e832-ee6a-494c-a8e0-57b37f5abfe8" (UID: "85b6e832-ee6a-494c-a8e0-57b37f5abfe8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 30 01:13:30.001174 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:30.001149 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85b6e832-ee6a-494c-a8e0-57b37f5abfe8-kube-api-access-jd2xm" (OuterVolumeSpecName: "kube-api-access-jd2xm") pod "85b6e832-ee6a-494c-a8e0-57b37f5abfe8" (UID: "85b6e832-ee6a-494c-a8e0-57b37f5abfe8"). InnerVolumeSpecName "kube-api-access-jd2xm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 01:13:30.001336 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:30.001309 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85b6e832-ee6a-494c-a8e0-57b37f5abfe8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "85b6e832-ee6a-494c-a8e0-57b37f5abfe8" (UID: "85b6e832-ee6a-494c-a8e0-57b37f5abfe8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 01:13:30.001436 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:30.001344 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85b6e832-ee6a-494c-a8e0-57b37f5abfe8-istio-token" (OuterVolumeSpecName: "istio-token") pod "85b6e832-ee6a-494c-a8e0-57b37f5abfe8" (UID: "85b6e832-ee6a-494c-a8e0-57b37f5abfe8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 01:13:30.099850 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:30.099812 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85b6e832-ee6a-494c-a8e0-57b37f5abfe8-istio-podinfo\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 01:13:30.099850 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:30.099843 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85b6e832-ee6a-494c-a8e0-57b37f5abfe8-istio-data\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 01:13:30.099850 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:30.099851 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85b6e832-ee6a-494c-a8e0-57b37f5abfe8-istio-envoy\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 01:13:30.099850 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:30.099860 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85b6e832-ee6a-494c-a8e0-57b37f5abfe8-workload-certs\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 01:13:30.100134 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:30.099868 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85b6e832-ee6a-494c-a8e0-57b37f5abfe8-istio-token\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 01:13:30.100134 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:30.099878 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85b6e832-ee6a-494c-a8e0-57b37f5abfe8-workload-socket\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 01:13:30.100134 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:30.099885 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85b6e832-ee6a-494c-a8e0-57b37f5abfe8-credential-socket\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 01:13:30.100134 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:30.099895 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jd2xm\" (UniqueName: \"kubernetes.io/projected/85b6e832-ee6a-494c-a8e0-57b37f5abfe8-kube-api-access-jd2xm\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 30 01:13:30.364087 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:30.364057 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-jbf4x_9ed1c3a7-6249-4633-b09a-f38dfb35647f/manager/0.log" Jul 30 01:13:30.661989 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:30.661912 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-ktnsg_ac97530b-0479-4196-967b-a1affa4fd298/server/0.log" Jul 30 01:13:30.775529 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:30.775494 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb" event={"ID":"85b6e832-ee6a-494c-a8e0-57b37f5abfe8","Type":"ContainerDied","Data":"adf12e4c738b202f5323826ae0881f09f456359c1d47a62b3e13f8a06e61971b"} Jul 30 01:13:30.776006 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:30.775552 2563 scope.go:117] "RemoveContainer" containerID="8a48bf8dd772636bae484392b24cdf08ea1ac6902b31ed91b6c32db60bbd56c0" Jul 30 01:13:30.776006 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:30.775568 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-xrx9m_2e7054c1-45e5-443c-8564-731e433d469d/manager/0.log" Jul 30 01:13:30.776006 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:30.775507 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb" Jul 30 01:13:30.793471 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:30.793449 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb"] Jul 30 01:13:30.797898 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:30.797877 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-c4c410-openshift-default-6fdc444484-qwcwb"] Jul 30 01:13:30.882856 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:30.882834 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5b74bc5486-frgtd_ce5296aa-53c2-4afb-ab15-02535e8eecd5/copy-manifests/0.log" Jul 30 01:13:30.890439 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:30.890418 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5b74bc5486-frgtd_ce5296aa-53c2-4afb-ab15-02535e8eecd5/manager/0.log" Jul 30 01:13:31.559902 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:31.559874 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29756190-nchqp_f2447b8f-3d7d-48f2-abaf-fd773fd8ecee/cleanup/0.log" Jul 30 01:13:32.475785 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:32.475750 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85b6e832-ee6a-494c-a8e0-57b37f5abfe8" path="/var/lib/kubelet/pods/85b6e832-ee6a-494c-a8e0-57b37f5abfe8/volumes" Jul 30 01:13:33.080918 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:33.080889 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-54cd44d85-rjqq4_9e19c7ad-fe70-44a0-a535-cb2889a76b52/authorino/0.log" Jul 30 01:13:33.190347 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:33.190295 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-z7sxq_eab785d4-4f65-4010-928f-1ff5e99e3ff1/manager/0.log" Jul 30 01:13:33.297416 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:33.297388 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-d9rft_3d79d9f4-f2c8-461b-8f97-fdbe48891d63/manager/0.log" Jul 30 01:13:33.518721 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:33.518692 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-xjfmq_5f54c640-8113-4b8b-831e-57adaed12fa1/registry-server/0.log" Jul 30 01:13:33.631915 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:33.631886 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-k9mh2_71b960ca-fb8c-4fcd-afff-88a9627a3b23/manager/0.log" Jul 30 01:13:33.844136 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:33.844052 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-cxzlt_a38e3d81-da42-4aca-b7cd-7e4a3f27d6c9/manager/0.log" Jul 30 01:13:34.173597 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:34.173519 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h_d9cd41c0-eeee-4253-b07b-3e93b15a60fd/istio-proxy/0.log" Jul 30 01:13:34.383994 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:34.383965 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-94d649dc8-9v89r_41a9676d-a4c8-4390-8de0-a7e6eae32c57/kube-auth-proxy/0.log" Jul 30 01:13:34.596339 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:34.596313 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-gf4jt_70445eac-de72-40b2-ad0e-92667eb8856d/istio-proxy/1.log" Jul 30 01:13:34.913143 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:34.913062 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-877bf98d5-xbkfd_90afa1c1-9879-49ee-9fac-27f2e7e963f3/router/0.log" Jul 30 01:13:35.248232 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:35.248160 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-ch4fl_3391cc48-9da3-4104-9a82-3b2d70e24a30/main/0.log" Jul 30 01:13:35.351123 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:35.351096 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-xhzsb_690de04c-56c5-4cf6-a383-5de5291d1161/main/0.log" Jul 30 01:13:41.479253 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:41.479218 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:13:41.555421 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:41.555391 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:13:42.926271 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:42.926240 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:13:44.577484 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:44.577457 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-j26vw_f099a95a-e24f-4e05-bba1-bccd334f000c/global-pull-secret-syncer/0.log" Jul 30 01:13:44.656348 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:44.656306 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" Jul 30 01:13:44.755696 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:44.755664 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-10.ec2.internal_d23058fd1d3df9b79e28eb07292f4443/haproxy/0.log" Jul 30 01:13:49.242225 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:49.242188 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-54cd44d85-rjqq4_9e19c7ad-fe70-44a0-a535-cb2889a76b52/authorino/0.log" Jul 30 01:13:49.278811 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:49.278783 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-z7sxq_eab785d4-4f65-4010-928f-1ff5e99e3ff1/manager/0.log" Jul 30 01:13:49.311781 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:49.311725 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-d9rft_3d79d9f4-f2c8-461b-8f97-fdbe48891d63/manager/0.log" Jul 30 01:13:49.406152 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:49.406118 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-xjfmq_5f54c640-8113-4b8b-831e-57adaed12fa1/registry-server/0.log" Jul 30 01:13:49.580155 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:49.580120 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-k9mh2_71b960ca-fb8c-4fcd-afff-88a9627a3b23/manager/0.log" Jul 30 01:13:49.705245 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:49.705209 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-cxzlt_a38e3d81-da42-4aca-b7cd-7e4a3f27d6c9/manager/0.log" Jul 30 01:13:51.733920 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:51.733886 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-phzr9_866649cb-1758-4709-82af-f07c8abadf4b/cluster-monitoring-operator/0.log" Jul 30 01:13:52.012558 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:52.012529 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6wt75_40d7b3cc-417f-4c61-a04d-e9dc9625bcfe/node-exporter/0.log" Jul 30 01:13:52.036675 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:52.036645 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6wt75_40d7b3cc-417f-4c61-a04d-e9dc9625bcfe/kube-rbac-proxy/0.log" Jul 30 01:13:52.066615 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:52.066589 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6wt75_40d7b3cc-417f-4c61-a04d-e9dc9625bcfe/init-textfile/0.log" Jul 30 01:13:54.098236 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:54.098204 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-tx22w_5ca5f4f2-1c36-4eeb-a43c-8533b4d52e84/networking-console-plugin/0.log" Jul 30 01:13:54.677825 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:54.677796 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-p9qsq/perf-node-gather-daemonset-j2x2s"] Jul 30 01:13:54.678182 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:54.678170 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85b6e832-ee6a-494c-a8e0-57b37f5abfe8" containerName="istio-proxy" Jul 30 01:13:54.678228 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:54.678184 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="85b6e832-ee6a-494c-a8e0-57b37f5abfe8" containerName="istio-proxy" Jul 30 01:13:54.678228 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:54.678194 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85b6e832-ee6a-494c-a8e0-57b37f5abfe8" containerName="istio-proxy" Jul 30 01:13:54.678228 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:54.678200 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="85b6e832-ee6a-494c-a8e0-57b37f5abfe8" containerName="istio-proxy" Jul 30 01:13:54.678228 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:54.678208 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1066810c-b530-4597-be77-4135a4cc34b2" containerName="main" Jul 30 01:13:54.678228 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:54.678213 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1066810c-b530-4597-be77-4135a4cc34b2" containerName="main" Jul 30 01:13:54.678228 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:54.678226 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7fe7def-87fe-4774-9f6a-c47701fd6096" containerName="istio-proxy" Jul 30 01:13:54.678410 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:54.678233 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7fe7def-87fe-4774-9f6a-c47701fd6096" containerName="istio-proxy" Jul 30 01:13:54.678410 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:54.678247 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f5ba62e-4c38-46a0-8707-c5946927cb31" containerName="registry-server" Jul 30 01:13:54.678410 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:54.678252 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f5ba62e-4c38-46a0-8707-c5946927cb31" containerName="registry-server" Jul 30 01:13:54.678410 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:54.678313 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f5ba62e-4c38-46a0-8707-c5946927cb31" containerName="registry-server" Jul 30 01:13:54.678410 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:54.678321 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7fe7def-87fe-4774-9f6a-c47701fd6096" containerName="istio-proxy" Jul 30 01:13:54.678410 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:54.678328 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1066810c-b530-4597-be77-4135a4cc34b2" containerName="main" Jul 30 01:13:54.678410 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:54.678335 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="85b6e832-ee6a-494c-a8e0-57b37f5abfe8" containerName="istio-proxy" Jul 30 01:13:54.678410 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:54.678341 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7fe7def-87fe-4774-9f6a-c47701fd6096" containerName="istio-proxy" Jul 30 01:13:54.678410 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:54.678347 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="85b6e832-ee6a-494c-a8e0-57b37f5abfe8" containerName="istio-proxy" Jul 30 01:13:54.682462 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:54.682439 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-p9qsq/perf-node-gather-daemonset-j2x2s" Jul 30 01:13:54.686588 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:54.686563 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-p9qsq\"/\"default-dockercfg-6smbp\"" Jul 30 01:13:54.687397 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:54.687376 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-p9qsq\"/\"kube-root-ca.crt\"" Jul 30 01:13:54.700661 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:54.700641 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-p9qsq\"/\"openshift-service-ca.crt\"" Jul 30 01:13:54.705828 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:54.705806 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-p9qsq/perf-node-gather-daemonset-j2x2s"] Jul 30 01:13:54.709260 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:54.709222 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvxzc\" (UniqueName: \"kubernetes.io/projected/839855f2-c477-48c8-8eb4-ff5e5fba1b56-kube-api-access-hvxzc\") pod \"perf-node-gather-daemonset-j2x2s\" (UID: \"839855f2-c477-48c8-8eb4-ff5e5fba1b56\") " pod="openshift-must-gather-p9qsq/perf-node-gather-daemonset-j2x2s" Jul 30 01:13:54.709378 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:54.709318 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/839855f2-c477-48c8-8eb4-ff5e5fba1b56-proc\") pod \"perf-node-gather-daemonset-j2x2s\" (UID: \"839855f2-c477-48c8-8eb4-ff5e5fba1b56\") " pod="openshift-must-gather-p9qsq/perf-node-gather-daemonset-j2x2s" Jul 30 01:13:54.709378 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:54.709367 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/839855f2-c477-48c8-8eb4-ff5e5fba1b56-sys\") pod \"perf-node-gather-daemonset-j2x2s\" (UID: \"839855f2-c477-48c8-8eb4-ff5e5fba1b56\") " pod="openshift-must-gather-p9qsq/perf-node-gather-daemonset-j2x2s" Jul 30 01:13:54.709498 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:54.709403 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/839855f2-c477-48c8-8eb4-ff5e5fba1b56-podres\") pod \"perf-node-gather-daemonset-j2x2s\" (UID: \"839855f2-c477-48c8-8eb4-ff5e5fba1b56\") " pod="openshift-must-gather-p9qsq/perf-node-gather-daemonset-j2x2s" Jul 30 01:13:54.709498 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:54.709441 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/839855f2-c477-48c8-8eb4-ff5e5fba1b56-lib-modules\") pod \"perf-node-gather-daemonset-j2x2s\" (UID: \"839855f2-c477-48c8-8eb4-ff5e5fba1b56\") " pod="openshift-must-gather-p9qsq/perf-node-gather-daemonset-j2x2s" Jul 30 01:13:54.810848 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:54.810816 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvxzc\" (UniqueName: \"kubernetes.io/projected/839855f2-c477-48c8-8eb4-ff5e5fba1b56-kube-api-access-hvxzc\") pod \"perf-node-gather-daemonset-j2x2s\" (UID: \"839855f2-c477-48c8-8eb4-ff5e5fba1b56\") " pod="openshift-must-gather-p9qsq/perf-node-gather-daemonset-j2x2s" Jul 30 01:13:54.811048 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:54.810871 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/839855f2-c477-48c8-8eb4-ff5e5fba1b56-proc\") pod \"perf-node-gather-daemonset-j2x2s\" (UID: \"839855f2-c477-48c8-8eb4-ff5e5fba1b56\") " pod="openshift-must-gather-p9qsq/perf-node-gather-daemonset-j2x2s" Jul 30 01:13:54.811048 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:54.810906 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/839855f2-c477-48c8-8eb4-ff5e5fba1b56-sys\") pod \"perf-node-gather-daemonset-j2x2s\" (UID: \"839855f2-c477-48c8-8eb4-ff5e5fba1b56\") " pod="openshift-must-gather-p9qsq/perf-node-gather-daemonset-j2x2s" Jul 30 01:13:54.811048 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:54.810930 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/839855f2-c477-48c8-8eb4-ff5e5fba1b56-podres\") pod \"perf-node-gather-daemonset-j2x2s\" (UID: \"839855f2-c477-48c8-8eb4-ff5e5fba1b56\") " pod="openshift-must-gather-p9qsq/perf-node-gather-daemonset-j2x2s" Jul 30 01:13:54.811048 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:54.810954 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/839855f2-c477-48c8-8eb4-ff5e5fba1b56-lib-modules\") pod \"perf-node-gather-daemonset-j2x2s\" (UID: \"839855f2-c477-48c8-8eb4-ff5e5fba1b56\") " pod="openshift-must-gather-p9qsq/perf-node-gather-daemonset-j2x2s" Jul 30 01:13:54.811048 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:54.811010 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/839855f2-c477-48c8-8eb4-ff5e5fba1b56-proc\") pod \"perf-node-gather-daemonset-j2x2s\" (UID: \"839855f2-c477-48c8-8eb4-ff5e5fba1b56\") " pod="openshift-must-gather-p9qsq/perf-node-gather-daemonset-j2x2s" Jul 30 01:13:54.811048 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:54.811015 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/839855f2-c477-48c8-8eb4-ff5e5fba1b56-sys\") pod \"perf-node-gather-daemonset-j2x2s\" (UID: \"839855f2-c477-48c8-8eb4-ff5e5fba1b56\") " pod="openshift-must-gather-p9qsq/perf-node-gather-daemonset-j2x2s" Jul 30 01:13:54.811304 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:54.811065 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/839855f2-c477-48c8-8eb4-ff5e5fba1b56-podres\") pod \"perf-node-gather-daemonset-j2x2s\" (UID: \"839855f2-c477-48c8-8eb4-ff5e5fba1b56\") " pod="openshift-must-gather-p9qsq/perf-node-gather-daemonset-j2x2s" Jul 30 01:13:54.811304 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:54.811107 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/839855f2-c477-48c8-8eb4-ff5e5fba1b56-lib-modules\") pod \"perf-node-gather-daemonset-j2x2s\" (UID: \"839855f2-c477-48c8-8eb4-ff5e5fba1b56\") " pod="openshift-must-gather-p9qsq/perf-node-gather-daemonset-j2x2s" Jul 30 01:13:54.824246 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:54.824215 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvxzc\" (UniqueName: \"kubernetes.io/projected/839855f2-c477-48c8-8eb4-ff5e5fba1b56-kube-api-access-hvxzc\") pod \"perf-node-gather-daemonset-j2x2s\" (UID: \"839855f2-c477-48c8-8eb4-ff5e5fba1b56\") " pod="openshift-must-gather-p9qsq/perf-node-gather-daemonset-j2x2s" Jul 30 01:13:54.825065 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:54.825018 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-b92tw_0ef8f120-f712-4ed9-adbd-40f785d4aa9d/console-operator/1.log" Jul 30 01:13:54.830035 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:54.830010 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-b92tw_0ef8f120-f712-4ed9-adbd-40f785d4aa9d/console-operator/2.log" Jul 30 01:13:54.993444 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:54.993363 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-p9qsq/perf-node-gather-daemonset-j2x2s" Jul 30 01:13:55.127315 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:55.127275 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-p9qsq/perf-node-gather-daemonset-j2x2s"] Jul 30 01:13:55.129775 ip-10-0-133-10 kubenswrapper[2563]: W0730 01:13:55.129751 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod839855f2_c477_48c8_8eb4_ff5e5fba1b56.slice/crio-13364fc90a9199f3321255022bdfa6369563ee2c852e54111c3ee823af6dc62a WatchSource:0}: Error finding container 13364fc90a9199f3321255022bdfa6369563ee2c852e54111c3ee823af6dc62a: Status 404 returned error can't find the container with id 13364fc90a9199f3321255022bdfa6369563ee2c852e54111c3ee823af6dc62a Jul 30 01:13:55.470064 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:55.469909 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-jzwkw_7ae2552a-b83f-49a5-b779-c80e727f40b4/download-server/0.log" Jul 30 01:13:55.902457 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:55.902412 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-p9qsq/perf-node-gather-daemonset-j2x2s" event={"ID":"839855f2-c477-48c8-8eb4-ff5e5fba1b56","Type":"ContainerStarted","Data":"18de3c509397fc5bf645f97fcd178d06d8faaf3aa95dc45fd415d64038f62b28"} Jul 30 01:13:55.902645 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:55.902480 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-p9qsq/perf-node-gather-daemonset-j2x2s" event={"ID":"839855f2-c477-48c8-8eb4-ff5e5fba1b56","Type":"ContainerStarted","Data":"13364fc90a9199f3321255022bdfa6369563ee2c852e54111c3ee823af6dc62a"} Jul 30 01:13:55.902645 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:55.902594 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-p9qsq/perf-node-gather-daemonset-j2x2s" Jul 30 01:13:55.919674 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:55.919622 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-p9qsq/perf-node-gather-daemonset-j2x2s" podStartSLOduration=1.919607016 podStartE2EDuration="1.919607016s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 01:13:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:13:55.919190245 +0000 UTC m=+3399.977946213" watchObservedRunningTime="2026-07-30 01:13:55.919607016 +0000 UTC m=+3399.978362983" Jul 30 01:13:56.110067 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:56.110027 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-549c6d54cf-j2fkq_a9fed7ee-e7db-4403-98d6-37274ab4f0e8/volume-data-source-validator/0.log" Jul 30 01:13:57.154166 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:57.154135 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6kh2h_bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166/dns/0.log" Jul 30 01:13:57.179661 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:57.179621 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6kh2h_bb9a9f0d-f590-4e7f-91b6-7e3eb5b5a166/kube-rbac-proxy/0.log" Jul 30 01:13:57.367067 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:57.367037 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zlqlw_add0c4dc-1cb0-4fa3-903a-6ce3b4a8b60a/dns-node-resolver/0.log" Jul 30 01:13:57.934303 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:57.934275 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gp6l9_756ee6d1-bf62-49db-954a-ed5d66384c54/node-ca/0.log" Jul 30 01:13:58.991631 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:58.991597 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h_d9cd41c0-eeee-4253-b07b-3e93b15a60fd/istio-proxy/0.log" Jul 30 01:13:59.160919 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:59.160886 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-94d649dc8-9v89r_41a9676d-a4c8-4390-8de0-a7e6eae32c57/kube-auth-proxy/0.log" Jul 30 01:13:59.223183 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:59.223143 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-gf4jt_70445eac-de72-40b2-ad0e-92667eb8856d/istio-proxy/0.log" Jul 30 01:13:59.248517 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:59.248434 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-gf4jt_70445eac-de72-40b2-ad0e-92667eb8856d/istio-proxy/1.log" Jul 30 01:13:59.356137 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:13:59.356109 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-877bf98d5-xbkfd_90afa1c1-9879-49ee-9fac-27f2e7e963f3/router/0.log" Jul 30 01:14:00.268532 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:14:00.268502 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qg9tq_7d7bcf0e-36d5-488a-aa06-817c6cf5831b/serve-healthcheck-canary/0.log" Jul 30 01:14:00.826440 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:14:00.826410 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-ddm85_3fa91894-839d-4035-8b46-a787fe957864/insights-operator/0.log" Jul 30 01:14:00.826989 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:14:00.826972 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-ddm85_3fa91894-839d-4035-8b46-a787fe957864/insights-operator/1.log" Jul 30 01:14:01.049235 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:14:01.049209 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-t7sqw_78222dee-1367-47b9-a2bc-e97144bbeadc/kube-rbac-proxy/0.log" Jul 30 01:14:01.083070 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:14:01.082979 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-t7sqw_78222dee-1367-47b9-a2bc-e97144bbeadc/exporter/0.log" Jul 30 01:14:01.103711 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:14:01.103686 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-t7sqw_78222dee-1367-47b9-a2bc-e97144bbeadc/extractor/0.log" Jul 30 01:14:01.918295 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:14:01.918269 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-p9qsq/perf-node-gather-daemonset-j2x2s" Jul 30 01:14:03.510033 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:14:03.509976 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9m2mr_ad9e265e-16ad-428a-a69a-f9a7948e5482/konnectivity-agent/0.log" limit=1024 Jul 30 01:14:03.600539 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:14:03.600509 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-jbf4x_9ed1c3a7-6249-4633-b09a-f38dfb35647f/manager/0.log" Jul 30 01:14:03.872137 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:14:03.872111 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-ktnsg_ac97530b-0479-4196-967b-a1affa4fd298/server/0.log" Jul 30 01:14:03.960663 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:14:03.960631 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-xrx9m_2e7054c1-45e5-443c-8564-731e433d469d/manager/0.log" Jul 30 01:14:03.997226 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:14:03.997185 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5b74bc5486-frgtd_ce5296aa-53c2-4afb-ab15-02535e8eecd5/manager/0.log" Jul 30 01:14:04.034012 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:14:04.033985 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5b74bc5486-frgtd_ce5296aa-53c2-4afb-ab15-02535e8eecd5/copy-manifests/0.log" Jul 30 01:14:05.094689 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:14:05.094616 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h_d9cd41c0-eeee-4253-b07b-3e93b15a60fd/istio-proxy/0.log" Jul 30 01:14:05.260946 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:14:05.260920 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-94d649dc8-9v89r_41a9676d-a4c8-4390-8de0-a7e6eae32c57/kube-auth-proxy/0.log" Jul 30 01:14:05.298425 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:14:05.298397 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-gf4jt_70445eac-de72-40b2-ad0e-92667eb8856d/istio-proxy/0.log" Jul 30 01:14:05.323250 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:14:05.323216 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-gf4jt_70445eac-de72-40b2-ad0e-92667eb8856d/istio-proxy/1.log" Jul 30 01:14:05.394164 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:14:05.394081 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-877bf98d5-xbkfd_90afa1c1-9879-49ee-9fac-27f2e7e963f3/router/0.log" Jul 30 01:14:06.875158 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:14:06.875121 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-ddh5h_d9cd41c0-eeee-4253-b07b-3e93b15a60fd/istio-proxy/0.log" Jul 30 01:14:07.034746 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:14:07.034704 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-94d649dc8-9v89r_41a9676d-a4c8-4390-8de0-a7e6eae32c57/kube-auth-proxy/0.log" Jul 30 01:14:07.075857 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:14:07.075832 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-gf4jt_70445eac-de72-40b2-ad0e-92667eb8856d/istio-proxy/0.log" Jul 30 01:14:07.103156 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:14:07.103130 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-gf4jt_70445eac-de72-40b2-ad0e-92667eb8856d/istio-proxy/1.log" Jul 30 01:14:07.165832 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:14:07.165729 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-877bf98d5-xbkfd_90afa1c1-9879-49ee-9fac-27f2e7e963f3/router/0.log" Jul 30 01:14:10.755067 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:14:10.754970 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-t4mlm_6dd73a3d-6fdf-4ae6-982e-cfea1dffdd79/migrator/0.log" Jul 30 01:14:10.775997 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:14:10.775973 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-t4mlm_6dd73a3d-6fdf-4ae6-982e-cfea1dffdd79/graceful-termination/0.log" Jul 30 01:14:11.188309 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:14:11.188231 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-xjl58_c9caa632-edc8-43e9-ab2a-b29758f8150f/kube-storage-version-migrator-operator/1.log" Jul 30 01:14:11.188960 ip-10-0-133-10 kubenswrapper[2563]: I0730 01:14:11.188940 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-xjl58_c9caa632-edc8-43e9-ab2a-b29758f8150f/kube-storage-version-migrator-operator/0.log"