Jul 15 20:59:52.149820 ip-10-0-137-185 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 15 20:59:52.149832 ip-10-0-137-185 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 15 20:59:52.149842 ip-10-0-137-185 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 15 20:59:52.150168 ip-10-0-137-185 systemd[1]: Failed to start Kubernetes Kubelet. Jul 15 21:00:02.191326 ip-10-0-137-185 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 15 21:00:02.191342 ip-10-0-137-185 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 5c4543af24954834b6c05a682ac5da5a -- Jul 15 21:01:35.327428 ip-10-0-137-185 systemd[1]: Starting Kubernetes Kubelet... Jul 15 21:01:35.848190 ip-10-0-137-185 kubenswrapper[2579]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 21:01:35.848190 ip-10-0-137-185 kubenswrapper[2579]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 15 21:01:35.848190 ip-10-0-137-185 kubenswrapper[2579]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 21:01:35.848190 ip-10-0-137-185 kubenswrapper[2579]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 15 21:01:35.848190 ip-10-0-137-185 kubenswrapper[2579]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 21:01:35.848190 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.786999 2579 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 15 21:01:35.848190 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794563 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 21:01:35.848190 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794576 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 21:01:35.848190 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794580 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 21:01:35.848190 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794583 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 21:01:35.848190 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794586 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 21:01:35.848190 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794590 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 21:01:35.848190 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794592 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 21:01:35.848190 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794595 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 21:01:35.850233 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794598 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 21:01:35.850233 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794600 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 21:01:35.850233 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794604 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 21:01:35.850233 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794607 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 21:01:35.850233 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794609 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 21:01:35.850233 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794612 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 21:01:35.850233 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794615 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 21:01:35.850233 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794617 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 21:01:35.850233 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794620 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 21:01:35.850233 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794623 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 21:01:35.850233 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794625 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 21:01:35.850233 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794628 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 21:01:35.850233 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794636 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 21:01:35.850233 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794639 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 21:01:35.850233 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794642 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 21:01:35.850233 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794645 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 21:01:35.850233 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794647 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 21:01:35.850233 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794650 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 21:01:35.850233 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794653 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 21:01:35.852430 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794656 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 21:01:35.852430 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794658 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 21:01:35.852430 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794661 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 21:01:35.852430 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794664 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 21:01:35.852430 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794667 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 21:01:35.852430 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794670 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 21:01:35.852430 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794672 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 21:01:35.852430 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794675 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 21:01:35.852430 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794677 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 21:01:35.852430 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794680 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 21:01:35.852430 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794683 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 21:01:35.852430 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794685 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 21:01:35.852430 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794688 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 21:01:35.852430 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794690 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 21:01:35.852430 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794693 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 21:01:35.852430 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794696 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 21:01:35.852430 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794699 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 21:01:35.852430 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794701 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 21:01:35.852430 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794704 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 21:01:35.852430 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794706 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 21:01:35.854271 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794709 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 21:01:35.854271 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794713 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 21:01:35.854271 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794717 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 21:01:35.854271 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794719 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 21:01:35.854271 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794724 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 21:01:35.854271 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794727 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 21:01:35.854271 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794730 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 21:01:35.854271 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794732 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 21:01:35.854271 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794735 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 21:01:35.854271 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794738 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 21:01:35.854271 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794741 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 21:01:35.854271 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794743 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 21:01:35.854271 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794746 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 21:01:35.854271 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794749 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 21:01:35.854271 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794751 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 21:01:35.854271 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794754 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 21:01:35.854271 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794757 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 21:01:35.854271 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794759 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 21:01:35.854271 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794763 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 21:01:35.860268 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794767 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 21:01:35.860268 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794770 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 21:01:35.860268 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794773 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 21:01:35.860268 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794776 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 21:01:35.860268 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794779 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 21:01:35.860268 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794782 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 21:01:35.860268 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794785 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 21:01:35.860268 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794787 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 21:01:35.860268 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794790 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 21:01:35.860268 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794794 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 21:01:35.860268 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794798 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 21:01:35.860268 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794801 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 21:01:35.860268 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794803 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 21:01:35.860268 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794806 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 21:01:35.860268 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794808 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 21:01:35.860268 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794811 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 21:01:35.860268 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794814 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 21:01:35.860268 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794816 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 21:01:35.860268 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794819 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 21:01:35.860268 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794822 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 21:01:35.862195 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794824 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 21:01:35.862195 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794826 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 21:01:35.862195 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794829 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 21:01:35.862195 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794840 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 21:01:35.862195 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794844 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 21:01:35.862195 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794848 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 15 21:01:35.862195 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794852 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 21:01:35.862195 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794855 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 21:01:35.862195 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794858 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 21:01:35.862195 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794860 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 21:01:35.862195 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794863 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 21:01:35.862195 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794865 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 21:01:35.862195 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.794870 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 21:01:35.862195 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795609 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 21:01:35.862195 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795616 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 21:01:35.862195 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795619 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 21:01:35.862195 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795622 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 21:01:35.862195 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795625 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 21:01:35.862195 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795629 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 21:01:35.864869 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795632 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 21:01:35.864869 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795635 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 21:01:35.864869 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795638 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 21:01:35.864869 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795641 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 21:01:35.864869 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795644 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 21:01:35.864869 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795648 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 21:01:35.864869 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795650 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 21:01:35.864869 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795653 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 21:01:35.864869 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795655 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 21:01:35.864869 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795661 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 21:01:35.864869 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795665 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 21:01:35.864869 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795668 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 21:01:35.864869 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795673 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 21:01:35.864869 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795677 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 21:01:35.864869 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795680 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 21:01:35.864869 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795683 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 21:01:35.864869 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795686 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 21:01:35.864869 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795689 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 21:01:35.864869 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795692 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 21:01:35.864869 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795695 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 21:01:35.867040 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795698 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 21:01:35.867040 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795700 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 21:01:35.867040 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795703 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 21:01:35.867040 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795706 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 21:01:35.867040 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795710 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 21:01:35.867040 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795713 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 21:01:35.867040 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795717 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 21:01:35.867040 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795720 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 21:01:35.867040 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795723 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 21:01:35.867040 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795726 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 21:01:35.867040 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795729 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 21:01:35.867040 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795732 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 21:01:35.867040 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795735 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 21:01:35.867040 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795738 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 21:01:35.867040 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795741 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 21:01:35.867040 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795743 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 21:01:35.867040 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795746 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 21:01:35.867040 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795749 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 21:01:35.867040 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795752 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 21:01:35.869339 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795754 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 21:01:35.869339 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795757 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 21:01:35.869339 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795760 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 15 21:01:35.869339 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795762 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 21:01:35.869339 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795764 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 21:01:35.869339 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795770 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 21:01:35.869339 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795773 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 21:01:35.869339 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795775 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 21:01:35.869339 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795778 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 21:01:35.869339 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795780 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 21:01:35.869339 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795802 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 21:01:35.869339 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795806 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 21:01:35.869339 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795809 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 21:01:35.869339 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795812 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 21:01:35.869339 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795815 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 21:01:35.869339 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795817 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 21:01:35.869339 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795820 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 21:01:35.869339 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795822 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 21:01:35.869339 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795825 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 21:01:35.869339 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795827 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 21:01:35.883979 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795830 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 21:01:35.883979 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795833 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 21:01:35.883979 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795835 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 21:01:35.883979 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795838 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 21:01:35.883979 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795841 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 21:01:35.883979 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795843 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 21:01:35.883979 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795846 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 21:01:35.883979 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795849 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 21:01:35.883979 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795851 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 21:01:35.883979 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795854 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 21:01:35.883979 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795857 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 21:01:35.883979 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795859 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 21:01:35.883979 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795862 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 21:01:35.883979 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795864 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 21:01:35.883979 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795867 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 21:01:35.883979 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795869 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 21:01:35.883979 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795872 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 21:01:35.883979 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795876 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 21:01:35.883979 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795879 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 21:01:35.883979 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795881 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 21:01:35.875775 ip-10-0-137-185 systemd[1]: Started Kubernetes Kubelet. Jul 15 21:01:35.885691 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795886 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 21:01:35.885691 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795889 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 21:01:35.885691 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795892 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 21:01:35.885691 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795895 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 21:01:35.885691 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795897 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 21:01:35.885691 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795900 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 21:01:35.885691 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795903 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 21:01:35.885691 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795905 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 21:01:35.885691 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795908 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 21:01:35.885691 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795911 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 21:01:35.885691 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795913 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 21:01:35.885691 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795916 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 21:01:35.885691 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795918 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 21:01:35.885691 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.795921 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 21:01:35.885691 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796335 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 21:01:35.885691 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796341 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 21:01:35.885691 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796344 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 21:01:35.885691 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796347 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 21:01:35.885691 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796350 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 21:01:35.888355 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796354 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 21:01:35.888355 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796356 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 21:01:35.888355 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796359 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 15 21:01:35.888355 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796362 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 21:01:35.888355 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796365 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 21:01:35.888355 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796367 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 21:01:35.888355 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796370 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 21:01:35.888355 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796373 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 21:01:35.888355 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796375 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 21:01:35.888355 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796378 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 21:01:35.888355 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796382 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 21:01:35.888355 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796385 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 21:01:35.888355 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796388 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 21:01:35.888355 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796392 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 21:01:35.888355 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796395 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 21:01:35.888355 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796398 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 21:01:35.888355 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796402 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 21:01:35.888355 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796405 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 21:01:35.888355 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796407 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 21:01:35.888355 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796410 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 21:01:35.902590 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796413 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 21:01:35.902590 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796415 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 21:01:35.902590 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796418 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 21:01:35.902590 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796420 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 21:01:35.902590 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796423 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 21:01:35.902590 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796426 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 21:01:35.902590 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796428 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 21:01:35.902590 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796430 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 21:01:35.902590 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796433 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 21:01:35.902590 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796435 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 21:01:35.902590 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796438 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 21:01:35.902590 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796440 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 21:01:35.902590 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796443 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 21:01:35.902590 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796445 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 21:01:35.902590 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796448 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 21:01:35.902590 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796451 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 21:01:35.902590 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796454 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 21:01:35.902590 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796457 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 21:01:35.902590 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796459 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 21:01:35.902590 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796461 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 21:01:35.915989 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796464 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 21:01:35.915989 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796466 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 21:01:35.915989 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796472 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 21:01:35.915989 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796475 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 21:01:35.915989 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796477 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 21:01:35.915989 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796481 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 21:01:35.915989 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796484 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 21:01:35.915989 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796487 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 21:01:35.915989 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796489 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 21:01:35.915989 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796492 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 21:01:35.915989 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796495 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 21:01:35.915989 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796497 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 21:01:35.915989 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796500 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 21:01:35.915989 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796502 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 21:01:35.915989 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796505 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 21:01:35.915989 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796508 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 21:01:35.915989 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796511 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 21:01:35.915989 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796513 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 21:01:35.915989 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796516 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 21:01:35.915989 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796518 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 21:01:35.918704 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796522 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 21:01:35.918704 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796524 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 21:01:35.918704 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796527 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 21:01:35.918704 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796529 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 21:01:35.918704 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796532 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 21:01:35.918704 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796534 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 21:01:35.918704 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796537 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 21:01:35.918704 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796539 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 21:01:35.918704 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796542 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 21:01:35.918704 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796545 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 21:01:35.918704 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796547 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 21:01:35.918704 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796550 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 21:01:35.918704 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796552 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 21:01:35.918704 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796556 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 21:01:35.918704 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796560 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 21:01:35.918704 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796562 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 21:01:35.918704 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796565 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 21:01:35.918704 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796568 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 21:01:35.918704 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796571 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 21:01:35.935262 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796574 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 21:01:35.935262 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796576 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 21:01:35.935262 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796579 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 21:01:35.935262 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796581 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 21:01:35.935262 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796584 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 21:01:35.935262 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796588 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 21:01:35.935262 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796591 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 21:01:35.935262 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796594 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 21:01:35.935262 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796597 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 21:01:35.935262 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796599 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 21:01:35.935262 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796602 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 21:01:35.935262 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796605 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 21:01:35.935262 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796607 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 21:01:35.935262 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796610 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 21:01:35.935262 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.796613 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 21:01:35.935262 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796692 2579 flags.go:64] FLAG: --address="0.0.0.0" Jul 15 21:01:35.935262 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796699 2579 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 15 21:01:35.935262 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796711 2579 flags.go:64] FLAG: --anonymous-auth="true" Jul 15 21:01:35.935262 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796716 2579 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 15 21:01:35.935262 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796720 2579 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 15 21:01:36.103511 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796723 2579 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 15 21:01:36.103511 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796727 2579 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 15 21:01:36.103511 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796736 2579 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 15 21:01:36.103511 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796739 2579 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 15 21:01:36.103511 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796743 2579 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 15 21:01:36.103511 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796746 2579 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 15 21:01:36.103511 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796749 2579 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 15 21:01:36.103511 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796754 2579 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 15 21:01:36.103511 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796757 2579 flags.go:64] FLAG: --cgroup-root="" Jul 15 21:01:36.103511 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796760 2579 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 15 21:01:36.103511 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796764 2579 flags.go:64] FLAG: --client-ca-file="" Jul 15 21:01:36.103511 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796767 2579 flags.go:64] FLAG: --cloud-provider="external" Jul 15 21:01:36.103511 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796770 2579 flags.go:64] FLAG: --cluster-dns="[]" Jul 15 21:01:36.103511 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796776 2579 flags.go:64] FLAG: --cluster-domain="" Jul 15 21:01:36.103511 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796779 2579 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 15 21:01:36.103511 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796782 2579 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 15 21:01:36.103511 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796785 2579 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 15 21:01:36.103511 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796788 2579 flags.go:64] FLAG: --container-log-max-files="5" Jul 15 21:01:36.103511 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796792 2579 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 15 21:01:36.103511 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796795 2579 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 15 21:01:36.103511 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796798 2579 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 15 21:01:36.103511 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796801 2579 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 15 21:01:36.103511 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796804 2579 flags.go:64] FLAG: --contention-profiling="false" Jul 15 21:01:36.103511 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796807 2579 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 15 21:01:36.105530 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796810 2579 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 15 21:01:36.105530 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796813 2579 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 15 21:01:36.105530 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796816 2579 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 15 21:01:36.105530 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796820 2579 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 15 21:01:36.105530 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796823 2579 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 15 21:01:36.105530 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796826 2579 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 15 21:01:36.105530 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796828 2579 flags.go:64] FLAG: --enable-load-reader="false" Jul 15 21:01:36.105530 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796831 2579 flags.go:64] FLAG: --enable-server="true" Jul 15 21:01:36.105530 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796834 2579 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 15 21:01:36.105530 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796839 2579 flags.go:64] FLAG: --event-burst="100" Jul 15 21:01:36.105530 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796842 2579 flags.go:64] FLAG: --event-qps="50" Jul 15 21:01:36.105530 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796846 2579 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 15 21:01:36.105530 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796849 2579 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 15 21:01:36.105530 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796852 2579 flags.go:64] FLAG: --eviction-hard="" Jul 15 21:01:36.105530 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796856 2579 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 15 21:01:36.105530 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796860 2579 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 15 21:01:36.105530 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796863 2579 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 15 21:01:36.105530 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796866 2579 flags.go:64] FLAG: --eviction-soft="" Jul 15 21:01:36.105530 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796870 2579 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 15 21:01:36.105530 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796873 2579 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 15 21:01:36.105530 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796876 2579 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 15 21:01:36.105530 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796879 2579 flags.go:64] FLAG: --experimental-mounter-path="" Jul 15 21:01:36.105530 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796881 2579 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 15 21:01:36.105530 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796884 2579 flags.go:64] FLAG: --fail-swap-on="true" Jul 15 21:01:36.105530 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796887 2579 flags.go:64] FLAG: --feature-gates="" Jul 15 21:01:36.108177 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796891 2579 flags.go:64] FLAG: --file-check-frequency="20s" Jul 15 21:01:36.108177 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796894 2579 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 15 21:01:36.108177 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796897 2579 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 15 21:01:36.108177 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796900 2579 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 15 21:01:36.108177 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796903 2579 flags.go:64] FLAG: --healthz-port="10248" Jul 15 21:01:36.108177 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796906 2579 flags.go:64] FLAG: --help="false" Jul 15 21:01:36.108177 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796909 2579 flags.go:64] FLAG: --hostname-override="ip-10-0-137-185.ec2.internal" Jul 15 21:01:36.108177 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796912 2579 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 15 21:01:36.108177 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796915 2579 flags.go:64] FLAG: --http-check-frequency="20s" Jul 15 21:01:36.108177 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796918 2579 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 15 21:01:36.108177 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796921 2579 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 15 21:01:36.108177 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796924 2579 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 15 21:01:36.108177 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796927 2579 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 15 21:01:36.108177 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796930 2579 flags.go:64] FLAG: --image-service-endpoint="" Jul 15 21:01:36.108177 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796933 2579 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 15 21:01:36.108177 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796936 2579 flags.go:64] FLAG: --kube-api-burst="100" Jul 15 21:01:36.108177 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796938 2579 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 15 21:01:36.108177 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796941 2579 flags.go:64] FLAG: --kube-api-qps="50" Jul 15 21:01:36.108177 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796944 2579 flags.go:64] FLAG: --kube-reserved="" Jul 15 21:01:36.108177 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796947 2579 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 15 21:01:36.108177 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796950 2579 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 15 21:01:36.108177 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796953 2579 flags.go:64] FLAG: --kubelet-cgroups="" Jul 15 21:01:36.108177 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796957 2579 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 15 21:01:36.108177 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796960 2579 flags.go:64] FLAG: --lock-file="" Jul 15 21:01:36.110343 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796962 2579 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 15 21:01:36.110343 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796966 2579 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 15 21:01:36.110343 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796969 2579 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 15 21:01:36.110343 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796974 2579 flags.go:64] FLAG: --log-json-split-stream="false" Jul 15 21:01:36.110343 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796977 2579 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 15 21:01:36.110343 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796980 2579 flags.go:64] FLAG: --log-text-split-stream="false" Jul 15 21:01:36.110343 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796983 2579 flags.go:64] FLAG: --logging-format="text" Jul 15 21:01:36.110343 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796986 2579 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 15 21:01:36.110343 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796989 2579 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 15 21:01:36.110343 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796991 2579 flags.go:64] FLAG: --manifest-url="" Jul 15 21:01:36.110343 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796994 2579 flags.go:64] FLAG: --manifest-url-header="" Jul 15 21:01:36.110343 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.796999 2579 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 15 21:01:36.110343 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797002 2579 flags.go:64] FLAG: --max-open-files="1000000" Jul 15 21:01:36.110343 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797005 2579 flags.go:64] FLAG: --max-pods="110" Jul 15 21:01:36.110343 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797008 2579 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 15 21:01:36.110343 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797012 2579 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 15 21:01:36.110343 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797015 2579 flags.go:64] FLAG: --memory-manager-policy="None" Jul 15 21:01:36.110343 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797018 2579 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 15 21:01:36.110343 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797021 2579 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 15 21:01:36.110343 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797024 2579 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 15 21:01:36.110343 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797027 2579 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 15 21:01:36.110343 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797034 2579 flags.go:64] FLAG: --node-status-max-images="50" Jul 15 21:01:36.110343 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797037 2579 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 15 21:01:36.110343 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797040 2579 flags.go:64] FLAG: --oom-score-adj="-999" Jul 15 21:01:36.110343 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797042 2579 flags.go:64] FLAG: --pod-cidr="" Jul 15 21:01:36.111166 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797045 2579 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 15 21:01:36.111166 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797050 2579 flags.go:64] FLAG: --pod-manifest-path="" Jul 15 21:01:36.111166 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797053 2579 flags.go:64] FLAG: --pod-max-pids="-1" Jul 15 21:01:36.111166 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797056 2579 flags.go:64] FLAG: --pods-per-core="0" Jul 15 21:01:36.111166 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797076 2579 flags.go:64] FLAG: --port="10250" Jul 15 21:01:36.111166 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797082 2579 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 15 21:01:36.111166 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797087 2579 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-090184651e3792734" Jul 15 21:01:36.111166 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797090 2579 flags.go:64] FLAG: --qos-reserved="" Jul 15 21:01:36.111166 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797094 2579 flags.go:64] FLAG: --read-only-port="10255" Jul 15 21:01:36.111166 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797097 2579 flags.go:64] FLAG: --register-node="true" Jul 15 21:01:36.111166 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797100 2579 flags.go:64] FLAG: --register-with-taints="" Jul 15 21:01:36.111166 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797104 2579 flags.go:64] FLAG: --registry-burst="10" Jul 15 21:01:36.111166 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797107 2579 flags.go:64] FLAG: --registry-qps="5" Jul 15 21:01:36.111166 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797110 2579 flags.go:64] FLAG: --reserved-cpus="" Jul 15 21:01:36.111166 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797112 2579 flags.go:64] FLAG: --reserved-memory="" Jul 15 21:01:36.111166 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797116 2579 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 15 21:01:36.111166 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797119 2579 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 15 21:01:36.111166 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797122 2579 flags.go:64] FLAG: --rotate-certificates="false" Jul 15 21:01:36.111166 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797124 2579 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 15 21:01:36.111166 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797127 2579 flags.go:64] FLAG: --runonce="false" Jul 15 21:01:36.111166 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797130 2579 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 15 21:01:36.111166 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797133 2579 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 15 21:01:36.111166 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797136 2579 flags.go:64] FLAG: --seccomp-default="false" Jul 15 21:01:36.111166 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797139 2579 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 15 21:01:36.111166 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797142 2579 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 15 21:01:36.112686 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797145 2579 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 15 21:01:36.112686 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797148 2579 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 15 21:01:36.112686 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797151 2579 flags.go:64] FLAG: --storage-driver-password="root" Jul 15 21:01:36.112686 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797153 2579 flags.go:64] FLAG: --storage-driver-secure="false" Jul 15 21:01:36.112686 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797156 2579 flags.go:64] FLAG: --storage-driver-table="stats" Jul 15 21:01:36.112686 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797159 2579 flags.go:64] FLAG: --storage-driver-user="root" Jul 15 21:01:36.112686 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797162 2579 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 15 21:01:36.112686 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797165 2579 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 15 21:01:36.112686 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797168 2579 flags.go:64] FLAG: --system-cgroups="" Jul 15 21:01:36.112686 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797171 2579 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 15 21:01:36.112686 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797176 2579 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 15 21:01:36.112686 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797179 2579 flags.go:64] FLAG: --tls-cert-file="" Jul 15 21:01:36.112686 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797182 2579 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 15 21:01:36.112686 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797188 2579 flags.go:64] FLAG: --tls-min-version="" Jul 15 21:01:36.112686 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797190 2579 flags.go:64] FLAG: --tls-private-key-file="" Jul 15 21:01:36.112686 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797195 2579 flags.go:64] FLAG: --topology-manager-policy="none" Jul 15 21:01:36.112686 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797198 2579 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 15 21:01:36.112686 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797201 2579 flags.go:64] FLAG: --topology-manager-scope="container" Jul 15 21:01:36.112686 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797205 2579 flags.go:64] FLAG: --v="2" Jul 15 21:01:36.112686 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797209 2579 flags.go:64] FLAG: --version="false" Jul 15 21:01:36.112686 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797213 2579 flags.go:64] FLAG: --vmodule="" Jul 15 21:01:36.112686 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797217 2579 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 15 21:01:36.112686 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.797220 2579 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 15 21:01:36.112686 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797339 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 21:01:36.113444 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797344 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 21:01:36.113444 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797348 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 21:01:36.113444 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797353 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 21:01:36.113444 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797357 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 21:01:36.113444 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797360 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 21:01:36.113444 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797363 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 21:01:36.113444 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797367 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 21:01:36.113444 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797371 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 21:01:36.113444 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797373 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 21:01:36.113444 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797376 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 21:01:36.113444 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797379 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 21:01:36.113444 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797382 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 21:01:36.113444 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797385 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 21:01:36.113444 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797388 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 21:01:36.113444 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797390 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 21:01:36.113444 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797393 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 21:01:36.113444 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797396 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 21:01:36.113444 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797398 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 21:01:36.113984 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797401 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 21:01:36.113984 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797403 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 21:01:36.113984 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797406 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 21:01:36.113984 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797413 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 21:01:36.113984 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797416 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 21:01:36.113984 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797420 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 21:01:36.113984 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797422 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 21:01:36.113984 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797425 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 21:01:36.113984 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797428 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 21:01:36.113984 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797431 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 21:01:36.113984 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797433 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 15 21:01:36.113984 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797436 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 21:01:36.113984 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797439 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 21:01:36.113984 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797441 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 21:01:36.113984 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797443 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 21:01:36.113984 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797446 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 21:01:36.113984 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797449 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 21:01:36.113984 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797451 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 21:01:36.113984 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797454 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 21:01:36.113984 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797456 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 21:01:36.254986 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797459 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 21:01:36.254986 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797461 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 21:01:36.254986 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797464 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 21:01:36.254986 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797466 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 21:01:36.254986 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797469 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 21:01:36.254986 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797471 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 21:01:36.254986 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797474 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 21:01:36.254986 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797476 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 21:01:36.254986 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797479 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 21:01:36.254986 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797482 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 21:01:36.254986 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797484 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 21:01:36.254986 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797487 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 21:01:36.254986 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797489 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 21:01:36.254986 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797492 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 21:01:36.254986 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797494 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 21:01:36.254986 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797499 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 21:01:36.254986 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797502 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 21:01:36.254986 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797506 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 21:01:36.254986 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797508 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 21:01:36.254986 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797511 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 21:01:36.254986 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797514 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 21:01:36.255942 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797516 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 21:01:36.255942 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797519 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 21:01:36.255942 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797521 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 21:01:36.255942 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797524 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 21:01:36.255942 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797526 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 21:01:36.255942 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797529 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 21:01:36.255942 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797531 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 21:01:36.255942 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797534 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 21:01:36.255942 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797536 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 21:01:36.255942 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797539 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 21:01:36.255942 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797541 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 21:01:36.255942 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797544 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 21:01:36.255942 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797546 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 21:01:36.255942 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797549 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 21:01:36.255942 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797551 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 21:01:36.255942 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797554 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 21:01:36.255942 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797557 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 21:01:36.255942 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797559 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 21:01:36.255942 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797562 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 21:01:36.255942 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797565 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 21:01:36.256566 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797567 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 21:01:36.256566 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797570 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 21:01:36.256566 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797572 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 21:01:36.256566 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797574 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 21:01:36.256566 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797577 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 21:01:36.256566 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797580 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 21:01:36.256566 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797584 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 21:01:36.256566 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797586 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 21:01:36.256566 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797593 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 21:01:36.256566 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797596 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 21:01:36.256566 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797599 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 21:01:36.256566 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797601 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 21:01:36.256566 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797604 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 21:01:36.256566 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797606 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 21:01:36.256566 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797609 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 21:01:36.256566 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797611 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 21:01:36.256566 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797614 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 21:01:36.256566 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797616 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 21:01:36.256566 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.797619 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 21:01:36.257399 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.798283 2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 21:01:36.257399 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.805438 2579 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 15 21:01:36.257399 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.805455 2579 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 15 21:01:36.257399 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.805476 2579 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 15 21:01:36.257399 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.805486 2579 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 15 21:01:36.257399 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805551 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 21:01:36.257399 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805556 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 21:01:36.257399 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805560 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 21:01:36.257399 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805563 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 21:01:36.257399 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805566 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 21:01:36.257399 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805569 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 21:01:36.257399 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805572 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 21:01:36.257399 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805575 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 15 21:01:36.257399 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805578 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 21:01:36.257399 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805581 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 21:01:36.258026 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805584 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 21:01:36.258026 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805586 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 21:01:36.258026 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805589 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 21:01:36.258026 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805591 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 21:01:36.258026 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805594 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 21:01:36.258026 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805597 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 21:01:36.258026 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805599 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 21:01:36.258026 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805602 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 21:01:36.258026 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805606 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 21:01:36.258026 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805611 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 21:01:36.258026 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805614 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 21:01:36.258026 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805617 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 21:01:36.258026 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805619 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 21:01:36.258026 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805622 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 21:01:36.258026 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805625 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 21:01:36.258026 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805628 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 21:01:36.258026 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805633 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 21:01:36.258026 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805636 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 21:01:36.258026 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805638 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 21:01:36.258643 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805642 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 21:01:36.258643 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805644 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 21:01:36.258643 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805647 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 21:01:36.258643 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805650 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 21:01:36.258643 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805652 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 21:01:36.258643 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805655 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 21:01:36.258643 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805657 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 21:01:36.258643 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805660 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 21:01:36.258643 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805662 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 21:01:36.258643 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805665 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 21:01:36.258643 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805668 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 21:01:36.258643 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805670 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 21:01:36.258643 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805674 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 21:01:36.258643 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805678 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 21:01:36.258643 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805681 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 21:01:36.258643 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805684 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 21:01:36.258643 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805687 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 21:01:36.258643 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805690 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 21:01:36.258643 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805693 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 21:01:36.259229 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805695 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 21:01:36.259229 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805698 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 21:01:36.259229 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805701 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 21:01:36.259229 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805704 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 21:01:36.259229 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805706 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 21:01:36.259229 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805709 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 21:01:36.259229 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805711 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 21:01:36.259229 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805714 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 21:01:36.259229 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805718 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 21:01:36.259229 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805721 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 21:01:36.259229 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805726 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 21:01:36.259229 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805729 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 21:01:36.259229 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805732 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 21:01:36.259229 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805735 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 21:01:36.259229 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805739 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 21:01:36.259229 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805742 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 21:01:36.259229 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805744 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 21:01:36.259229 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805747 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 21:01:36.259229 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805750 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 21:01:36.259750 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805753 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 21:01:36.259750 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805755 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 21:01:36.259750 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805758 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 21:01:36.259750 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805760 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 21:01:36.259750 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805763 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 21:01:36.259750 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805765 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 21:01:36.259750 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805768 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 21:01:36.259750 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805770 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 21:01:36.259750 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805773 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 21:01:36.259750 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805775 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 21:01:36.259750 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805778 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 21:01:36.259750 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805780 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 21:01:36.259750 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805783 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 21:01:36.259750 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805785 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 21:01:36.259750 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805788 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 21:01:36.259750 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805791 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 21:01:36.259750 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805793 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 21:01:36.259750 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805796 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 21:01:36.259750 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805798 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 21:01:36.259750 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805801 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 21:01:36.260339 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805803 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 21:01:36.260339 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805806 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 21:01:36.260339 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805809 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 21:01:36.260339 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805812 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 21:01:36.260339 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805815 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 21:01:36.260339 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805818 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 21:01:36.260339 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805821 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 21:01:36.260339 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805824 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 21:01:36.260339 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805827 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 21:01:36.260339 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805830 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 21:01:36.260339 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805832 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 21:01:36.260339 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805835 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 21:01:36.260339 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.805840 2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 21:01:36.260339 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805953 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 21:01:36.260339 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805959 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 21:01:36.260753 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805962 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 21:01:36.260753 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805964 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 21:01:36.260753 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805967 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 21:01:36.260753 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805969 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 21:01:36.260753 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805972 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 21:01:36.260753 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805974 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 21:01:36.260753 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805977 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 21:01:36.260753 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805979 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 21:01:36.260753 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805982 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 21:01:36.260753 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805984 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 21:01:36.260753 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805987 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 21:01:36.260753 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805991 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 21:01:36.260753 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805994 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 21:01:36.260753 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805996 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 21:01:36.260753 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.805999 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 21:01:36.260753 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806001 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 21:01:36.260753 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806004 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 21:01:36.260753 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806006 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 21:01:36.260753 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806010 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 21:01:36.261354 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806014 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 21:01:36.261354 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806017 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 21:01:36.261354 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806022 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 21:01:36.261354 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806024 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 21:01:36.261354 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806027 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 21:01:36.261354 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806029 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 21:01:36.261354 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806032 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 21:01:36.261354 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806035 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 21:01:36.261354 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806038 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 21:01:36.261354 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806041 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 21:01:36.261354 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806043 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 21:01:36.261354 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806046 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 21:01:36.261354 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806048 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 21:01:36.261354 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806051 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 21:01:36.261354 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806053 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 21:01:36.261354 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806056 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 21:01:36.261354 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806058 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 21:01:36.261354 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806077 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 21:01:36.261354 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806080 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 21:01:36.261354 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806082 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 21:01:36.261899 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806085 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 21:01:36.261899 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806087 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 21:01:36.261899 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806090 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 21:01:36.261899 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806093 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 21:01:36.261899 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806098 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 21:01:36.261899 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806101 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 15 21:01:36.261899 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806103 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 21:01:36.261899 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806106 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 21:01:36.261899 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806108 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 21:01:36.261899 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806111 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 21:01:36.261899 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806113 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 21:01:36.261899 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806116 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 21:01:36.261899 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806119 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 21:01:36.261899 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806121 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 21:01:36.261899 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806125 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 21:01:36.261899 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806128 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 21:01:36.261899 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806130 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 21:01:36.261899 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806133 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 21:01:36.261899 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806136 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 21:01:36.261899 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806139 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 21:01:36.262473 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806143 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 21:01:36.262473 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806146 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 21:01:36.262473 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806148 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 21:01:36.262473 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806151 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 21:01:36.262473 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806154 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 21:01:36.262473 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806156 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 21:01:36.262473 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806159 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 21:01:36.262473 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806161 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 21:01:36.262473 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806164 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 21:01:36.262473 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806167 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 21:01:36.262473 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806169 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 21:01:36.262473 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806172 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 21:01:36.262473 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806174 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 21:01:36.262473 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806177 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 21:01:36.262473 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806179 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 21:01:36.262473 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806182 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 21:01:36.262473 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806185 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 21:01:36.262473 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806188 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 21:01:36.262473 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806191 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 21:01:36.262473 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806193 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 21:01:36.263002 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806196 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 21:01:36.263002 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806198 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 21:01:36.263002 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806202 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 21:01:36.263002 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806205 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 21:01:36.263002 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806208 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 21:01:36.263002 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806211 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 21:01:36.263002 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806216 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 21:01:36.263002 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806219 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 21:01:36.263002 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806221 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 21:01:36.263002 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806224 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 21:01:36.263002 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806227 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 21:01:36.263002 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806230 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 21:01:36.263002 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806233 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 21:01:36.263002 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806236 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 21:01:36.263002 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806238 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 21:01:36.263002 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806241 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 21:01:36.263002 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806243 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 21:01:36.263002 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:35.806246 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 21:01:36.263600 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.806251 2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 21:01:36.263600 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.806363 2579 server.go:962] "Client rotation is on, will bootstrap in background" Jul 15 21:01:36.263600 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.809344 2579 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 15 21:01:36.263600 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.812121 2579 server.go:1019] "Starting client certificate rotation" Jul 15 21:01:36.263600 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.812348 2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 15 21:01:36.263600 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.812387 2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 15 21:01:36.263600 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.815336 2579 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 15 21:01:36.263600 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.820266 2579 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 15 21:01:36.263600 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.833602 2579 log.go:25] "Validated CRI v1 runtime API" Jul 15 21:01:36.263600 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.836929 2579 log.go:25] "Validated CRI v1 image API" Jul 15 21:01:36.263600 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.838462 2579 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 15 21:01:36.263600 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.841166 2579 fs.go:135] Filesystem UUIDs: map[6a714920-b71e-4eb2-90f7-5521a2656712:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 908b08f4-d4fc-4037-aea1-657418ee2065:/dev/nvme0n1p4] Jul 15 21:01:36.263954 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.841186 2579 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 15 21:01:36.264058 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.847084 2579 manager.go:216] Machine: {Timestamp:2026-07-15 21:01:35.845733497 +0000 UTC m=+0.387046049 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099137 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d5eac29ffb260ee563b9700526062 SystemUUID:ec2d5eac-29ff-b260-ee56-3b9700526062 BootID:5c4543af-2495-4834-b6c0-5a682ac5da5a Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:8c:9d:38:39:65 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:8c:9d:38:39:65 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:92:23:8d:2c:70:3d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 15 21:01:36.264058 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.847188 2579 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 15 21:01:36.264058 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.847273 2579 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 15 21:01:36.264058 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.851226 2579 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 15 21:01:36.264058 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.851258 2579 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-185.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 15 21:01:36.264058 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.852294 2579 topology_manager.go:138] "Creating topology manager with none policy" Jul 15 21:01:36.264058 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.852305 2579 container_manager_linux.go:309] "Creating device plugin manager" Jul 15 21:01:36.264058 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.852313 2579 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 15 21:01:36.264793 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.852846 2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 15 21:01:36.264793 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.853269 2579 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 15 21:01:36.264793 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.855198 2579 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 15 21:01:36.264793 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.858788 2579 state_mem.go:36] "Initialized new in-memory state store" Jul 15 21:01:36.264793 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.858915 2579 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 15 21:01:36.264793 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.858942 2579 kubelet.go:486] "Attempting to sync node with API server" Jul 15 21:01:36.264793 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.858952 2579 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 15 21:01:36.264793 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.858966 2579 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 15 21:01:36.264793 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.858976 2579 kubelet.go:398] "Adding apiserver pod source" Jul 15 21:01:36.264793 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.858985 2579 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 15 21:01:36.264793 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.859342 2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 15 21:01:36.264793 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.859357 2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 15 21:01:36.264793 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.862473 2579 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 15 21:01:36.264793 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.870170 2579 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 15 21:01:36.264793 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.870202 2579 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 15 21:01:36.264793 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.870359 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 15 21:01:36.264793 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.870371 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 15 21:01:36.264793 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.870377 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 15 21:01:36.264793 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.870384 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 15 21:01:36.264793 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.870392 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 15 21:01:36.264793 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.870398 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 15 21:01:36.264793 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.871027 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 15 21:01:36.264793 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.871034 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 15 21:01:36.264793 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.871040 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 15 21:01:36.264793 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.871046 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 15 21:01:36.264793 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.871055 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 15 21:01:36.264793 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.871078 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 15 21:01:36.264793 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.871886 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 15 21:01:36.264793 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.871893 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 15 21:01:36.264793 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:35.872298 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-185.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 15 21:01:36.264793 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:35.872323 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 15 21:01:36.264793 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.874921 2579 server.go:1268] "Started kubelet" Jul 15 21:01:36.264793 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.874986 2579 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 15 21:01:36.264793 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.874985 2579 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 15 21:01:36.264793 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.875048 2579 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 15 21:01:36.264793 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.876394 2579 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 15 21:01:36.264793 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.877146 2579 server.go:310] "Adding debug handlers to kubelet server" Jul 15 21:01:36.264793 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:35.880446 2579 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 15 21:01:36.264793 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.883805 2579 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 15 21:01:36.264793 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.883821 2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 15 21:01:36.265919 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:35.883899 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-185.ec2.internal\" not found" Jul 15 21:01:36.265919 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.884496 2579 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 15 21:01:36.265919 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.884596 2579 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-185.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 15 21:01:36.265919 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.884688 2579 factory.go:55] Registering systemd factory Jul 15 21:01:36.265919 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.884704 2579 factory.go:223] Registration of the systemd container factory successfully Jul 15 21:01:36.265919 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.884807 2579 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 15 21:01:36.265919 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.884821 2579 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 15 21:01:36.265919 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.884965 2579 reconstruct.go:97] "Volume reconstruction finished" Jul 15 21:01:36.265919 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.884985 2579 reconciler.go:29] "Reconciler: start to sync state" Jul 15 21:01:36.265919 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:35.884646 2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-185.ec2.internal.18c29254eee2f024 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-185.ec2.internal,UID:ip-10-0-137-185.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-185.ec2.internal,},FirstTimestamp:2026-07-15 21:01:35.874895908 +0000 UTC m=+0.416208449,LastTimestamp:2026-07-15 21:01:35.874895908 +0000 UTC m=+0.416208449,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-185.ec2.internal,}" Jul 15 21:01:36.265919 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.885984 2579 factory.go:153] Registering CRI-O factory Jul 15 21:01:36.265919 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.885998 2579 factory.go:223] Registration of the crio container factory successfully Jul 15 21:01:36.265919 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.886049 2579 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 15 21:01:36.265919 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.886093 2579 factory.go:103] Registering Raw factory Jul 15 21:01:36.265919 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.886108 2579 manager.go:1196] Started watching for new ooms in manager Jul 15 21:01:36.265919 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.886598 2579 manager.go:349] Starting recovery of all containers Jul 15 21:01:36.265919 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:35.888601 2579 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-137-185.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 15 21:01:36.265919 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:35.888691 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 15 21:01:36.265919 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.897832 2579 manager.go:354] Recovery completed Jul 15 21:01:36.265919 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.904196 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 21:01:36.265919 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.906361 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasSufficientMemory" Jul 15 21:01:36.265919 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.906393 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 21:01:36.265919 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.906404 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasSufficientPID" Jul 15 21:01:36.265919 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.906934 2579 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 15 21:01:36.265919 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.906943 2579 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 15 21:01:36.265919 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.906957 2579 state_mem.go:36] "Initialized new in-memory state store" Jul 15 21:01:36.265919 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.906964 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 21:01:36.265919 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.907700 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasSufficientMemory" Jul 15 21:01:36.265919 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.907741 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 21:01:36.265919 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.907755 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasSufficientPID" Jul 15 21:01:36.265919 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.908238 2579 policy_none.go:49] "None policy: Start" Jul 15 21:01:36.265919 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.908251 2579 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 15 21:01:36.373934 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.908262 2579 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 15 21:01:36.373934 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:35.908960 2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-185.ec2.internal.18c29254f0c35a4c default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-185.ec2.internal,UID:ip-10-0-137-185.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-137-185.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-137-185.ec2.internal,},FirstTimestamp:2026-07-15 21:01:35.906380364 +0000 UTC m=+0.447692904,LastTimestamp:2026-07-15 21:01:35.906380364 +0000 UTC m=+0.447692904,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-185.ec2.internal,}" Jul 15 21:01:36.373934 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.909290 2579 policy_none.go:47] "Start" Jul 15 21:01:36.373934 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:35.920292 2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-185.ec2.internal.18c29254f0c3a079 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-185.ec2.internal,UID:ip-10-0-137-185.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-137-185.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-137-185.ec2.internal,},FirstTimestamp:2026-07-15 21:01:35.906398329 +0000 UTC m=+0.447710870,LastTimestamp:2026-07-15 21:01:35.906398329 +0000 UTC m=+0.447710870,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-185.ec2.internal,}" Jul 15 21:01:36.373934 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:35.931021 2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-185.ec2.internal.18c29254f0c3c56e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-185.ec2.internal,UID:ip-10-0-137-185.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-137-185.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-137-185.ec2.internal,},FirstTimestamp:2026-07-15 21:01:35.90640779 +0000 UTC m=+0.447720332,LastTimestamp:2026-07-15 21:01:35.90640779 +0000 UTC m=+0.447720332,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-185.ec2.internal,}" Jul 15 21:01:36.373934 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.933878 2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mljz6" Jul 15 21:01:36.373934 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.941571 2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mljz6" Jul 15 21:01:36.373934 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:35.944838 2579 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-137-185.ec2.internal.18c29254f0c35a4c\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-185.ec2.internal.18c29254f0c35a4c default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-185.ec2.internal,UID:ip-10-0-137-185.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-137-185.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-137-185.ec2.internal,},FirstTimestamp:2026-07-15 21:01:35.906380364 +0000 UTC m=+0.447692904,LastTimestamp:2026-07-15 21:01:35.907710358 +0000 UTC m=+0.449022900,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-185.ec2.internal,}" Jul 15 21:01:36.373934 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.973351 2579 manager.go:337] "Starting Device Plugin manager" Jul 15 21:01:36.373934 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:35.973567 2579 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 15 21:01:36.373934 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.973582 2579 server.go:87] "Starting device plugin registration server" Jul 15 21:01:36.373934 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.973871 2579 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 15 21:01:36.374421 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.973884 2579 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 15 21:01:36.374421 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.973979 2579 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 15 21:01:36.374421 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.974124 2579 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 15 21:01:36.374421 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.974133 2579 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 15 21:01:36.374421 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:35.974582 2579 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 15 21:01:36.374421 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:35.974636 2579 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-185.ec2.internal\" not found" Jul 15 21:01:36.374421 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.989264 2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 15 21:01:36.374421 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.990556 2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 15 21:01:36.374421 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.990599 2579 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 15 21:01:36.374421 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.990621 2579 kubelet.go:2448] "Starting kubelet main sync loop" Jul 15 21:01:36.374421 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:35.990656 2579 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 15 21:01:36.374421 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:35.992287 2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 21:01:36.374421 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:36.074343 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 21:01:36.374421 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:36.075364 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasSufficientMemory" Jul 15 21:01:36.374421 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:36.075394 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 21:01:36.374421 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:36.075408 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasSufficientPID" Jul 15 21:01:36.374421 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:36.075437 2579 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-185.ec2.internal" Jul 15 21:01:36.374421 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:36.083999 2579 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-185.ec2.internal" Jul 15 21:01:36.374421 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:36.084018 2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-185.ec2.internal\": node \"ip-10-0-137-185.ec2.internal\" not found" Jul 15 21:01:36.374421 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:36.091215 2579 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-185.ec2.internal"] Jul 15 21:01:36.374421 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:36.091272 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 21:01:36.374421 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:36.092009 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasSufficientMemory" Jul 15 21:01:36.374421 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:36.092034 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 21:01:36.374421 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:36.092056 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasSufficientPID" Jul 15 21:01:36.374421 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:36.093327 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 21:01:36.374421 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:36.093435 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal" Jul 15 21:01:36.374421 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:36.093460 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 21:01:36.374421 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:36.094076 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasSufficientMemory" Jul 15 21:01:36.374421 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:36.094089 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasSufficientMemory" Jul 15 21:01:36.374421 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:36.094102 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 21:01:36.374421 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:36.094115 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasSufficientPID" Jul 15 21:01:36.374421 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:36.094114 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 21:01:36.374421 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:36.094186 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasSufficientPID" Jul 15 21:01:36.374421 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:36.095386 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-185.ec2.internal" Jul 15 21:01:36.374421 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:36.095414 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 21:01:36.375383 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:36.096128 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasSufficientMemory" Jul 15 21:01:36.375383 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:36.096153 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 21:01:36.375383 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:36.096163 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasSufficientPID" Jul 15 21:01:36.375383 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:36.142753 2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-185.ec2.internal\" not found" node="ip-10-0-137-185.ec2.internal" Jul 15 21:01:36.375383 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:36.147267 2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-185.ec2.internal\" not found" node="ip-10-0-137-185.ec2.internal" Jul 15 21:01:36.375383 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:36.151168 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-185.ec2.internal\" not found" Jul 15 21:01:36.375383 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:36.187057 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b338db47c4937c088ae739177bb426e7-config\") pod \"kube-apiserver-proxy-ip-10-0-137-185.ec2.internal\" (UID: \"b338db47c4937c088ae739177bb426e7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-185.ec2.internal" Jul 15 21:01:36.375383 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:36.187097 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/29da48fefeff7a1251df734706134371-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal\" (UID: \"29da48fefeff7a1251df734706134371\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal" Jul 15 21:01:36.375383 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:36.187137 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/29da48fefeff7a1251df734706134371-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal\" (UID: \"29da48fefeff7a1251df734706134371\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal" Jul 15 21:01:36.375383 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:36.251616 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-185.ec2.internal\" not found" Jul 15 21:01:36.375383 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:36.287970 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b338db47c4937c088ae739177bb426e7-config\") pod \"kube-apiserver-proxy-ip-10-0-137-185.ec2.internal\" (UID: \"b338db47c4937c088ae739177bb426e7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-185.ec2.internal" Jul 15 21:01:36.375383 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:36.287999 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/29da48fefeff7a1251df734706134371-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal\" (UID: \"29da48fefeff7a1251df734706134371\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal" Jul 15 21:01:36.375383 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:36.288017 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/29da48fefeff7a1251df734706134371-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal\" (UID: \"29da48fefeff7a1251df734706134371\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal" Jul 15 21:01:36.375383 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:36.288092 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b338db47c4937c088ae739177bb426e7-config\") pod \"kube-apiserver-proxy-ip-10-0-137-185.ec2.internal\" (UID: \"b338db47c4937c088ae739177bb426e7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-185.ec2.internal" Jul 15 21:01:36.375383 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:36.288115 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/29da48fefeff7a1251df734706134371-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal\" (UID: \"29da48fefeff7a1251df734706134371\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal" Jul 15 21:01:36.375383 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:36.288115 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/29da48fefeff7a1251df734706134371-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal\" (UID: \"29da48fefeff7a1251df734706134371\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal" Jul 15 21:01:36.375383 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:36.352258 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-185.ec2.internal\" not found" Jul 15 21:01:36.529017 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:36.444887 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal" Jul 15 21:01:36.529017 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:36.449997 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-185.ec2.internal" Jul 15 21:01:36.529017 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:36.452786 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-185.ec2.internal\" not found" Jul 15 21:01:36.638113 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:36.553505 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-185.ec2.internal\" not found" Jul 15 21:01:36.777277 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:36.654126 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-185.ec2.internal\" not found" Jul 15 21:01:36.777277 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:36.754508 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-185.ec2.internal\" not found" Jul 15 21:01:36.911457 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:36.807350 2579 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 21:01:36.911457 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:36.811190 2579 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 15 21:01:36.911457 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:36.811336 2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 15 21:01:36.911457 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:36.811374 2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 15 21:01:36.911457 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:36.855537 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-185.ec2.internal\" not found" Jul 15 21:01:37.056052 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:36.946062 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-14 20:56:36 +0000 UTC" deadline="2027-12-30 00:07:46.354249074 +0000 UTC" Jul 15 21:01:37.056052 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:36.946101 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12771h6m9.408150638s" Jul 15 21:01:37.056052 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:36.956260 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-185.ec2.internal\" not found" Jul 15 21:01:37.056052 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.023006 2579 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 21:01:37.056052 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.053332 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 21:01:37.056052 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.054566 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasSufficientMemory" Jul 15 21:01:37.056052 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.054598 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 21:01:37.056052 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.054610 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasSufficientPID" Jul 15 21:01:37.060464 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:37.060323 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-185.ec2.internal\" not found" Jul 15 21:01:37.193796 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:37.161375 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-185.ec2.internal\" not found" Jul 15 21:01:37.332705 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:37.261730 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-185.ec2.internal\" not found" Jul 15 21:01:37.332705 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.332161 2579 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 21:01:37.471960 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.384594 2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal" Jul 15 21:01:37.471960 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.404754 2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 15 21:01:37.471960 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.405842 2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-185.ec2.internal" Jul 15 21:01:37.471960 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.417599 2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 15 21:01:37.890034 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.861594 2579 apiserver.go:52] "Watching apiserver" Jul 15 21:01:37.890034 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.864750 2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 21:01:37.890034 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.872105 2579 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 15 21:01:37.890034 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.873467 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal","openshift-multus/network-metrics-daemon-r59ft","openshift-network-diagnostics/network-check-target-dg4dh","openshift-dns/node-resolver-jw6xs","openshift-image-registry/node-ca-w2rr5","openshift-multus/multus-5vj8l","openshift-multus/multus-additional-cni-plugins-mstwf","openshift-network-operator/iptables-alerter-lpbxn","openshift-ovn-kubernetes/ovnkube-node-m5r2m","kube-system/global-pull-secret-syncer-4ctrp","kube-system/konnectivity-agent-g65nf","kube-system/kube-apiserver-proxy-ip-10-0-137-185.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-72wpg","openshift-cluster-node-tuning-operator/tuned-l78d9"] Jul 15 21:01:37.890034 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.877492 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dg4dh" Jul 15 21:01:37.890034 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:37.877564 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dg4dh" podUID="eeaa6fcf-b320-4851-9bf9-583c963e07ba" Jul 15 21:01:37.890034 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.879867 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:37.890034 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.882431 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mstwf" Jul 15 21:01:37.890034 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.884262 2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 15 21:01:37.890034 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.884425 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 15 21:01:37.890034 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.884653 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 15 21:01:37.890034 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.884832 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-fpkdl\"" Jul 15 21:01:37.890034 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.884860 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 15 21:01:37.890034 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.885087 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jw6xs" Jul 15 21:01:37.890034 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.885487 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 15 21:01:37.890034 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.886825 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 15 21:01:37.890034 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.886838 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 15 21:01:37.890034 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.887024 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 15 21:01:37.890034 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.887174 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 15 21:01:37.890034 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.887305 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 15 21:01:37.890034 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.887461 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-v65k9\"" Jul 15 21:01:37.890034 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.887528 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 15 21:01:37.890034 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.887593 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 15 21:01:37.890034 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.889013 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-w5dmv\"" Jul 15 21:01:37.890034 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.889217 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 15 21:01:37.890034 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.889377 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 15 21:01:37.899846 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.896596 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwcjv\" (UniqueName: \"kubernetes.io/projected/05324e48-e436-4e6a-9b35-5d0c661a82a4-kube-api-access-xwcjv\") pod \"node-resolver-jw6xs\" (UID: \"05324e48-e436-4e6a-9b35-5d0c661a82a4\") " pod="openshift-dns/node-resolver-jw6xs" Jul 15 21:01:37.899846 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.896628 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0db09c1e-d24d-4519-93c8-c185018c3d24-run-systemd\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:37.899846 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.896654 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0db09c1e-d24d-4519-93c8-c185018c3d24-run-ovn\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:37.899846 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.896683 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05324e48-e436-4e6a-9b35-5d0c661a82a4-tmp-dir\") pod \"node-resolver-jw6xs\" (UID: \"05324e48-e436-4e6a-9b35-5d0c661a82a4\") " pod="openshift-dns/node-resolver-jw6xs" Jul 15 21:01:37.899846 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.896721 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0db09c1e-d24d-4519-93c8-c185018c3d24-host-kubelet\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:37.899846 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.896746 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0db09c1e-d24d-4519-93c8-c185018c3d24-host-cni-netd\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:37.899846 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.896772 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0db09c1e-d24d-4519-93c8-c185018c3d24-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:37.899846 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.896796 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0db09c1e-d24d-4519-93c8-c185018c3d24-ovnkube-config\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:37.899846 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.896818 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0db09c1e-d24d-4519-93c8-c185018c3d24-env-overrides\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:37.899846 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.896841 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/397ae254-d712-41e8-a481-3243dbb511a9-cnibin\") pod \"multus-additional-cni-plugins-mstwf\" (UID: \"397ae254-d712-41e8-a481-3243dbb511a9\") " pod="openshift-multus/multus-additional-cni-plugins-mstwf" Jul 15 21:01:37.899846 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.896910 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0db09c1e-d24d-4519-93c8-c185018c3d24-run-openvswitch\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:37.899846 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.896949 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0db09c1e-d24d-4519-93c8-c185018c3d24-host-cni-bin\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:37.899846 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.896980 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/397ae254-d712-41e8-a481-3243dbb511a9-os-release\") pod \"multus-additional-cni-plugins-mstwf\" (UID: \"397ae254-d712-41e8-a481-3243dbb511a9\") " pod="openshift-multus/multus-additional-cni-plugins-mstwf" Jul 15 21:01:37.899846 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.897018 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0db09c1e-d24d-4519-93c8-c185018c3d24-systemd-units\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:38.013565 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.897044 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0db09c1e-d24d-4519-93c8-c185018c3d24-host-slash\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:38.013565 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.897065 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0db09c1e-d24d-4519-93c8-c185018c3d24-host-run-ovn-kubernetes\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:38.013565 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.897095 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0db09c1e-d24d-4519-93c8-c185018c3d24-ovnkube-script-lib\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:38.013565 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.897136 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/397ae254-d712-41e8-a481-3243dbb511a9-system-cni-dir\") pod \"multus-additional-cni-plugins-mstwf\" (UID: \"397ae254-d712-41e8-a481-3243dbb511a9\") " pod="openshift-multus/multus-additional-cni-plugins-mstwf" Jul 15 21:01:38.013565 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.897160 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/05324e48-e436-4e6a-9b35-5d0c661a82a4-hosts-file\") pod \"node-resolver-jw6xs\" (UID: \"05324e48-e436-4e6a-9b35-5d0c661a82a4\") " pod="openshift-dns/node-resolver-jw6xs" Jul 15 21:01:38.013565 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.897224 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0db09c1e-d24d-4519-93c8-c185018c3d24-ovn-node-metrics-cert\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:38.013565 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.897252 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/397ae254-d712-41e8-a481-3243dbb511a9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mstwf\" (UID: \"397ae254-d712-41e8-a481-3243dbb511a9\") " pod="openshift-multus/multus-additional-cni-plugins-mstwf" Jul 15 21:01:38.013565 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.897276 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/397ae254-d712-41e8-a481-3243dbb511a9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mstwf\" (UID: \"397ae254-d712-41e8-a481-3243dbb511a9\") " pod="openshift-multus/multus-additional-cni-plugins-mstwf" Jul 15 21:01:38.013565 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.897300 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0db09c1e-d24d-4519-93c8-c185018c3d24-etc-openvswitch\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:38.013565 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.897336 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0db09c1e-d24d-4519-93c8-c185018c3d24-host-run-netns\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:38.013565 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.897359 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0db09c1e-d24d-4519-93c8-c185018c3d24-var-lib-openvswitch\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:38.013565 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.897386 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhpjr\" (UniqueName: \"kubernetes.io/projected/0db09c1e-d24d-4519-93c8-c185018c3d24-kube-api-access-jhpjr\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:38.013565 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.897444 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/397ae254-d712-41e8-a481-3243dbb511a9-cni-binary-copy\") pod \"multus-additional-cni-plugins-mstwf\" (UID: \"397ae254-d712-41e8-a481-3243dbb511a9\") " pod="openshift-multus/multus-additional-cni-plugins-mstwf" Jul 15 21:01:38.014881 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.897471 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fptjd\" (UniqueName: \"kubernetes.io/projected/eeaa6fcf-b320-4851-9bf9-583c963e07ba-kube-api-access-fptjd\") pod \"network-check-target-dg4dh\" (UID: \"eeaa6fcf-b320-4851-9bf9-583c963e07ba\") " pod="openshift-network-diagnostics/network-check-target-dg4dh" Jul 15 21:01:38.014881 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.897497 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0db09c1e-d24d-4519-93c8-c185018c3d24-node-log\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:38.014881 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.897519 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0db09c1e-d24d-4519-93c8-c185018c3d24-log-socket\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:38.014881 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.897544 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/397ae254-d712-41e8-a481-3243dbb511a9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mstwf\" (UID: \"397ae254-d712-41e8-a481-3243dbb511a9\") " pod="openshift-multus/multus-additional-cni-plugins-mstwf" Jul 15 21:01:38.014881 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.897586 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hwt7\" (UniqueName: \"kubernetes.io/projected/397ae254-d712-41e8-a481-3243dbb511a9-kube-api-access-8hwt7\") pod \"multus-additional-cni-plugins-mstwf\" (UID: \"397ae254-d712-41e8-a481-3243dbb511a9\") " pod="openshift-multus/multus-additional-cni-plugins-mstwf" Jul 15 21:01:38.014881 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.899980 2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 15 21:01:38.014881 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.936048 2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-g2p7z" Jul 15 21:01:38.014881 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.946387 2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-g2p7z" Jul 15 21:01:38.014881 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.998603 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwcjv\" (UniqueName: \"kubernetes.io/projected/05324e48-e436-4e6a-9b35-5d0c661a82a4-kube-api-access-xwcjv\") pod \"node-resolver-jw6xs\" (UID: \"05324e48-e436-4e6a-9b35-5d0c661a82a4\") " pod="openshift-dns/node-resolver-jw6xs" Jul 15 21:01:38.014881 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.998640 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0db09c1e-d24d-4519-93c8-c185018c3d24-run-systemd\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:38.014881 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.998668 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0db09c1e-d24d-4519-93c8-c185018c3d24-run-ovn\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:38.014881 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.998693 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05324e48-e436-4e6a-9b35-5d0c661a82a4-tmp-dir\") pod \"node-resolver-jw6xs\" (UID: \"05324e48-e436-4e6a-9b35-5d0c661a82a4\") " pod="openshift-dns/node-resolver-jw6xs" Jul 15 21:01:38.014881 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.998713 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0db09c1e-d24d-4519-93c8-c185018c3d24-host-kubelet\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:38.014881 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.998727 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0db09c1e-d24d-4519-93c8-c185018c3d24-run-systemd\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:38.014881 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.998732 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0db09c1e-d24d-4519-93c8-c185018c3d24-host-cni-netd\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:38.014881 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.998770 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0db09c1e-d24d-4519-93c8-c185018c3d24-host-cni-netd\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:38.025803 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.998789 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0db09c1e-d24d-4519-93c8-c185018c3d24-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:38.025803 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.998814 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0db09c1e-d24d-4519-93c8-c185018c3d24-host-kubelet\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:38.025803 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.998818 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0db09c1e-d24d-4519-93c8-c185018c3d24-ovnkube-config\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:38.025803 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.998843 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0db09c1e-d24d-4519-93c8-c185018c3d24-env-overrides\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:38.025803 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.998845 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0db09c1e-d24d-4519-93c8-c185018c3d24-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:38.025803 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.998865 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/397ae254-d712-41e8-a481-3243dbb511a9-cnibin\") pod \"multus-additional-cni-plugins-mstwf\" (UID: \"397ae254-d712-41e8-a481-3243dbb511a9\") " pod="openshift-multus/multus-additional-cni-plugins-mstwf" Jul 15 21:01:38.025803 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.998870 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0db09c1e-d24d-4519-93c8-c185018c3d24-run-ovn\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:38.025803 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.998887 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0db09c1e-d24d-4519-93c8-c185018c3d24-run-openvswitch\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:38.025803 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.998923 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0db09c1e-d24d-4519-93c8-c185018c3d24-run-openvswitch\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:38.025803 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.998942 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0db09c1e-d24d-4519-93c8-c185018c3d24-host-cni-bin\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:38.025803 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.998962 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/397ae254-d712-41e8-a481-3243dbb511a9-cnibin\") pod \"multus-additional-cni-plugins-mstwf\" (UID: \"397ae254-d712-41e8-a481-3243dbb511a9\") " pod="openshift-multus/multus-additional-cni-plugins-mstwf" Jul 15 21:01:38.025803 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.998970 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/397ae254-d712-41e8-a481-3243dbb511a9-os-release\") pod \"multus-additional-cni-plugins-mstwf\" (UID: \"397ae254-d712-41e8-a481-3243dbb511a9\") " pod="openshift-multus/multus-additional-cni-plugins-mstwf" Jul 15 21:01:38.025803 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.999022 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kghcd\" (UniqueName: \"kubernetes.io/projected/e24afff0-0ef0-4463-92fc-848462106a1b-kube-api-access-kghcd\") pod \"node-ca-w2rr5\" (UID: \"e24afff0-0ef0-4463-92fc-848462106a1b\") " pod="openshift-image-registry/node-ca-w2rr5" Jul 15 21:01:38.025803 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.999053 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0db09c1e-d24d-4519-93c8-c185018c3d24-systemd-units\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:38.027198 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.999102 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0db09c1e-d24d-4519-93c8-c185018c3d24-host-slash\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:38.027198 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.999129 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0db09c1e-d24d-4519-93c8-c185018c3d24-host-run-ovn-kubernetes\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:38.027198 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.999152 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0db09c1e-d24d-4519-93c8-c185018c3d24-ovnkube-script-lib\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:38.027198 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.999177 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/397ae254-d712-41e8-a481-3243dbb511a9-system-cni-dir\") pod \"multus-additional-cni-plugins-mstwf\" (UID: \"397ae254-d712-41e8-a481-3243dbb511a9\") " pod="openshift-multus/multus-additional-cni-plugins-mstwf" Jul 15 21:01:38.027198 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.999199 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e24afff0-0ef0-4463-92fc-848462106a1b-host\") pod \"node-ca-w2rr5\" (UID: \"e24afff0-0ef0-4463-92fc-848462106a1b\") " pod="openshift-image-registry/node-ca-w2rr5" Jul 15 21:01:38.027198 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.999224 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/05324e48-e436-4e6a-9b35-5d0c661a82a4-hosts-file\") pod \"node-resolver-jw6xs\" (UID: \"05324e48-e436-4e6a-9b35-5d0c661a82a4\") " pod="openshift-dns/node-resolver-jw6xs" Jul 15 21:01:38.027198 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.999247 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0db09c1e-d24d-4519-93c8-c185018c3d24-ovn-node-metrics-cert\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:38.027198 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.999270 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/397ae254-d712-41e8-a481-3243dbb511a9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mstwf\" (UID: \"397ae254-d712-41e8-a481-3243dbb511a9\") " pod="openshift-multus/multus-additional-cni-plugins-mstwf" Jul 15 21:01:38.027198 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.999293 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/397ae254-d712-41e8-a481-3243dbb511a9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mstwf\" (UID: \"397ae254-d712-41e8-a481-3243dbb511a9\") " pod="openshift-multus/multus-additional-cni-plugins-mstwf" Jul 15 21:01:38.027198 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.999317 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0db09c1e-d24d-4519-93c8-c185018c3d24-etc-openvswitch\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:38.027198 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.999340 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0db09c1e-d24d-4519-93c8-c185018c3d24-host-run-netns\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:38.027198 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.999364 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0db09c1e-d24d-4519-93c8-c185018c3d24-var-lib-openvswitch\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:38.027198 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.999388 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhpjr\" (UniqueName: \"kubernetes.io/projected/0db09c1e-d24d-4519-93c8-c185018c3d24-kube-api-access-jhpjr\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:38.027198 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.999412 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/397ae254-d712-41e8-a481-3243dbb511a9-cni-binary-copy\") pod \"multus-additional-cni-plugins-mstwf\" (UID: \"397ae254-d712-41e8-a481-3243dbb511a9\") " pod="openshift-multus/multus-additional-cni-plugins-mstwf" Jul 15 21:01:38.028109 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.999426 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/397ae254-d712-41e8-a481-3243dbb511a9-system-cni-dir\") pod \"multus-additional-cni-plugins-mstwf\" (UID: \"397ae254-d712-41e8-a481-3243dbb511a9\") " pod="openshift-multus/multus-additional-cni-plugins-mstwf" Jul 15 21:01:38.028109 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.999438 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e24afff0-0ef0-4463-92fc-848462106a1b-serviceca\") pod \"node-ca-w2rr5\" (UID: \"e24afff0-0ef0-4463-92fc-848462106a1b\") " pod="openshift-image-registry/node-ca-w2rr5" Jul 15 21:01:38.028109 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.999463 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fptjd\" (UniqueName: \"kubernetes.io/projected/eeaa6fcf-b320-4851-9bf9-583c963e07ba-kube-api-access-fptjd\") pod \"network-check-target-dg4dh\" (UID: \"eeaa6fcf-b320-4851-9bf9-583c963e07ba\") " pod="openshift-network-diagnostics/network-check-target-dg4dh" Jul 15 21:01:38.028109 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.999470 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0db09c1e-d24d-4519-93c8-c185018c3d24-host-cni-bin\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:38.028109 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.999466 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0db09c1e-d24d-4519-93c8-c185018c3d24-ovnkube-config\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:38.028109 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.999504 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0db09c1e-d24d-4519-93c8-c185018c3d24-systemd-units\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:38.028109 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.999508 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0db09c1e-d24d-4519-93c8-c185018c3d24-host-slash\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:38.028109 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.999558 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/05324e48-e436-4e6a-9b35-5d0c661a82a4-hosts-file\") pod \"node-resolver-jw6xs\" (UID: \"05324e48-e436-4e6a-9b35-5d0c661a82a4\") " pod="openshift-dns/node-resolver-jw6xs" Jul 15 21:01:38.028109 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.999572 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/397ae254-d712-41e8-a481-3243dbb511a9-os-release\") pod \"multus-additional-cni-plugins-mstwf\" (UID: \"397ae254-d712-41e8-a481-3243dbb511a9\") " pod="openshift-multus/multus-additional-cni-plugins-mstwf" Jul 15 21:01:38.028109 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.999595 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0db09c1e-d24d-4519-93c8-c185018c3d24-host-run-ovn-kubernetes\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:38.028109 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.999604 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0db09c1e-d24d-4519-93c8-c185018c3d24-node-log\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:38.028109 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.999628 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0db09c1e-d24d-4519-93c8-c185018c3d24-log-socket\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:38.028109 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.999651 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/397ae254-d712-41e8-a481-3243dbb511a9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mstwf\" (UID: \"397ae254-d712-41e8-a481-3243dbb511a9\") " pod="openshift-multus/multus-additional-cni-plugins-mstwf" Jul 15 21:01:38.028109 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.999682 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hwt7\" (UniqueName: \"kubernetes.io/projected/397ae254-d712-41e8-a481-3243dbb511a9-kube-api-access-8hwt7\") pod \"multus-additional-cni-plugins-mstwf\" (UID: \"397ae254-d712-41e8-a481-3243dbb511a9\") " pod="openshift-multus/multus-additional-cni-plugins-mstwf" Jul 15 21:01:38.029889 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.999848 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0db09c1e-d24d-4519-93c8-c185018c3d24-var-lib-openvswitch\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:38.029889 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.999857 2579 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 15 21:01:38.029889 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.999919 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0db09c1e-d24d-4519-93c8-c185018c3d24-node-log\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:38.029889 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.999960 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0db09c1e-d24d-4519-93c8-c185018c3d24-log-socket\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:38.029889 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.999986 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0db09c1e-d24d-4519-93c8-c185018c3d24-ovnkube-script-lib\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:38.029889 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.999992 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/397ae254-d712-41e8-a481-3243dbb511a9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mstwf\" (UID: \"397ae254-d712-41e8-a481-3243dbb511a9\") " pod="openshift-multus/multus-additional-cni-plugins-mstwf" Jul 15 21:01:38.029889 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.000035 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0db09c1e-d24d-4519-93c8-c185018c3d24-host-run-netns\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:38.029889 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.000066 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0db09c1e-d24d-4519-93c8-c185018c3d24-etc-openvswitch\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:38.029889 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.000432 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/397ae254-d712-41e8-a481-3243dbb511a9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mstwf\" (UID: \"397ae254-d712-41e8-a481-3243dbb511a9\") " pod="openshift-multus/multus-additional-cni-plugins-mstwf" Jul 15 21:01:38.029889 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.000458 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/397ae254-d712-41e8-a481-3243dbb511a9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mstwf\" (UID: \"397ae254-d712-41e8-a481-3243dbb511a9\") " pod="openshift-multus/multus-additional-cni-plugins-mstwf" Jul 15 21:01:38.029889 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:37.999386 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0db09c1e-d24d-4519-93c8-c185018c3d24-env-overrides\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:38.029889 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.000559 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/397ae254-d712-41e8-a481-3243dbb511a9-cni-binary-copy\") pod \"multus-additional-cni-plugins-mstwf\" (UID: \"397ae254-d712-41e8-a481-3243dbb511a9\") " pod="openshift-multus/multus-additional-cni-plugins-mstwf" Jul 15 21:01:38.029889 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.004221 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0db09c1e-d24d-4519-93c8-c185018c3d24-ovn-node-metrics-cert\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:38.029889 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:38.008546 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 21:01:38.029889 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:38.008566 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 21:01:38.030777 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:38.008578 2579 projected.go:196] Error preparing data for projected volume kube-api-access-fptjd for pod openshift-network-diagnostics/network-check-target-dg4dh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 21:01:38.030777 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:38.008649 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eeaa6fcf-b320-4851-9bf9-583c963e07ba-kube-api-access-fptjd podName:eeaa6fcf-b320-4851-9bf9-583c963e07ba nodeName:}" failed. No retries permitted until 2026-07-15 21:01:38.508630865 +0000 UTC m=+3.049943412 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-fptjd" (UniqueName: "kubernetes.io/projected/eeaa6fcf-b320-4851-9bf9-583c963e07ba-kube-api-access-fptjd") pod "network-check-target-dg4dh" (UID: "eeaa6fcf-b320-4851-9bf9-583c963e07ba") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 21:01:38.030777 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.010082 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hwt7\" (UniqueName: \"kubernetes.io/projected/397ae254-d712-41e8-a481-3243dbb511a9-kube-api-access-8hwt7\") pod \"multus-additional-cni-plugins-mstwf\" (UID: \"397ae254-d712-41e8-a481-3243dbb511a9\") " pod="openshift-multus/multus-additional-cni-plugins-mstwf" Jul 15 21:01:38.030777 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.010407 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhpjr\" (UniqueName: \"kubernetes.io/projected/0db09c1e-d24d-4519-93c8-c185018c3d24-kube-api-access-jhpjr\") pod \"ovnkube-node-m5r2m\" (UID: \"0db09c1e-d24d-4519-93c8-c185018c3d24\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:38.082853 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.082821 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-w2rr5" Jul 15 21:01:38.083826 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.083684 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05324e48-e436-4e6a-9b35-5d0c661a82a4-tmp-dir\") pod \"node-resolver-jw6xs\" (UID: \"05324e48-e436-4e6a-9b35-5d0c661a82a4\") " pod="openshift-dns/node-resolver-jw6xs" Jul 15 21:01:38.086381 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.086190 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-kk7mp\"" Jul 15 21:01:38.086381 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.086358 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 15 21:01:38.086827 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.086782 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 15 21:01:38.086827 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.086819 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwcjv\" (UniqueName: \"kubernetes.io/projected/05324e48-e436-4e6a-9b35-5d0c661a82a4-kube-api-access-xwcjv\") pod \"node-resolver-jw6xs\" (UID: \"05324e48-e436-4e6a-9b35-5d0c661a82a4\") " pod="openshift-dns/node-resolver-jw6xs" Jul 15 21:01:38.086991 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.086871 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 15 21:01:38.087135 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.087113 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5vj8l" Jul 15 21:01:38.087247 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.087229 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 21:01:38.089787 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.089763 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lpbxn" Jul 15 21:01:38.091058 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.091036 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-grr45\"" Jul 15 21:01:38.091253 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.091093 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 15 21:01:38.092505 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.092485 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r59ft" Jul 15 21:01:38.092585 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:38.092555 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r59ft" podUID="0dafa58d-15c7-46c9-81bd-c3c86bbf8e42" Jul 15 21:01:38.095099 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.094963 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4ctrp" Jul 15 21:01:38.095099 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:38.095047 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4ctrp" podUID="3f26570b-5abd-4460-ba74-e59ba65032a1" Jul 15 21:01:38.097456 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.097440 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-g65nf" Jul 15 21:01:38.099917 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.099891 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3edce062-4a75-40dc-80c3-a85ee51aee76-system-cni-dir\") pod \"multus-5vj8l\" (UID: \"3edce062-4a75-40dc-80c3-a85ee51aee76\") " pod="openshift-multus/multus-5vj8l" Jul 15 21:01:38.100024 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.099931 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3edce062-4a75-40dc-80c3-a85ee51aee76-host-run-netns\") pod \"multus-5vj8l\" (UID: \"3edce062-4a75-40dc-80c3-a85ee51aee76\") " pod="openshift-multus/multus-5vj8l" Jul 15 21:01:38.100024 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.099950 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-72wpg" Jul 15 21:01:38.100024 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.099976 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3edce062-4a75-40dc-80c3-a85ee51aee76-host-var-lib-cni-bin\") pod \"multus-5vj8l\" (UID: \"3edce062-4a75-40dc-80c3-a85ee51aee76\") " pod="openshift-multus/multus-5vj8l" Jul 15 21:01:38.100380 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.100038 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e24afff0-0ef0-4463-92fc-848462106a1b-serviceca\") pod \"node-ca-w2rr5\" (UID: \"e24afff0-0ef0-4463-92fc-848462106a1b\") " pod="openshift-image-registry/node-ca-w2rr5" Jul 15 21:01:38.100380 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.100066 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3edce062-4a75-40dc-80c3-a85ee51aee76-cni-binary-copy\") pod \"multus-5vj8l\" (UID: \"3edce062-4a75-40dc-80c3-a85ee51aee76\") " pod="openshift-multus/multus-5vj8l" Jul 15 21:01:38.100380 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.100132 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3edce062-4a75-40dc-80c3-a85ee51aee76-host-run-multus-certs\") pod \"multus-5vj8l\" (UID: \"3edce062-4a75-40dc-80c3-a85ee51aee76\") " pod="openshift-multus/multus-5vj8l" Jul 15 21:01:38.100380 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.100245 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3edce062-4a75-40dc-80c3-a85ee51aee76-cnibin\") pod \"multus-5vj8l\" (UID: \"3edce062-4a75-40dc-80c3-a85ee51aee76\") " pod="openshift-multus/multus-5vj8l" Jul 15 21:01:38.100380 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.100302 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3edce062-4a75-40dc-80c3-a85ee51aee76-host-run-k8s-cni-cncf-io\") pod \"multus-5vj8l\" (UID: \"3edce062-4a75-40dc-80c3-a85ee51aee76\") " pod="openshift-multus/multus-5vj8l" Jul 15 21:01:38.100380 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.100332 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3edce062-4a75-40dc-80c3-a85ee51aee76-host-var-lib-cni-multus\") pod \"multus-5vj8l\" (UID: \"3edce062-4a75-40dc-80c3-a85ee51aee76\") " pod="openshift-multus/multus-5vj8l" Jul 15 21:01:38.100380 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.100356 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3edce062-4a75-40dc-80c3-a85ee51aee76-host-var-lib-kubelet\") pod \"multus-5vj8l\" (UID: \"3edce062-4a75-40dc-80c3-a85ee51aee76\") " pod="openshift-multus/multus-5vj8l" Jul 15 21:01:38.100594 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.100412 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3edce062-4a75-40dc-80c3-a85ee51aee76-os-release\") pod \"multus-5vj8l\" (UID: \"3edce062-4a75-40dc-80c3-a85ee51aee76\") " pod="openshift-multus/multus-5vj8l" Jul 15 21:01:38.100594 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.100442 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3edce062-4a75-40dc-80c3-a85ee51aee76-multus-daemon-config\") pod \"multus-5vj8l\" (UID: \"3edce062-4a75-40dc-80c3-a85ee51aee76\") " pod="openshift-multus/multus-5vj8l" Jul 15 21:01:38.100594 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.100483 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3edce062-4a75-40dc-80c3-a85ee51aee76-multus-conf-dir\") pod \"multus-5vj8l\" (UID: \"3edce062-4a75-40dc-80c3-a85ee51aee76\") " pod="openshift-multus/multus-5vj8l" Jul 15 21:01:38.100594 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.100543 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kghcd\" (UniqueName: \"kubernetes.io/projected/e24afff0-0ef0-4463-92fc-848462106a1b-kube-api-access-kghcd\") pod \"node-ca-w2rr5\" (UID: \"e24afff0-0ef0-4463-92fc-848462106a1b\") " pod="openshift-image-registry/node-ca-w2rr5" Jul 15 21:01:38.100594 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.100565 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3edce062-4a75-40dc-80c3-a85ee51aee76-multus-cni-dir\") pod \"multus-5vj8l\" (UID: \"3edce062-4a75-40dc-80c3-a85ee51aee76\") " pod="openshift-multus/multus-5vj8l" Jul 15 21:01:38.100594 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.100580 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e24afff0-0ef0-4463-92fc-848462106a1b-serviceca\") pod \"node-ca-w2rr5\" (UID: \"e24afff0-0ef0-4463-92fc-848462106a1b\") " pod="openshift-image-registry/node-ca-w2rr5" Jul 15 21:01:38.100776 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.100582 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3edce062-4a75-40dc-80c3-a85ee51aee76-hostroot\") pod \"multus-5vj8l\" (UID: \"3edce062-4a75-40dc-80c3-a85ee51aee76\") " pod="openshift-multus/multus-5vj8l" Jul 15 21:01:38.100776 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.100637 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3edce062-4a75-40dc-80c3-a85ee51aee76-etc-kubernetes\") pod \"multus-5vj8l\" (UID: \"3edce062-4a75-40dc-80c3-a85ee51aee76\") " pod="openshift-multus/multus-5vj8l" Jul 15 21:01:38.100776 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.100658 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxc9f\" (UniqueName: \"kubernetes.io/projected/3edce062-4a75-40dc-80c3-a85ee51aee76-kube-api-access-kxc9f\") pod \"multus-5vj8l\" (UID: \"3edce062-4a75-40dc-80c3-a85ee51aee76\") " pod="openshift-multus/multus-5vj8l" Jul 15 21:01:38.100776 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.100681 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e24afff0-0ef0-4463-92fc-848462106a1b-host\") pod \"node-ca-w2rr5\" (UID: \"e24afff0-0ef0-4463-92fc-848462106a1b\") " pod="openshift-image-registry/node-ca-w2rr5" Jul 15 21:01:38.100776 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.100698 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3edce062-4a75-40dc-80c3-a85ee51aee76-multus-socket-dir-parent\") pod \"multus-5vj8l\" (UID: \"3edce062-4a75-40dc-80c3-a85ee51aee76\") " pod="openshift-multus/multus-5vj8l" Jul 15 21:01:38.100776 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.100750 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e24afff0-0ef0-4463-92fc-848462106a1b-host\") pod \"node-ca-w2rr5\" (UID: \"e24afff0-0ef0-4463-92fc-848462106a1b\") " pod="openshift-image-registry/node-ca-w2rr5" Jul 15 21:01:38.104681 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.104445 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-l78d9" Jul 15 21:01:38.107712 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.107672 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 15 21:01:38.108172 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.107822 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 15 21:01:38.108172 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.108010 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 15 21:01:38.108172 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.108017 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 15 21:01:38.108172 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.108032 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-cq9vw\"" Jul 15 21:01:38.108172 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.108054 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-d7vs5\"" Jul 15 21:01:38.108172 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.108054 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 15 21:01:38.108505 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.108241 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 15 21:01:38.109049 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.108835 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-nd9wk\"" Jul 15 21:01:38.109220 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.109197 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 15 21:01:38.109304 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.109285 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 15 21:01:38.109472 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.109038 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal" event={"ID":"29da48fefeff7a1251df734706134371","Type":"ContainerStarted","Data":"c462aad08634d05cec3305155127935464845726d602cf446f30af22957a1c2c"} Jul 15 21:01:38.112298 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.112275 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 15 21:01:38.112388 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.112282 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 15 21:01:38.113319 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.113297 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kghcd\" (UniqueName: \"kubernetes.io/projected/e24afff0-0ef0-4463-92fc-848462106a1b-kube-api-access-kghcd\") pod \"node-ca-w2rr5\" (UID: \"e24afff0-0ef0-4463-92fc-848462106a1b\") " pod="openshift-image-registry/node-ca-w2rr5" Jul 15 21:01:38.116887 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.116866 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-2gwfp\"" Jul 15 21:01:38.150823 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:38.150797 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb338db47c4937c088ae739177bb426e7.slice/crio-9c62b8f917e3ae36b22622172732c07db9e4319f71103e762e87791a0dfb7f08 WatchSource:0}: Error finding container 9c62b8f917e3ae36b22622172732c07db9e4319f71103e762e87791a0dfb7f08: Status 404 returned error can't find the container with id 9c62b8f917e3ae36b22622172732c07db9e4319f71103e762e87791a0dfb7f08 Jul 15 21:01:38.185026 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.184990 2579 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 15 21:01:38.195530 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.195509 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:01:38.200962 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.200939 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3edce062-4a75-40dc-80c3-a85ee51aee76-cni-binary-copy\") pod \"multus-5vj8l\" (UID: \"3edce062-4a75-40dc-80c3-a85ee51aee76\") " pod="openshift-multus/multus-5vj8l" Jul 15 21:01:38.201073 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.200974 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3edce062-4a75-40dc-80c3-a85ee51aee76-cnibin\") pod \"multus-5vj8l\" (UID: \"3edce062-4a75-40dc-80c3-a85ee51aee76\") " pod="openshift-multus/multus-5vj8l" Jul 15 21:01:38.201073 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.200993 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3edce062-4a75-40dc-80c3-a85ee51aee76-host-run-k8s-cni-cncf-io\") pod \"multus-5vj8l\" (UID: \"3edce062-4a75-40dc-80c3-a85ee51aee76\") " pod="openshift-multus/multus-5vj8l" Jul 15 21:01:38.201073 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.201009 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3edce062-4a75-40dc-80c3-a85ee51aee76-host-var-lib-kubelet\") pod \"multus-5vj8l\" (UID: \"3edce062-4a75-40dc-80c3-a85ee51aee76\") " pod="openshift-multus/multus-5vj8l" Jul 15 21:01:38.201073 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.201068 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3edce062-4a75-40dc-80c3-a85ee51aee76-host-var-lib-kubelet\") pod \"multus-5vj8l\" (UID: \"3edce062-4a75-40dc-80c3-a85ee51aee76\") " pod="openshift-multus/multus-5vj8l" Jul 15 21:01:38.201349 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.201100 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3edce062-4a75-40dc-80c3-a85ee51aee76-host-run-k8s-cni-cncf-io\") pod \"multus-5vj8l\" (UID: \"3edce062-4a75-40dc-80c3-a85ee51aee76\") " pod="openshift-multus/multus-5vj8l" Jul 15 21:01:38.201349 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.201119 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3edce062-4a75-40dc-80c3-a85ee51aee76-cnibin\") pod \"multus-5vj8l\" (UID: \"3edce062-4a75-40dc-80c3-a85ee51aee76\") " pod="openshift-multus/multus-5vj8l" Jul 15 21:01:38.201349 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.201235 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/47738f2f-70c7-45f4-b2f3-6cd99846e503-registration-dir\") pod \"aws-ebs-csi-driver-node-72wpg\" (UID: \"47738f2f-70c7-45f4-b2f3-6cd99846e503\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-72wpg" Jul 15 21:01:38.201349 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.201262 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1aac0a47-4b65-48e2-8fca-03b686fe5512-var-lib-kubelet\") pod \"tuned-l78d9\" (UID: \"1aac0a47-4b65-48e2-8fca-03b686fe5512\") " pod="openshift-cluster-node-tuning-operator/tuned-l78d9" Jul 15 21:01:38.201349 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.201284 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1aac0a47-4b65-48e2-8fca-03b686fe5512-etc-tuned\") pod \"tuned-l78d9\" (UID: \"1aac0a47-4b65-48e2-8fca-03b686fe5512\") " pod="openshift-cluster-node-tuning-operator/tuned-l78d9" Jul 15 21:01:38.201349 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.201308 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3edce062-4a75-40dc-80c3-a85ee51aee76-os-release\") pod \"multus-5vj8l\" (UID: \"3edce062-4a75-40dc-80c3-a85ee51aee76\") " pod="openshift-multus/multus-5vj8l" Jul 15 21:01:38.201349 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.201332 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f7dba3ba-aec0-4137-b84e-4ce93d09c5a0-konnectivity-ca\") pod \"konnectivity-agent-g65nf\" (UID: \"f7dba3ba-aec0-4137-b84e-4ce93d09c5a0\") " pod="kube-system/konnectivity-agent-g65nf" Jul 15 21:01:38.201709 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.201359 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1aac0a47-4b65-48e2-8fca-03b686fe5512-lib-modules\") pod \"tuned-l78d9\" (UID: \"1aac0a47-4b65-48e2-8fca-03b686fe5512\") " pod="openshift-cluster-node-tuning-operator/tuned-l78d9" Jul 15 21:01:38.201709 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.201440 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3edce062-4a75-40dc-80c3-a85ee51aee76-os-release\") pod \"multus-5vj8l\" (UID: \"3edce062-4a75-40dc-80c3-a85ee51aee76\") " pod="openshift-multus/multus-5vj8l" Jul 15 21:01:38.201709 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.201440 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3edce062-4a75-40dc-80c3-a85ee51aee76-multus-cni-dir\") pod \"multus-5vj8l\" (UID: \"3edce062-4a75-40dc-80c3-a85ee51aee76\") " pod="openshift-multus/multus-5vj8l" Jul 15 21:01:38.201709 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.201514 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3edce062-4a75-40dc-80c3-a85ee51aee76-hostroot\") pod \"multus-5vj8l\" (UID: \"3edce062-4a75-40dc-80c3-a85ee51aee76\") " pod="openshift-multus/multus-5vj8l" Jul 15 21:01:38.201709 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.201550 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3edce062-4a75-40dc-80c3-a85ee51aee76-cni-binary-copy\") pod \"multus-5vj8l\" (UID: \"3edce062-4a75-40dc-80c3-a85ee51aee76\") " pod="openshift-multus/multus-5vj8l" Jul 15 21:01:38.201709 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.201568 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3edce062-4a75-40dc-80c3-a85ee51aee76-hostroot\") pod \"multus-5vj8l\" (UID: \"3edce062-4a75-40dc-80c3-a85ee51aee76\") " pod="openshift-multus/multus-5vj8l" Jul 15 21:01:38.201709 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.201587 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxc9f\" (UniqueName: \"kubernetes.io/projected/3edce062-4a75-40dc-80c3-a85ee51aee76-kube-api-access-kxc9f\") pod \"multus-5vj8l\" (UID: \"3edce062-4a75-40dc-80c3-a85ee51aee76\") " pod="openshift-multus/multus-5vj8l" Jul 15 21:01:38.201709 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.201619 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1aac0a47-4b65-48e2-8fca-03b686fe5512-run\") pod \"tuned-l78d9\" (UID: \"1aac0a47-4b65-48e2-8fca-03b686fe5512\") " pod="openshift-cluster-node-tuning-operator/tuned-l78d9" Jul 15 21:01:38.201709 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.201644 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1aac0a47-4b65-48e2-8fca-03b686fe5512-sys\") pod \"tuned-l78d9\" (UID: \"1aac0a47-4b65-48e2-8fca-03b686fe5512\") " pod="openshift-cluster-node-tuning-operator/tuned-l78d9" Jul 15 21:01:38.201709 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.201653 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3edce062-4a75-40dc-80c3-a85ee51aee76-multus-cni-dir\") pod \"multus-5vj8l\" (UID: \"3edce062-4a75-40dc-80c3-a85ee51aee76\") " pod="openshift-multus/multus-5vj8l" Jul 15 21:01:38.201709 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.201667 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1aac0a47-4b65-48e2-8fca-03b686fe5512-tmp\") pod \"tuned-l78d9\" (UID: \"1aac0a47-4b65-48e2-8fca-03b686fe5512\") " pod="openshift-cluster-node-tuning-operator/tuned-l78d9" Jul 15 21:01:38.201709 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.201693 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3edce062-4a75-40dc-80c3-a85ee51aee76-etc-kubernetes\") pod \"multus-5vj8l\" (UID: \"3edce062-4a75-40dc-80c3-a85ee51aee76\") " pod="openshift-multus/multus-5vj8l" Jul 15 21:01:38.201709 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.201719 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3edce062-4a75-40dc-80c3-a85ee51aee76-etc-kubernetes\") pod \"multus-5vj8l\" (UID: \"3edce062-4a75-40dc-80c3-a85ee51aee76\") " pod="openshift-multus/multus-5vj8l" Jul 15 21:01:38.202231 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.201759 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3edce062-4a75-40dc-80c3-a85ee51aee76-host-var-lib-cni-bin\") pod \"multus-5vj8l\" (UID: \"3edce062-4a75-40dc-80c3-a85ee51aee76\") " pod="openshift-multus/multus-5vj8l" Jul 15 21:01:38.202231 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.201777 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3edce062-4a75-40dc-80c3-a85ee51aee76-host-run-multus-certs\") pod \"multus-5vj8l\" (UID: \"3edce062-4a75-40dc-80c3-a85ee51aee76\") " pod="openshift-multus/multus-5vj8l" Jul 15 21:01:38.202231 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.201795 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3edce062-4a75-40dc-80c3-a85ee51aee76-system-cni-dir\") pod \"multus-5vj8l\" (UID: \"3edce062-4a75-40dc-80c3-a85ee51aee76\") " pod="openshift-multus/multus-5vj8l" Jul 15 21:01:38.202231 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.201815 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3edce062-4a75-40dc-80c3-a85ee51aee76-host-var-lib-cni-bin\") pod \"multus-5vj8l\" (UID: \"3edce062-4a75-40dc-80c3-a85ee51aee76\") " pod="openshift-multus/multus-5vj8l" Jul 15 21:01:38.202231 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.201828 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3edce062-4a75-40dc-80c3-a85ee51aee76-host-run-multus-certs\") pod \"multus-5vj8l\" (UID: \"3edce062-4a75-40dc-80c3-a85ee51aee76\") " pod="openshift-multus/multus-5vj8l" Jul 15 21:01:38.202231 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.201819 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1964ea3b-7a10-404e-b765-29e2636d299b-iptables-alerter-script\") pod \"iptables-alerter-lpbxn\" (UID: \"1964ea3b-7a10-404e-b765-29e2636d299b\") " pod="openshift-network-operator/iptables-alerter-lpbxn" Jul 15 21:01:38.202231 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.201853 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3edce062-4a75-40dc-80c3-a85ee51aee76-system-cni-dir\") pod \"multus-5vj8l\" (UID: \"3edce062-4a75-40dc-80c3-a85ee51aee76\") " pod="openshift-multus/multus-5vj8l" Jul 15 21:01:38.202231 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.201865 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fx5tg\" (UniqueName: \"kubernetes.io/projected/0dafa58d-15c7-46c9-81bd-c3c86bbf8e42-kube-api-access-fx5tg\") pod \"network-metrics-daemon-r59ft\" (UID: \"0dafa58d-15c7-46c9-81bd-c3c86bbf8e42\") " pod="openshift-multus/network-metrics-daemon-r59ft" Jul 15 21:01:38.202231 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.201899 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0dafa58d-15c7-46c9-81bd-c3c86bbf8e42-metrics-certs\") pod \"network-metrics-daemon-r59ft\" (UID: \"0dafa58d-15c7-46c9-81bd-c3c86bbf8e42\") " pod="openshift-multus/network-metrics-daemon-r59ft" Jul 15 21:01:38.202231 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.201921 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1aac0a47-4b65-48e2-8fca-03b686fe5512-etc-kubernetes\") pod \"tuned-l78d9\" (UID: \"1aac0a47-4b65-48e2-8fca-03b686fe5512\") " pod="openshift-cluster-node-tuning-operator/tuned-l78d9" Jul 15 21:01:38.202231 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.201947 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cctrg\" (UniqueName: \"kubernetes.io/projected/1aac0a47-4b65-48e2-8fca-03b686fe5512-kube-api-access-cctrg\") pod \"tuned-l78d9\" (UID: \"1aac0a47-4b65-48e2-8fca-03b686fe5512\") " pod="openshift-cluster-node-tuning-operator/tuned-l78d9" Jul 15 21:01:38.202231 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.201976 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xf7jq\" (UniqueName: \"kubernetes.io/projected/1964ea3b-7a10-404e-b765-29e2636d299b-kube-api-access-xf7jq\") pod \"iptables-alerter-lpbxn\" (UID: \"1964ea3b-7a10-404e-b765-29e2636d299b\") " pod="openshift-network-operator/iptables-alerter-lpbxn" Jul 15 21:01:38.202231 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.202000 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1aac0a47-4b65-48e2-8fca-03b686fe5512-host\") pod \"tuned-l78d9\" (UID: \"1aac0a47-4b65-48e2-8fca-03b686fe5512\") " pod="openshift-cluster-node-tuning-operator/tuned-l78d9" Jul 15 21:01:38.202231 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.202027 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3edce062-4a75-40dc-80c3-a85ee51aee76-multus-daemon-config\") pod \"multus-5vj8l\" (UID: \"3edce062-4a75-40dc-80c3-a85ee51aee76\") " pod="openshift-multus/multus-5vj8l" Jul 15 21:01:38.202753 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.202056 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3f26570b-5abd-4460-ba74-e59ba65032a1-original-pull-secret\") pod \"global-pull-secret-syncer-4ctrp\" (UID: \"3f26570b-5abd-4460-ba74-e59ba65032a1\") " pod="kube-system/global-pull-secret-syncer-4ctrp" Jul 15 21:01:38.202753 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.202152 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/47738f2f-70c7-45f4-b2f3-6cd99846e503-kubelet-dir\") pod \"aws-ebs-csi-driver-node-72wpg\" (UID: \"47738f2f-70c7-45f4-b2f3-6cd99846e503\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-72wpg" Jul 15 21:01:38.202753 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.202179 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1964ea3b-7a10-404e-b765-29e2636d299b-host-slash\") pod \"iptables-alerter-lpbxn\" (UID: \"1964ea3b-7a10-404e-b765-29e2636d299b\") " pod="openshift-network-operator/iptables-alerter-lpbxn" Jul 15 21:01:38.202753 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.202216 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1aac0a47-4b65-48e2-8fca-03b686fe5512-etc-sysctl-d\") pod \"tuned-l78d9\" (UID: \"1aac0a47-4b65-48e2-8fca-03b686fe5512\") " pod="openshift-cluster-node-tuning-operator/tuned-l78d9" Jul 15 21:01:38.202753 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.202247 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3edce062-4a75-40dc-80c3-a85ee51aee76-multus-conf-dir\") pod \"multus-5vj8l\" (UID: \"3edce062-4a75-40dc-80c3-a85ee51aee76\") " pod="openshift-multus/multus-5vj8l" Jul 15 21:01:38.202753 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.202272 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f7dba3ba-aec0-4137-b84e-4ce93d09c5a0-agent-certs\") pod \"konnectivity-agent-g65nf\" (UID: \"f7dba3ba-aec0-4137-b84e-4ce93d09c5a0\") " pod="kube-system/konnectivity-agent-g65nf" Jul 15 21:01:38.202753 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.202297 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1aac0a47-4b65-48e2-8fca-03b686fe5512-etc-systemd\") pod \"tuned-l78d9\" (UID: \"1aac0a47-4b65-48e2-8fca-03b686fe5512\") " pod="openshift-cluster-node-tuning-operator/tuned-l78d9" Jul 15 21:01:38.202753 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.202309 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3edce062-4a75-40dc-80c3-a85ee51aee76-multus-conf-dir\") pod \"multus-5vj8l\" (UID: \"3edce062-4a75-40dc-80c3-a85ee51aee76\") " pod="openshift-multus/multus-5vj8l" Jul 15 21:01:38.202753 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.202365 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1aac0a47-4b65-48e2-8fca-03b686fe5512-etc-sysconfig\") pod \"tuned-l78d9\" (UID: \"1aac0a47-4b65-48e2-8fca-03b686fe5512\") " pod="openshift-cluster-node-tuning-operator/tuned-l78d9" Jul 15 21:01:38.202753 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.202407 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1aac0a47-4b65-48e2-8fca-03b686fe5512-etc-sysctl-conf\") pod \"tuned-l78d9\" (UID: \"1aac0a47-4b65-48e2-8fca-03b686fe5512\") " pod="openshift-cluster-node-tuning-operator/tuned-l78d9" Jul 15 21:01:38.202753 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.202435 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3f26570b-5abd-4460-ba74-e59ba65032a1-dbus\") pod \"global-pull-secret-syncer-4ctrp\" (UID: \"3f26570b-5abd-4460-ba74-e59ba65032a1\") " pod="kube-system/global-pull-secret-syncer-4ctrp" Jul 15 21:01:38.202753 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.202469 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/47738f2f-70c7-45f4-b2f3-6cd99846e503-etc-selinux\") pod \"aws-ebs-csi-driver-node-72wpg\" (UID: \"47738f2f-70c7-45f4-b2f3-6cd99846e503\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-72wpg" Jul 15 21:01:38.202753 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.202507 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3edce062-4a75-40dc-80c3-a85ee51aee76-multus-socket-dir-parent\") pod \"multus-5vj8l\" (UID: \"3edce062-4a75-40dc-80c3-a85ee51aee76\") " pod="openshift-multus/multus-5vj8l" Jul 15 21:01:38.202753 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.202534 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3edce062-4a75-40dc-80c3-a85ee51aee76-host-var-lib-cni-multus\") pod \"multus-5vj8l\" (UID: \"3edce062-4a75-40dc-80c3-a85ee51aee76\") " pod="openshift-multus/multus-5vj8l" Jul 15 21:01:38.203188 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.202561 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/47738f2f-70c7-45f4-b2f3-6cd99846e503-socket-dir\") pod \"aws-ebs-csi-driver-node-72wpg\" (UID: \"47738f2f-70c7-45f4-b2f3-6cd99846e503\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-72wpg" Jul 15 21:01:38.203188 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.202580 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3edce062-4a75-40dc-80c3-a85ee51aee76-multus-daemon-config\") pod \"multus-5vj8l\" (UID: \"3edce062-4a75-40dc-80c3-a85ee51aee76\") " pod="openshift-multus/multus-5vj8l" Jul 15 21:01:38.203188 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.202586 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/47738f2f-70c7-45f4-b2f3-6cd99846e503-device-dir\") pod \"aws-ebs-csi-driver-node-72wpg\" (UID: \"47738f2f-70c7-45f4-b2f3-6cd99846e503\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-72wpg" Jul 15 21:01:38.203188 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.202613 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3edce062-4a75-40dc-80c3-a85ee51aee76-multus-socket-dir-parent\") pod \"multus-5vj8l\" (UID: \"3edce062-4a75-40dc-80c3-a85ee51aee76\") " pod="openshift-multus/multus-5vj8l" Jul 15 21:01:38.203188 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.202631 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3edce062-4a75-40dc-80c3-a85ee51aee76-host-run-netns\") pod \"multus-5vj8l\" (UID: \"3edce062-4a75-40dc-80c3-a85ee51aee76\") " pod="openshift-multus/multus-5vj8l" Jul 15 21:01:38.203188 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.202632 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3edce062-4a75-40dc-80c3-a85ee51aee76-host-var-lib-cni-multus\") pod \"multus-5vj8l\" (UID: \"3edce062-4a75-40dc-80c3-a85ee51aee76\") " pod="openshift-multus/multus-5vj8l" Jul 15 21:01:38.203188 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.202656 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/47738f2f-70c7-45f4-b2f3-6cd99846e503-sys-fs\") pod \"aws-ebs-csi-driver-node-72wpg\" (UID: \"47738f2f-70c7-45f4-b2f3-6cd99846e503\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-72wpg" Jul 15 21:01:38.203188 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.202675 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7qm7\" (UniqueName: \"kubernetes.io/projected/47738f2f-70c7-45f4-b2f3-6cd99846e503-kube-api-access-m7qm7\") pod \"aws-ebs-csi-driver-node-72wpg\" (UID: \"47738f2f-70c7-45f4-b2f3-6cd99846e503\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-72wpg" Jul 15 21:01:38.203188 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.202692 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1aac0a47-4b65-48e2-8fca-03b686fe5512-etc-modprobe-d\") pod \"tuned-l78d9\" (UID: \"1aac0a47-4b65-48e2-8fca-03b686fe5512\") " pod="openshift-cluster-node-tuning-operator/tuned-l78d9" Jul 15 21:01:38.203188 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.202719 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3edce062-4a75-40dc-80c3-a85ee51aee76-host-run-netns\") pod \"multus-5vj8l\" (UID: \"3edce062-4a75-40dc-80c3-a85ee51aee76\") " pod="openshift-multus/multus-5vj8l" Jul 15 21:01:38.203188 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.202731 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3f26570b-5abd-4460-ba74-e59ba65032a1-kubelet-config\") pod \"global-pull-secret-syncer-4ctrp\" (UID: \"3f26570b-5abd-4460-ba74-e59ba65032a1\") " pod="kube-system/global-pull-secret-syncer-4ctrp" Jul 15 21:01:38.203652 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.203636 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mstwf" Jul 15 21:01:38.206967 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:38.206888 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0db09c1e_d24d_4519_93c8_c185018c3d24.slice/crio-46ed4ed72e8ad19bf539890086fc08723c8e1b9e0c550b47a92a474a7efacf40 WatchSource:0}: Error finding container 46ed4ed72e8ad19bf539890086fc08723c8e1b9e0c550b47a92a474a7efacf40: Status 404 returned error can't find the container with id 46ed4ed72e8ad19bf539890086fc08723c8e1b9e0c550b47a92a474a7efacf40 Jul 15 21:01:38.212990 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:38.212968 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod397ae254_d712_41e8_a481_3243dbb511a9.slice/crio-9929c2145b9138bcb1927ba76dc0d4c9d6ff1f824b447c57c3329f9cc1320dd1 WatchSource:0}: Error finding container 9929c2145b9138bcb1927ba76dc0d4c9d6ff1f824b447c57c3329f9cc1320dd1: Status 404 returned error can't find the container with id 9929c2145b9138bcb1927ba76dc0d4c9d6ff1f824b447c57c3329f9cc1320dd1 Jul 15 21:01:38.219160 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.219123 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxc9f\" (UniqueName: \"kubernetes.io/projected/3edce062-4a75-40dc-80c3-a85ee51aee76-kube-api-access-kxc9f\") pod \"multus-5vj8l\" (UID: \"3edce062-4a75-40dc-80c3-a85ee51aee76\") " pod="openshift-multus/multus-5vj8l" Jul 15 21:01:38.303475 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.303380 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1aac0a47-4b65-48e2-8fca-03b686fe5512-etc-sysconfig\") pod \"tuned-l78d9\" (UID: \"1aac0a47-4b65-48e2-8fca-03b686fe5512\") " pod="openshift-cluster-node-tuning-operator/tuned-l78d9" Jul 15 21:01:38.303475 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.303417 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1aac0a47-4b65-48e2-8fca-03b686fe5512-etc-sysctl-conf\") pod \"tuned-l78d9\" (UID: \"1aac0a47-4b65-48e2-8fca-03b686fe5512\") " pod="openshift-cluster-node-tuning-operator/tuned-l78d9" Jul 15 21:01:38.303475 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.303439 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3f26570b-5abd-4460-ba74-e59ba65032a1-dbus\") pod \"global-pull-secret-syncer-4ctrp\" (UID: \"3f26570b-5abd-4460-ba74-e59ba65032a1\") " pod="kube-system/global-pull-secret-syncer-4ctrp" Jul 15 21:01:38.303755 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.303483 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/47738f2f-70c7-45f4-b2f3-6cd99846e503-etc-selinux\") pod \"aws-ebs-csi-driver-node-72wpg\" (UID: \"47738f2f-70c7-45f4-b2f3-6cd99846e503\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-72wpg" Jul 15 21:01:38.303755 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.303529 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/47738f2f-70c7-45f4-b2f3-6cd99846e503-socket-dir\") pod \"aws-ebs-csi-driver-node-72wpg\" (UID: \"47738f2f-70c7-45f4-b2f3-6cd99846e503\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-72wpg" Jul 15 21:01:38.303755 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.303536 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1aac0a47-4b65-48e2-8fca-03b686fe5512-etc-sysconfig\") pod \"tuned-l78d9\" (UID: \"1aac0a47-4b65-48e2-8fca-03b686fe5512\") " pod="openshift-cluster-node-tuning-operator/tuned-l78d9" Jul 15 21:01:38.303755 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.303552 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3f26570b-5abd-4460-ba74-e59ba65032a1-dbus\") pod \"global-pull-secret-syncer-4ctrp\" (UID: \"3f26570b-5abd-4460-ba74-e59ba65032a1\") " pod="kube-system/global-pull-secret-syncer-4ctrp" Jul 15 21:01:38.303755 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.303555 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/47738f2f-70c7-45f4-b2f3-6cd99846e503-device-dir\") pod \"aws-ebs-csi-driver-node-72wpg\" (UID: \"47738f2f-70c7-45f4-b2f3-6cd99846e503\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-72wpg" Jul 15 21:01:38.303755 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.303590 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1aac0a47-4b65-48e2-8fca-03b686fe5512-etc-sysctl-conf\") pod \"tuned-l78d9\" (UID: \"1aac0a47-4b65-48e2-8fca-03b686fe5512\") " pod="openshift-cluster-node-tuning-operator/tuned-l78d9" Jul 15 21:01:38.303755 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.303599 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/47738f2f-70c7-45f4-b2f3-6cd99846e503-sys-fs\") pod \"aws-ebs-csi-driver-node-72wpg\" (UID: \"47738f2f-70c7-45f4-b2f3-6cd99846e503\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-72wpg" Jul 15 21:01:38.303755 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.303610 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/47738f2f-70c7-45f4-b2f3-6cd99846e503-device-dir\") pod \"aws-ebs-csi-driver-node-72wpg\" (UID: \"47738f2f-70c7-45f4-b2f3-6cd99846e503\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-72wpg" Jul 15 21:01:38.303755 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.303616 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/47738f2f-70c7-45f4-b2f3-6cd99846e503-etc-selinux\") pod \"aws-ebs-csi-driver-node-72wpg\" (UID: \"47738f2f-70c7-45f4-b2f3-6cd99846e503\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-72wpg" Jul 15 21:01:38.303755 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.303627 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7qm7\" (UniqueName: \"kubernetes.io/projected/47738f2f-70c7-45f4-b2f3-6cd99846e503-kube-api-access-m7qm7\") pod \"aws-ebs-csi-driver-node-72wpg\" (UID: \"47738f2f-70c7-45f4-b2f3-6cd99846e503\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-72wpg" Jul 15 21:01:38.303755 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.303653 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1aac0a47-4b65-48e2-8fca-03b686fe5512-etc-modprobe-d\") pod \"tuned-l78d9\" (UID: \"1aac0a47-4b65-48e2-8fca-03b686fe5512\") " pod="openshift-cluster-node-tuning-operator/tuned-l78d9" Jul 15 21:01:38.303755 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.303676 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/47738f2f-70c7-45f4-b2f3-6cd99846e503-sys-fs\") pod \"aws-ebs-csi-driver-node-72wpg\" (UID: \"47738f2f-70c7-45f4-b2f3-6cd99846e503\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-72wpg" Jul 15 21:01:38.303755 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.303680 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3f26570b-5abd-4460-ba74-e59ba65032a1-kubelet-config\") pod \"global-pull-secret-syncer-4ctrp\" (UID: \"3f26570b-5abd-4460-ba74-e59ba65032a1\") " pod="kube-system/global-pull-secret-syncer-4ctrp" Jul 15 21:01:38.303755 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.303666 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/47738f2f-70c7-45f4-b2f3-6cd99846e503-socket-dir\") pod \"aws-ebs-csi-driver-node-72wpg\" (UID: \"47738f2f-70c7-45f4-b2f3-6cd99846e503\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-72wpg" Jul 15 21:01:38.304410 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.303729 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3f26570b-5abd-4460-ba74-e59ba65032a1-kubelet-config\") pod \"global-pull-secret-syncer-4ctrp\" (UID: \"3f26570b-5abd-4460-ba74-e59ba65032a1\") " pod="kube-system/global-pull-secret-syncer-4ctrp" Jul 15 21:01:38.304410 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.303739 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/47738f2f-70c7-45f4-b2f3-6cd99846e503-registration-dir\") pod \"aws-ebs-csi-driver-node-72wpg\" (UID: \"47738f2f-70c7-45f4-b2f3-6cd99846e503\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-72wpg" Jul 15 21:01:38.304410 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.303743 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1aac0a47-4b65-48e2-8fca-03b686fe5512-etc-modprobe-d\") pod \"tuned-l78d9\" (UID: \"1aac0a47-4b65-48e2-8fca-03b686fe5512\") " pod="openshift-cluster-node-tuning-operator/tuned-l78d9" Jul 15 21:01:38.304410 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.303774 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1aac0a47-4b65-48e2-8fca-03b686fe5512-var-lib-kubelet\") pod \"tuned-l78d9\" (UID: \"1aac0a47-4b65-48e2-8fca-03b686fe5512\") " pod="openshift-cluster-node-tuning-operator/tuned-l78d9" Jul 15 21:01:38.304410 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.303779 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/47738f2f-70c7-45f4-b2f3-6cd99846e503-registration-dir\") pod \"aws-ebs-csi-driver-node-72wpg\" (UID: \"47738f2f-70c7-45f4-b2f3-6cd99846e503\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-72wpg" Jul 15 21:01:38.304410 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.303800 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1aac0a47-4b65-48e2-8fca-03b686fe5512-etc-tuned\") pod \"tuned-l78d9\" (UID: \"1aac0a47-4b65-48e2-8fca-03b686fe5512\") " pod="openshift-cluster-node-tuning-operator/tuned-l78d9" Jul 15 21:01:38.304410 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.303831 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f7dba3ba-aec0-4137-b84e-4ce93d09c5a0-konnectivity-ca\") pod \"konnectivity-agent-g65nf\" (UID: \"f7dba3ba-aec0-4137-b84e-4ce93d09c5a0\") " pod="kube-system/konnectivity-agent-g65nf" Jul 15 21:01:38.304410 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.303861 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1aac0a47-4b65-48e2-8fca-03b686fe5512-var-lib-kubelet\") pod \"tuned-l78d9\" (UID: \"1aac0a47-4b65-48e2-8fca-03b686fe5512\") " pod="openshift-cluster-node-tuning-operator/tuned-l78d9" Jul 15 21:01:38.304410 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.303933 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1aac0a47-4b65-48e2-8fca-03b686fe5512-lib-modules\") pod \"tuned-l78d9\" (UID: \"1aac0a47-4b65-48e2-8fca-03b686fe5512\") " pod="openshift-cluster-node-tuning-operator/tuned-l78d9" Jul 15 21:01:38.304410 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.303964 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1aac0a47-4b65-48e2-8fca-03b686fe5512-run\") pod \"tuned-l78d9\" (UID: \"1aac0a47-4b65-48e2-8fca-03b686fe5512\") " pod="openshift-cluster-node-tuning-operator/tuned-l78d9" Jul 15 21:01:38.304410 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.303979 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1aac0a47-4b65-48e2-8fca-03b686fe5512-sys\") pod \"tuned-l78d9\" (UID: \"1aac0a47-4b65-48e2-8fca-03b686fe5512\") " pod="openshift-cluster-node-tuning-operator/tuned-l78d9" Jul 15 21:01:38.304410 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.303994 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1aac0a47-4b65-48e2-8fca-03b686fe5512-tmp\") pod \"tuned-l78d9\" (UID: \"1aac0a47-4b65-48e2-8fca-03b686fe5512\") " pod="openshift-cluster-node-tuning-operator/tuned-l78d9" Jul 15 21:01:38.304410 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.304019 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1964ea3b-7a10-404e-b765-29e2636d299b-iptables-alerter-script\") pod \"iptables-alerter-lpbxn\" (UID: \"1964ea3b-7a10-404e-b765-29e2636d299b\") " pod="openshift-network-operator/iptables-alerter-lpbxn" Jul 15 21:01:38.304410 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.304049 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fx5tg\" (UniqueName: \"kubernetes.io/projected/0dafa58d-15c7-46c9-81bd-c3c86bbf8e42-kube-api-access-fx5tg\") pod \"network-metrics-daemon-r59ft\" (UID: \"0dafa58d-15c7-46c9-81bd-c3c86bbf8e42\") " pod="openshift-multus/network-metrics-daemon-r59ft" Jul 15 21:01:38.305101 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.304058 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1aac0a47-4b65-48e2-8fca-03b686fe5512-run\") pod \"tuned-l78d9\" (UID: \"1aac0a47-4b65-48e2-8fca-03b686fe5512\") " pod="openshift-cluster-node-tuning-operator/tuned-l78d9" Jul 15 21:01:38.305101 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.304098 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0dafa58d-15c7-46c9-81bd-c3c86bbf8e42-metrics-certs\") pod \"network-metrics-daemon-r59ft\" (UID: \"0dafa58d-15c7-46c9-81bd-c3c86bbf8e42\") " pod="openshift-multus/network-metrics-daemon-r59ft" Jul 15 21:01:38.305101 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.304123 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1aac0a47-4b65-48e2-8fca-03b686fe5512-etc-kubernetes\") pod \"tuned-l78d9\" (UID: \"1aac0a47-4b65-48e2-8fca-03b686fe5512\") " pod="openshift-cluster-node-tuning-operator/tuned-l78d9" Jul 15 21:01:38.305101 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.304153 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1aac0a47-4b65-48e2-8fca-03b686fe5512-lib-modules\") pod \"tuned-l78d9\" (UID: \"1aac0a47-4b65-48e2-8fca-03b686fe5512\") " pod="openshift-cluster-node-tuning-operator/tuned-l78d9" Jul 15 21:01:38.305101 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.304148 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cctrg\" (UniqueName: \"kubernetes.io/projected/1aac0a47-4b65-48e2-8fca-03b686fe5512-kube-api-access-cctrg\") pod \"tuned-l78d9\" (UID: \"1aac0a47-4b65-48e2-8fca-03b686fe5512\") " pod="openshift-cluster-node-tuning-operator/tuned-l78d9" Jul 15 21:01:38.305101 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.304207 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xf7jq\" (UniqueName: \"kubernetes.io/projected/1964ea3b-7a10-404e-b765-29e2636d299b-kube-api-access-xf7jq\") pod \"iptables-alerter-lpbxn\" (UID: \"1964ea3b-7a10-404e-b765-29e2636d299b\") " pod="openshift-network-operator/iptables-alerter-lpbxn" Jul 15 21:01:38.305101 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.304234 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1aac0a47-4b65-48e2-8fca-03b686fe5512-host\") pod \"tuned-l78d9\" (UID: \"1aac0a47-4b65-48e2-8fca-03b686fe5512\") " pod="openshift-cluster-node-tuning-operator/tuned-l78d9" Jul 15 21:01:38.305101 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.304262 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3f26570b-5abd-4460-ba74-e59ba65032a1-original-pull-secret\") pod \"global-pull-secret-syncer-4ctrp\" (UID: \"3f26570b-5abd-4460-ba74-e59ba65032a1\") " pod="kube-system/global-pull-secret-syncer-4ctrp" Jul 15 21:01:38.305101 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.304289 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/47738f2f-70c7-45f4-b2f3-6cd99846e503-kubelet-dir\") pod \"aws-ebs-csi-driver-node-72wpg\" (UID: \"47738f2f-70c7-45f4-b2f3-6cd99846e503\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-72wpg" Jul 15 21:01:38.305101 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.304315 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1964ea3b-7a10-404e-b765-29e2636d299b-host-slash\") pod \"iptables-alerter-lpbxn\" (UID: \"1964ea3b-7a10-404e-b765-29e2636d299b\") " pod="openshift-network-operator/iptables-alerter-lpbxn" Jul 15 21:01:38.305101 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.304326 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1aac0a47-4b65-48e2-8fca-03b686fe5512-etc-kubernetes\") pod \"tuned-l78d9\" (UID: \"1aac0a47-4b65-48e2-8fca-03b686fe5512\") " pod="openshift-cluster-node-tuning-operator/tuned-l78d9" Jul 15 21:01:38.305101 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.304339 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1aac0a47-4b65-48e2-8fca-03b686fe5512-etc-sysctl-d\") pod \"tuned-l78d9\" (UID: \"1aac0a47-4b65-48e2-8fca-03b686fe5512\") " pod="openshift-cluster-node-tuning-operator/tuned-l78d9" Jul 15 21:01:38.305101 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.304370 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f7dba3ba-aec0-4137-b84e-4ce93d09c5a0-agent-certs\") pod \"konnectivity-agent-g65nf\" (UID: \"f7dba3ba-aec0-4137-b84e-4ce93d09c5a0\") " pod="kube-system/konnectivity-agent-g65nf" Jul 15 21:01:38.305101 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.304395 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1aac0a47-4b65-48e2-8fca-03b686fe5512-etc-systemd\") pod \"tuned-l78d9\" (UID: \"1aac0a47-4b65-48e2-8fca-03b686fe5512\") " pod="openshift-cluster-node-tuning-operator/tuned-l78d9" Jul 15 21:01:38.305842 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.304402 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/47738f2f-70c7-45f4-b2f3-6cd99846e503-kubelet-dir\") pod \"aws-ebs-csi-driver-node-72wpg\" (UID: \"47738f2f-70c7-45f4-b2f3-6cd99846e503\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-72wpg" Jul 15 21:01:38.305842 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:38.304407 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 21:01:38.305842 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:38.304472 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 21:01:38.305842 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.304467 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f7dba3ba-aec0-4137-b84e-4ce93d09c5a0-konnectivity-ca\") pod \"konnectivity-agent-g65nf\" (UID: \"f7dba3ba-aec0-4137-b84e-4ce93d09c5a0\") " pod="kube-system/konnectivity-agent-g65nf" Jul 15 21:01:38.305842 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:38.304499 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f26570b-5abd-4460-ba74-e59ba65032a1-original-pull-secret podName:3f26570b-5abd-4460-ba74-e59ba65032a1 nodeName:}" failed. No retries permitted until 2026-07-15 21:01:38.804478632 +0000 UTC m=+3.345791161 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3f26570b-5abd-4460-ba74-e59ba65032a1-original-pull-secret") pod "global-pull-secret-syncer-4ctrp" (UID: "3f26570b-5abd-4460-ba74-e59ba65032a1") : object "kube-system"/"original-pull-secret" not registered Jul 15 21:01:38.305842 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.304522 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1aac0a47-4b65-48e2-8fca-03b686fe5512-sys\") pod \"tuned-l78d9\" (UID: \"1aac0a47-4b65-48e2-8fca-03b686fe5512\") " pod="openshift-cluster-node-tuning-operator/tuned-l78d9" Jul 15 21:01:38.305842 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:38.304546 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0dafa58d-15c7-46c9-81bd-c3c86bbf8e42-metrics-certs podName:0dafa58d-15c7-46c9-81bd-c3c86bbf8e42 nodeName:}" failed. No retries permitted until 2026-07-15 21:01:38.804529875 +0000 UTC m=+3.345842422 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0dafa58d-15c7-46c9-81bd-c3c86bbf8e42-metrics-certs") pod "network-metrics-daemon-r59ft" (UID: "0dafa58d-15c7-46c9-81bd-c3c86bbf8e42") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 21:01:38.305842 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.304595 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1aac0a47-4b65-48e2-8fca-03b686fe5512-host\") pod \"tuned-l78d9\" (UID: \"1aac0a47-4b65-48e2-8fca-03b686fe5512\") " pod="openshift-cluster-node-tuning-operator/tuned-l78d9" Jul 15 21:01:38.305842 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.304598 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1964ea3b-7a10-404e-b765-29e2636d299b-host-slash\") pod \"iptables-alerter-lpbxn\" (UID: \"1964ea3b-7a10-404e-b765-29e2636d299b\") " pod="openshift-network-operator/iptables-alerter-lpbxn" Jul 15 21:01:38.305842 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.304637 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1aac0a47-4b65-48e2-8fca-03b686fe5512-etc-systemd\") pod \"tuned-l78d9\" (UID: \"1aac0a47-4b65-48e2-8fca-03b686fe5512\") " pod="openshift-cluster-node-tuning-operator/tuned-l78d9" Jul 15 21:01:38.305842 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.304678 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1964ea3b-7a10-404e-b765-29e2636d299b-iptables-alerter-script\") pod \"iptables-alerter-lpbxn\" (UID: \"1964ea3b-7a10-404e-b765-29e2636d299b\") " pod="openshift-network-operator/iptables-alerter-lpbxn" Jul 15 21:01:38.305842 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.304725 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1aac0a47-4b65-48e2-8fca-03b686fe5512-etc-sysctl-d\") pod \"tuned-l78d9\" (UID: \"1aac0a47-4b65-48e2-8fca-03b686fe5512\") " pod="openshift-cluster-node-tuning-operator/tuned-l78d9" Jul 15 21:01:38.306481 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.306300 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1aac0a47-4b65-48e2-8fca-03b686fe5512-etc-tuned\") pod \"tuned-l78d9\" (UID: \"1aac0a47-4b65-48e2-8fca-03b686fe5512\") " pod="openshift-cluster-node-tuning-operator/tuned-l78d9" Jul 15 21:01:38.306680 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.306654 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1aac0a47-4b65-48e2-8fca-03b686fe5512-tmp\") pod \"tuned-l78d9\" (UID: \"1aac0a47-4b65-48e2-8fca-03b686fe5512\") " pod="openshift-cluster-node-tuning-operator/tuned-l78d9" Jul 15 21:01:38.306962 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.306941 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f7dba3ba-aec0-4137-b84e-4ce93d09c5a0-agent-certs\") pod \"konnectivity-agent-g65nf\" (UID: \"f7dba3ba-aec0-4137-b84e-4ce93d09c5a0\") " pod="kube-system/konnectivity-agent-g65nf" Jul 15 21:01:38.330293 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.330261 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fx5tg\" (UniqueName: \"kubernetes.io/projected/0dafa58d-15c7-46c9-81bd-c3c86bbf8e42-kube-api-access-fx5tg\") pod \"network-metrics-daemon-r59ft\" (UID: \"0dafa58d-15c7-46c9-81bd-c3c86bbf8e42\") " pod="openshift-multus/network-metrics-daemon-r59ft" Jul 15 21:01:38.335534 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.335513 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cctrg\" (UniqueName: \"kubernetes.io/projected/1aac0a47-4b65-48e2-8fca-03b686fe5512-kube-api-access-cctrg\") pod \"tuned-l78d9\" (UID: \"1aac0a47-4b65-48e2-8fca-03b686fe5512\") " pod="openshift-cluster-node-tuning-operator/tuned-l78d9" Jul 15 21:01:38.336172 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.336156 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xf7jq\" (UniqueName: \"kubernetes.io/projected/1964ea3b-7a10-404e-b765-29e2636d299b-kube-api-access-xf7jq\") pod \"iptables-alerter-lpbxn\" (UID: \"1964ea3b-7a10-404e-b765-29e2636d299b\") " pod="openshift-network-operator/iptables-alerter-lpbxn" Jul 15 21:01:38.341411 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.341385 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7qm7\" (UniqueName: \"kubernetes.io/projected/47738f2f-70c7-45f4-b2f3-6cd99846e503-kube-api-access-m7qm7\") pod \"aws-ebs-csi-driver-node-72wpg\" (UID: \"47738f2f-70c7-45f4-b2f3-6cd99846e503\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-72wpg" Jul 15 21:01:38.384177 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.384140 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jw6xs" Jul 15 21:01:38.396498 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:38.396458 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05324e48_e436_4e6a_9b35_5d0c661a82a4.slice/crio-09608f7c9475faec2056d023624840e481013b50f76d76f561fd0d7b62a32ae7 WatchSource:0}: Error finding container 09608f7c9475faec2056d023624840e481013b50f76d76f561fd0d7b62a32ae7: Status 404 returned error can't find the container with id 09608f7c9475faec2056d023624840e481013b50f76d76f561fd0d7b62a32ae7 Jul 15 21:01:38.412721 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.412702 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-w2rr5" Jul 15 21:01:38.421561 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.421536 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5vj8l" Jul 15 21:01:38.421789 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:38.421760 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode24afff0_0ef0_4463_92fc_848462106a1b.slice/crio-0e58c70cc49e24434b9db8a6df47aafbaaec1ce673af5bff6554da376f7735d7 WatchSource:0}: Error finding container 0e58c70cc49e24434b9db8a6df47aafbaaec1ce673af5bff6554da376f7735d7: Status 404 returned error can't find the container with id 0e58c70cc49e24434b9db8a6df47aafbaaec1ce673af5bff6554da376f7735d7 Jul 15 21:01:38.430019 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:38.429995 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3edce062_4a75_40dc_80c3_a85ee51aee76.slice/crio-d245179d744764c61a1e06c61ea83eb079a42970a199273d4c17d3cde7077e0c WatchSource:0}: Error finding container d245179d744764c61a1e06c61ea83eb079a42970a199273d4c17d3cde7077e0c: Status 404 returned error can't find the container with id d245179d744764c61a1e06c61ea83eb079a42970a199273d4c17d3cde7077e0c Jul 15 21:01:38.437671 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.437654 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lpbxn" Jul 15 21:01:38.444732 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.444712 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-g65nf" Jul 15 21:01:38.446266 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:38.446247 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1964ea3b_7a10_404e_b765_29e2636d299b.slice/crio-ad273621aa4caf0411983a06a91c8f52442b3e1a606770852d2c6a0df9a4ec54 WatchSource:0}: Error finding container ad273621aa4caf0411983a06a91c8f52442b3e1a606770852d2c6a0df9a4ec54: Status 404 returned error can't find the container with id ad273621aa4caf0411983a06a91c8f52442b3e1a606770852d2c6a0df9a4ec54 Jul 15 21:01:38.456841 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.456786 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-72wpg" Jul 15 21:01:38.461088 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.461032 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-l78d9" Jul 15 21:01:38.466596 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:38.466569 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47738f2f_70c7_45f4_b2f3_6cd99846e503.slice/crio-c3c0eb7650787cc6f308d9d486a20f2e32819211584d7f6233b50f9009a720ad WatchSource:0}: Error finding container c3c0eb7650787cc6f308d9d486a20f2e32819211584d7f6233b50f9009a720ad: Status 404 returned error can't find the container with id c3c0eb7650787cc6f308d9d486a20f2e32819211584d7f6233b50f9009a720ad Jul 15 21:01:38.470903 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:01:38.470878 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1aac0a47_4b65_48e2_8fca_03b686fe5512.slice/crio-1c13f0f15e719448a9ee793899cd7c9ef5241a4f9b8965264c331208fe4b45b8 WatchSource:0}: Error finding container 1c13f0f15e719448a9ee793899cd7c9ef5241a4f9b8965264c331208fe4b45b8: Status 404 returned error can't find the container with id 1c13f0f15e719448a9ee793899cd7c9ef5241a4f9b8965264c331208fe4b45b8 Jul 15 21:01:38.489602 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.489560 2579 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 21:01:38.606717 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.606676 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fptjd\" (UniqueName: \"kubernetes.io/projected/eeaa6fcf-b320-4851-9bf9-583c963e07ba-kube-api-access-fptjd\") pod \"network-check-target-dg4dh\" (UID: \"eeaa6fcf-b320-4851-9bf9-583c963e07ba\") " pod="openshift-network-diagnostics/network-check-target-dg4dh" Jul 15 21:01:38.606915 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:38.606854 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 21:01:38.606915 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:38.606871 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 21:01:38.606915 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:38.606881 2579 projected.go:196] Error preparing data for projected volume kube-api-access-fptjd for pod openshift-network-diagnostics/network-check-target-dg4dh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 21:01:38.607081 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:38.606938 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eeaa6fcf-b320-4851-9bf9-583c963e07ba-kube-api-access-fptjd podName:eeaa6fcf-b320-4851-9bf9-583c963e07ba nodeName:}" failed. No retries permitted until 2026-07-15 21:01:39.606922527 +0000 UTC m=+4.148235060 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-fptjd" (UniqueName: "kubernetes.io/projected/eeaa6fcf-b320-4851-9bf9-583c963e07ba-kube-api-access-fptjd") pod "network-check-target-dg4dh" (UID: "eeaa6fcf-b320-4851-9bf9-583c963e07ba") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 21:01:38.809165 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.809123 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0dafa58d-15c7-46c9-81bd-c3c86bbf8e42-metrics-certs\") pod \"network-metrics-daemon-r59ft\" (UID: \"0dafa58d-15c7-46c9-81bd-c3c86bbf8e42\") " pod="openshift-multus/network-metrics-daemon-r59ft" Jul 15 21:01:38.809368 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:38.809186 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3f26570b-5abd-4460-ba74-e59ba65032a1-original-pull-secret\") pod \"global-pull-secret-syncer-4ctrp\" (UID: \"3f26570b-5abd-4460-ba74-e59ba65032a1\") " pod="kube-system/global-pull-secret-syncer-4ctrp" Jul 15 21:01:38.809368 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:38.809329 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 21:01:38.809477 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:38.809397 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0dafa58d-15c7-46c9-81bd-c3c86bbf8e42-metrics-certs podName:0dafa58d-15c7-46c9-81bd-c3c86bbf8e42 nodeName:}" failed. No retries permitted until 2026-07-15 21:01:39.80937612 +0000 UTC m=+4.350688668 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0dafa58d-15c7-46c9-81bd-c3c86bbf8e42-metrics-certs") pod "network-metrics-daemon-r59ft" (UID: "0dafa58d-15c7-46c9-81bd-c3c86bbf8e42") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 21:01:38.809477 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:38.809329 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 21:01:38.809588 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:38.809487 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f26570b-5abd-4460-ba74-e59ba65032a1-original-pull-secret podName:3f26570b-5abd-4460-ba74-e59ba65032a1 nodeName:}" failed. No retries permitted until 2026-07-15 21:01:39.809471564 +0000 UTC m=+4.350784095 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3f26570b-5abd-4460-ba74-e59ba65032a1-original-pull-secret") pod "global-pull-secret-syncer-4ctrp" (UID: "3f26570b-5abd-4460-ba74-e59ba65032a1") : object "kube-system"/"original-pull-secret" not registered Jul 15 21:01:39.086111 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:39.085899 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 20:56:38 +0000 UTC" deadline="2028-01-21 05:52:26.097808286 +0000 UTC" Jul 15 21:01:39.086111 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:39.085969 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13304h50m47.01184334s" Jul 15 21:01:39.095292 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:39.095250 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-g65nf" event={"ID":"f7dba3ba-aec0-4137-b84e-4ce93d09c5a0","Type":"ContainerStarted","Data":"ec47e22e2098675d6461916c9a7481056c1cf502d7ea1d8996d2864754fbd856"} Jul 15 21:01:39.108735 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:39.108651 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lpbxn" event={"ID":"1964ea3b-7a10-404e-b765-29e2636d299b","Type":"ContainerStarted","Data":"ad273621aa4caf0411983a06a91c8f52442b3e1a606770852d2c6a0df9a4ec54"} Jul 15 21:01:39.111728 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:39.111691 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5vj8l" event={"ID":"3edce062-4a75-40dc-80c3-a85ee51aee76","Type":"ContainerStarted","Data":"d245179d744764c61a1e06c61ea83eb079a42970a199273d4c17d3cde7077e0c"} Jul 15 21:01:39.113610 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:39.113573 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jw6xs" event={"ID":"05324e48-e436-4e6a-9b35-5d0c661a82a4","Type":"ContainerStarted","Data":"09608f7c9475faec2056d023624840e481013b50f76d76f561fd0d7b62a32ae7"} Jul 15 21:01:39.119733 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:39.119702 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mstwf" event={"ID":"397ae254-d712-41e8-a481-3243dbb511a9","Type":"ContainerStarted","Data":"9929c2145b9138bcb1927ba76dc0d4c9d6ff1f824b447c57c3329f9cc1320dd1"} Jul 15 21:01:39.121537 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:39.121509 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-185.ec2.internal" event={"ID":"b338db47c4937c088ae739177bb426e7","Type":"ContainerStarted","Data":"9c62b8f917e3ae36b22622172732c07db9e4319f71103e762e87791a0dfb7f08"} Jul 15 21:01:39.125118 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:39.125092 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-l78d9" event={"ID":"1aac0a47-4b65-48e2-8fca-03b686fe5512","Type":"ContainerStarted","Data":"1c13f0f15e719448a9ee793899cd7c9ef5241a4f9b8965264c331208fe4b45b8"} Jul 15 21:01:39.127435 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:39.127409 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-w2rr5" event={"ID":"e24afff0-0ef0-4463-92fc-848462106a1b","Type":"ContainerStarted","Data":"0e58c70cc49e24434b9db8a6df47aafbaaec1ce673af5bff6554da376f7735d7"} Jul 15 21:01:39.129009 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:39.128906 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" event={"ID":"0db09c1e-d24d-4519-93c8-c185018c3d24","Type":"ContainerStarted","Data":"46ed4ed72e8ad19bf539890086fc08723c8e1b9e0c550b47a92a474a7efacf40"} Jul 15 21:01:39.130534 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:39.130480 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-72wpg" event={"ID":"47738f2f-70c7-45f4-b2f3-6cd99846e503","Type":"ContainerStarted","Data":"c3c0eb7650787cc6f308d9d486a20f2e32819211584d7f6233b50f9009a720ad"} Jul 15 21:01:39.619405 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:39.619365 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fptjd\" (UniqueName: \"kubernetes.io/projected/eeaa6fcf-b320-4851-9bf9-583c963e07ba-kube-api-access-fptjd\") pod \"network-check-target-dg4dh\" (UID: \"eeaa6fcf-b320-4851-9bf9-583c963e07ba\") " pod="openshift-network-diagnostics/network-check-target-dg4dh" Jul 15 21:01:39.619613 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:39.619575 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 21:01:39.619613 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:39.619605 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 21:01:39.619724 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:39.619621 2579 projected.go:196] Error preparing data for projected volume kube-api-access-fptjd for pod openshift-network-diagnostics/network-check-target-dg4dh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 21:01:39.619724 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:39.619685 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eeaa6fcf-b320-4851-9bf9-583c963e07ba-kube-api-access-fptjd podName:eeaa6fcf-b320-4851-9bf9-583c963e07ba nodeName:}" failed. No retries permitted until 2026-07-15 21:01:41.619663608 +0000 UTC m=+6.160976154 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-fptjd" (UniqueName: "kubernetes.io/projected/eeaa6fcf-b320-4851-9bf9-583c963e07ba-kube-api-access-fptjd") pod "network-check-target-dg4dh" (UID: "eeaa6fcf-b320-4851-9bf9-583c963e07ba") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 21:01:39.820482 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:39.820440 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0dafa58d-15c7-46c9-81bd-c3c86bbf8e42-metrics-certs\") pod \"network-metrics-daemon-r59ft\" (UID: \"0dafa58d-15c7-46c9-81bd-c3c86bbf8e42\") " pod="openshift-multus/network-metrics-daemon-r59ft" Jul 15 21:01:39.820662 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:39.820511 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3f26570b-5abd-4460-ba74-e59ba65032a1-original-pull-secret\") pod \"global-pull-secret-syncer-4ctrp\" (UID: \"3f26570b-5abd-4460-ba74-e59ba65032a1\") " pod="kube-system/global-pull-secret-syncer-4ctrp" Jul 15 21:01:39.820799 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:39.820685 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 21:01:39.820799 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:39.820748 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f26570b-5abd-4460-ba74-e59ba65032a1-original-pull-secret podName:3f26570b-5abd-4460-ba74-e59ba65032a1 nodeName:}" failed. No retries permitted until 2026-07-15 21:01:41.82073033 +0000 UTC m=+6.362042864 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3f26570b-5abd-4460-ba74-e59ba65032a1-original-pull-secret") pod "global-pull-secret-syncer-4ctrp" (UID: "3f26570b-5abd-4460-ba74-e59ba65032a1") : object "kube-system"/"original-pull-secret" not registered Jul 15 21:01:39.820949 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:39.820871 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 21:01:39.821008 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:39.820948 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0dafa58d-15c7-46c9-81bd-c3c86bbf8e42-metrics-certs podName:0dafa58d-15c7-46c9-81bd-c3c86bbf8e42 nodeName:}" failed. No retries permitted until 2026-07-15 21:01:41.820929213 +0000 UTC m=+6.362241747 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0dafa58d-15c7-46c9-81bd-c3c86bbf8e42-metrics-certs") pod "network-metrics-daemon-r59ft" (UID: "0dafa58d-15c7-46c9-81bd-c3c86bbf8e42") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 21:01:39.991957 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:39.991922 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r59ft" Jul 15 21:01:39.992125 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:39.992078 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r59ft" podUID="0dafa58d-15c7-46c9-81bd-c3c86bbf8e42" Jul 15 21:01:39.992221 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:39.992132 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4ctrp" Jul 15 21:01:39.992293 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:39.992268 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4ctrp" podUID="3f26570b-5abd-4460-ba74-e59ba65032a1" Jul 15 21:01:39.993175 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:39.993009 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dg4dh" Jul 15 21:01:39.993175 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:39.993124 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dg4dh" podUID="eeaa6fcf-b320-4851-9bf9-583c963e07ba" Jul 15 21:01:40.090317 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:40.090258 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 20:56:38 +0000 UTC" deadline="2028-02-10 05:36:24.447604007 +0000 UTC" Jul 15 21:01:40.090317 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:40.090300 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13784h34m44.357308442s" Jul 15 21:01:41.148688 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:41.148521 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal" event={"ID":"29da48fefeff7a1251df734706134371","Type":"ContainerStarted","Data":"fee3b09797bea3791ae86f85fd81e06d8e8a518e2ce17e1ab7550df7e9d6f35d"} Jul 15 21:01:41.643566 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:41.643531 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fptjd\" (UniqueName: \"kubernetes.io/projected/eeaa6fcf-b320-4851-9bf9-583c963e07ba-kube-api-access-fptjd\") pod \"network-check-target-dg4dh\" (UID: \"eeaa6fcf-b320-4851-9bf9-583c963e07ba\") " pod="openshift-network-diagnostics/network-check-target-dg4dh" Jul 15 21:01:41.643775 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:41.643722 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 21:01:41.643775 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:41.643744 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 21:01:41.643775 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:41.643758 2579 projected.go:196] Error preparing data for projected volume kube-api-access-fptjd for pod openshift-network-diagnostics/network-check-target-dg4dh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 21:01:41.644006 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:41.643820 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eeaa6fcf-b320-4851-9bf9-583c963e07ba-kube-api-access-fptjd podName:eeaa6fcf-b320-4851-9bf9-583c963e07ba nodeName:}" failed. No retries permitted until 2026-07-15 21:01:45.643801596 +0000 UTC m=+10.185114128 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-fptjd" (UniqueName: "kubernetes.io/projected/eeaa6fcf-b320-4851-9bf9-583c963e07ba-kube-api-access-fptjd") pod "network-check-target-dg4dh" (UID: "eeaa6fcf-b320-4851-9bf9-583c963e07ba") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 21:01:41.845509 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:41.844788 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3f26570b-5abd-4460-ba74-e59ba65032a1-original-pull-secret\") pod \"global-pull-secret-syncer-4ctrp\" (UID: \"3f26570b-5abd-4460-ba74-e59ba65032a1\") " pod="kube-system/global-pull-secret-syncer-4ctrp" Jul 15 21:01:41.845509 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:41.844910 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0dafa58d-15c7-46c9-81bd-c3c86bbf8e42-metrics-certs\") pod \"network-metrics-daemon-r59ft\" (UID: \"0dafa58d-15c7-46c9-81bd-c3c86bbf8e42\") " pod="openshift-multus/network-metrics-daemon-r59ft" Jul 15 21:01:41.845509 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:41.845042 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 21:01:41.845509 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:41.845128 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0dafa58d-15c7-46c9-81bd-c3c86bbf8e42-metrics-certs podName:0dafa58d-15c7-46c9-81bd-c3c86bbf8e42 nodeName:}" failed. No retries permitted until 2026-07-15 21:01:45.845087272 +0000 UTC m=+10.386399808 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0dafa58d-15c7-46c9-81bd-c3c86bbf8e42-metrics-certs") pod "network-metrics-daemon-r59ft" (UID: "0dafa58d-15c7-46c9-81bd-c3c86bbf8e42") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 21:01:41.845509 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:41.845179 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 21:01:41.845509 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:41.845247 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f26570b-5abd-4460-ba74-e59ba65032a1-original-pull-secret podName:3f26570b-5abd-4460-ba74-e59ba65032a1 nodeName:}" failed. No retries permitted until 2026-07-15 21:01:45.84522741 +0000 UTC m=+10.386539940 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3f26570b-5abd-4460-ba74-e59ba65032a1-original-pull-secret") pod "global-pull-secret-syncer-4ctrp" (UID: "3f26570b-5abd-4460-ba74-e59ba65032a1") : object "kube-system"/"original-pull-secret" not registered Jul 15 21:01:41.992226 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:41.991101 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dg4dh" Jul 15 21:01:41.992226 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:41.991241 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dg4dh" podUID="eeaa6fcf-b320-4851-9bf9-583c963e07ba" Jul 15 21:01:41.992226 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:41.991688 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r59ft" Jul 15 21:01:41.992226 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:41.991813 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r59ft" podUID="0dafa58d-15c7-46c9-81bd-c3c86bbf8e42" Jul 15 21:01:41.992226 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:41.991867 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4ctrp" Jul 15 21:01:41.992226 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:41.991938 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4ctrp" podUID="3f26570b-5abd-4460-ba74-e59ba65032a1" Jul 15 21:01:43.996301 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:43.996267 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dg4dh" Jul 15 21:01:43.996803 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:43.996308 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r59ft" Jul 15 21:01:43.996803 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:43.996401 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dg4dh" podUID="eeaa6fcf-b320-4851-9bf9-583c963e07ba" Jul 15 21:01:43.996803 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:43.996481 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r59ft" podUID="0dafa58d-15c7-46c9-81bd-c3c86bbf8e42" Jul 15 21:01:43.996803 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:43.996519 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4ctrp" Jul 15 21:01:43.996803 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:43.996619 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4ctrp" podUID="3f26570b-5abd-4460-ba74-e59ba65032a1" Jul 15 21:01:45.679422 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:45.679337 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fptjd\" (UniqueName: \"kubernetes.io/projected/eeaa6fcf-b320-4851-9bf9-583c963e07ba-kube-api-access-fptjd\") pod \"network-check-target-dg4dh\" (UID: \"eeaa6fcf-b320-4851-9bf9-583c963e07ba\") " pod="openshift-network-diagnostics/network-check-target-dg4dh" Jul 15 21:01:45.679923 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:45.679509 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 21:01:45.679923 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:45.679537 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 21:01:45.679923 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:45.679552 2579 projected.go:196] Error preparing data for projected volume kube-api-access-fptjd for pod openshift-network-diagnostics/network-check-target-dg4dh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 21:01:45.679923 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:45.679619 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eeaa6fcf-b320-4851-9bf9-583c963e07ba-kube-api-access-fptjd podName:eeaa6fcf-b320-4851-9bf9-583c963e07ba nodeName:}" failed. No retries permitted until 2026-07-15 21:01:53.679598254 +0000 UTC m=+18.220910805 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-fptjd" (UniqueName: "kubernetes.io/projected/eeaa6fcf-b320-4851-9bf9-583c963e07ba-kube-api-access-fptjd") pod "network-check-target-dg4dh" (UID: "eeaa6fcf-b320-4851-9bf9-583c963e07ba") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 21:01:45.880784 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:45.880673 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0dafa58d-15c7-46c9-81bd-c3c86bbf8e42-metrics-certs\") pod \"network-metrics-daemon-r59ft\" (UID: \"0dafa58d-15c7-46c9-81bd-c3c86bbf8e42\") " pod="openshift-multus/network-metrics-daemon-r59ft" Jul 15 21:01:45.880784 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:45.880729 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3f26570b-5abd-4460-ba74-e59ba65032a1-original-pull-secret\") pod \"global-pull-secret-syncer-4ctrp\" (UID: \"3f26570b-5abd-4460-ba74-e59ba65032a1\") " pod="kube-system/global-pull-secret-syncer-4ctrp" Jul 15 21:01:45.881032 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:45.880867 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 21:01:45.881032 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:45.880869 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 21:01:45.881032 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:45.880924 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f26570b-5abd-4460-ba74-e59ba65032a1-original-pull-secret podName:3f26570b-5abd-4460-ba74-e59ba65032a1 nodeName:}" failed. No retries permitted until 2026-07-15 21:01:53.880910754 +0000 UTC m=+18.422223283 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3f26570b-5abd-4460-ba74-e59ba65032a1-original-pull-secret") pod "global-pull-secret-syncer-4ctrp" (UID: "3f26570b-5abd-4460-ba74-e59ba65032a1") : object "kube-system"/"original-pull-secret" not registered Jul 15 21:01:45.881032 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:45.880940 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0dafa58d-15c7-46c9-81bd-c3c86bbf8e42-metrics-certs podName:0dafa58d-15c7-46c9-81bd-c3c86bbf8e42 nodeName:}" failed. No retries permitted until 2026-07-15 21:01:53.880933145 +0000 UTC m=+18.422245675 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0dafa58d-15c7-46c9-81bd-c3c86bbf8e42-metrics-certs") pod "network-metrics-daemon-r59ft" (UID: "0dafa58d-15c7-46c9-81bd-c3c86bbf8e42") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 21:01:45.996403 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:45.996323 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r59ft" Jul 15 21:01:45.996561 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:45.996468 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r59ft" podUID="0dafa58d-15c7-46c9-81bd-c3c86bbf8e42" Jul 15 21:01:45.996561 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:45.996480 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dg4dh" Jul 15 21:01:45.996561 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:45.996508 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4ctrp" Jul 15 21:01:45.996716 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:45.996556 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dg4dh" podUID="eeaa6fcf-b320-4851-9bf9-583c963e07ba" Jul 15 21:01:45.996716 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:45.996639 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4ctrp" podUID="3f26570b-5abd-4460-ba74-e59ba65032a1" Jul 15 21:01:47.991404 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:47.991369 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4ctrp" Jul 15 21:01:47.991806 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:47.991490 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4ctrp" podUID="3f26570b-5abd-4460-ba74-e59ba65032a1" Jul 15 21:01:47.991806 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:47.991371 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dg4dh" Jul 15 21:01:47.991806 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:47.991529 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r59ft" Jul 15 21:01:47.991806 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:47.991599 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dg4dh" podUID="eeaa6fcf-b320-4851-9bf9-583c963e07ba" Jul 15 21:01:47.991806 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:47.991674 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r59ft" podUID="0dafa58d-15c7-46c9-81bd-c3c86bbf8e42" Jul 15 21:01:49.991995 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:49.991907 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dg4dh" Jul 15 21:01:49.992423 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:49.991908 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r59ft" Jul 15 21:01:49.992423 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:49.992040 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dg4dh" podUID="eeaa6fcf-b320-4851-9bf9-583c963e07ba" Jul 15 21:01:49.992423 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:49.992151 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r59ft" podUID="0dafa58d-15c7-46c9-81bd-c3c86bbf8e42" Jul 15 21:01:49.992423 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:49.991927 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4ctrp" Jul 15 21:01:49.992423 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:49.992248 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4ctrp" podUID="3f26570b-5abd-4460-ba74-e59ba65032a1" Jul 15 21:01:50.173230 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:50.173193 2579 generic.go:356] "Generic (PLEG): container finished" podID="29da48fefeff7a1251df734706134371" containerID="fee3b09797bea3791ae86f85fd81e06d8e8a518e2ce17e1ab7550df7e9d6f35d" exitCode=0 Jul 15 21:01:50.173400 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:50.173244 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal" event={"ID":"29da48fefeff7a1251df734706134371","Type":"ContainerDied","Data":"fee3b09797bea3791ae86f85fd81e06d8e8a518e2ce17e1ab7550df7e9d6f35d"} Jul 15 21:01:51.991203 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:51.991166 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r59ft" Jul 15 21:01:51.991642 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:51.991163 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dg4dh" Jul 15 21:01:51.991642 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:51.991303 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r59ft" podUID="0dafa58d-15c7-46c9-81bd-c3c86bbf8e42" Jul 15 21:01:51.991642 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:51.991184 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4ctrp" Jul 15 21:01:51.991642 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:51.991405 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dg4dh" podUID="eeaa6fcf-b320-4851-9bf9-583c963e07ba" Jul 15 21:01:51.991642 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:51.991498 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4ctrp" podUID="3f26570b-5abd-4460-ba74-e59ba65032a1" Jul 15 21:01:53.742032 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:53.741994 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fptjd\" (UniqueName: \"kubernetes.io/projected/eeaa6fcf-b320-4851-9bf9-583c963e07ba-kube-api-access-fptjd\") pod \"network-check-target-dg4dh\" (UID: \"eeaa6fcf-b320-4851-9bf9-583c963e07ba\") " pod="openshift-network-diagnostics/network-check-target-dg4dh" Jul 15 21:01:53.742529 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:53.742191 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 21:01:53.742529 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:53.742213 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 21:01:53.742529 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:53.742225 2579 projected.go:196] Error preparing data for projected volume kube-api-access-fptjd for pod openshift-network-diagnostics/network-check-target-dg4dh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 21:01:53.742529 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:53.742295 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eeaa6fcf-b320-4851-9bf9-583c963e07ba-kube-api-access-fptjd podName:eeaa6fcf-b320-4851-9bf9-583c963e07ba nodeName:}" failed. No retries permitted until 2026-07-15 21:02:09.7422792 +0000 UTC m=+34.283591739 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-fptjd" (UniqueName: "kubernetes.io/projected/eeaa6fcf-b320-4851-9bf9-583c963e07ba-kube-api-access-fptjd") pod "network-check-target-dg4dh" (UID: "eeaa6fcf-b320-4851-9bf9-583c963e07ba") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 21:01:53.943299 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:53.943259 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0dafa58d-15c7-46c9-81bd-c3c86bbf8e42-metrics-certs\") pod \"network-metrics-daemon-r59ft\" (UID: \"0dafa58d-15c7-46c9-81bd-c3c86bbf8e42\") " pod="openshift-multus/network-metrics-daemon-r59ft" Jul 15 21:01:53.943469 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:53.943311 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3f26570b-5abd-4460-ba74-e59ba65032a1-original-pull-secret\") pod \"global-pull-secret-syncer-4ctrp\" (UID: \"3f26570b-5abd-4460-ba74-e59ba65032a1\") " pod="kube-system/global-pull-secret-syncer-4ctrp" Jul 15 21:01:53.943469 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:53.943423 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 21:01:53.943573 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:53.943488 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0dafa58d-15c7-46c9-81bd-c3c86bbf8e42-metrics-certs podName:0dafa58d-15c7-46c9-81bd-c3c86bbf8e42 nodeName:}" failed. No retries permitted until 2026-07-15 21:02:09.943473524 +0000 UTC m=+34.484786053 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0dafa58d-15c7-46c9-81bd-c3c86bbf8e42-metrics-certs") pod "network-metrics-daemon-r59ft" (UID: "0dafa58d-15c7-46c9-81bd-c3c86bbf8e42") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 21:01:53.943573 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:53.943426 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 21:01:53.943573 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:53.943569 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f26570b-5abd-4460-ba74-e59ba65032a1-original-pull-secret podName:3f26570b-5abd-4460-ba74-e59ba65032a1 nodeName:}" failed. No retries permitted until 2026-07-15 21:02:09.943553268 +0000 UTC m=+34.484865801 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3f26570b-5abd-4460-ba74-e59ba65032a1-original-pull-secret") pod "global-pull-secret-syncer-4ctrp" (UID: "3f26570b-5abd-4460-ba74-e59ba65032a1") : object "kube-system"/"original-pull-secret" not registered Jul 15 21:01:53.992199 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:53.991702 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dg4dh" Jul 15 21:01:53.992199 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:53.991746 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r59ft" Jul 15 21:01:53.992199 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:53.991752 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4ctrp" Jul 15 21:01:53.992199 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:53.991829 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dg4dh" podUID="eeaa6fcf-b320-4851-9bf9-583c963e07ba" Jul 15 21:01:53.992199 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:53.991966 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r59ft" podUID="0dafa58d-15c7-46c9-81bd-c3c86bbf8e42" Jul 15 21:01:53.992199 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:53.992111 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4ctrp" podUID="3f26570b-5abd-4460-ba74-e59ba65032a1" Jul 15 21:01:55.992181 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:55.992149 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r59ft" Jul 15 21:01:55.992607 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:55.992258 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4ctrp" Jul 15 21:01:55.992607 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:55.992278 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r59ft" podUID="0dafa58d-15c7-46c9-81bd-c3c86bbf8e42" Jul 15 21:01:55.992607 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:55.992363 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4ctrp" podUID="3f26570b-5abd-4460-ba74-e59ba65032a1" Jul 15 21:01:55.992607 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:55.992388 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dg4dh" Jul 15 21:01:55.992607 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:55.992465 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dg4dh" podUID="eeaa6fcf-b320-4851-9bf9-583c963e07ba" Jul 15 21:01:57.991291 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:57.991251 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dg4dh" Jul 15 21:01:57.991291 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:57.991285 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r59ft" Jul 15 21:01:57.991784 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:57.991253 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4ctrp" Jul 15 21:01:57.991784 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:57.991388 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dg4dh" podUID="eeaa6fcf-b320-4851-9bf9-583c963e07ba" Jul 15 21:01:57.991784 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:57.991487 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r59ft" podUID="0dafa58d-15c7-46c9-81bd-c3c86bbf8e42" Jul 15 21:01:57.991784 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:57.991597 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4ctrp" podUID="3f26570b-5abd-4460-ba74-e59ba65032a1" Jul 15 21:01:59.193832 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:59.193586 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jw6xs" event={"ID":"05324e48-e436-4e6a-9b35-5d0c661a82a4","Type":"ContainerStarted","Data":"09839174c1912ad3dfa3542ed42a247185a875fa35ff56390f35061ef5106fe2"} Jul 15 21:01:59.195670 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:59.195639 2579 generic.go:356] "Generic (PLEG): container finished" podID="397ae254-d712-41e8-a481-3243dbb511a9" containerID="1e26e5f6ce565fbcdd35ace7d301cf6aaad6a7fcb85318f224adc0edf58a8613" exitCode=0 Jul 15 21:01:59.195804 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:59.195722 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mstwf" event={"ID":"397ae254-d712-41e8-a481-3243dbb511a9","Type":"ContainerDied","Data":"1e26e5f6ce565fbcdd35ace7d301cf6aaad6a7fcb85318f224adc0edf58a8613"} Jul 15 21:01:59.197337 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:59.197313 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-185.ec2.internal" event={"ID":"b338db47c4937c088ae739177bb426e7","Type":"ContainerStarted","Data":"2bb5b985f9314e21d71b35190c564fdd5745138f4d57d4760e36215c12ee98a7"} Jul 15 21:01:59.200225 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:59.200188 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal" event={"ID":"29da48fefeff7a1251df734706134371","Type":"ContainerStarted","Data":"265be0b76c327b8ee50a91bd41330450144bf42d7dc390bf059556946fbcde2c"} Jul 15 21:01:59.202346 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:59.202320 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-l78d9" event={"ID":"1aac0a47-4b65-48e2-8fca-03b686fe5512","Type":"ContainerStarted","Data":"e7bf563c2c409fe529cf7cb225576d38cbc6516ae0dc266c411f0afd50968fb4"} Jul 15 21:01:59.203717 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:59.203659 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-w2rr5" event={"ID":"e24afff0-0ef0-4463-92fc-848462106a1b","Type":"ContainerStarted","Data":"ddcab92ad5f85225be9a6ba60eed5b95e210d4aaa3830913602e2802a1683078"} Jul 15 21:01:59.204988 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:59.204959 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" event={"ID":"0db09c1e-d24d-4519-93c8-c185018c3d24","Type":"ContainerStarted","Data":"6660380d2453897faf1a93751eef157f25f912591d6324e56456ede50870dec1"} Jul 15 21:01:59.206534 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:59.206502 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-72wpg" event={"ID":"47738f2f-70c7-45f4-b2f3-6cd99846e503","Type":"ContainerStarted","Data":"afaf14ce878a3b1ada445e37197b0f786e4902d3fe4991a0a7e2e3bcfc38ad8f"} Jul 15 21:01:59.208034 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:59.207971 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-g65nf" event={"ID":"f7dba3ba-aec0-4137-b84e-4ce93d09c5a0","Type":"ContainerStarted","Data":"532924af10856346d192f8706f44a3b3a07e96387eb328d9d9c9e94589b65993"} Jul 15 21:01:59.209330 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:59.209311 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5vj8l" event={"ID":"3edce062-4a75-40dc-80c3-a85ee51aee76","Type":"ContainerStarted","Data":"becdd78b2401ef387aaf5cb0e673f11e6c20ef56f6a7a630e7ff04f3b1908a54"} Jul 15 21:01:59.211770 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:59.211730 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-jw6xs" podStartSLOduration=1.676055305 podStartE2EDuration="22.211719783s" totalImagesPullingTime="20.535664478s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 21:01:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:01:59.211583849 +0000 UTC m=+23.752896399" watchObservedRunningTime="2026-07-15 21:01:59.211719783 +0000 UTC m=+23.753032332" Jul 15 21:01:59.228972 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:59.228929 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-g65nf" podStartSLOduration=1.774714329 podStartE2EDuration="22.228913813s" totalImagesPullingTime="20.454199484s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:01:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:01:59.228659812 +0000 UTC m=+23.769972363" watchObservedRunningTime="2026-07-15 21:01:59.228913813 +0000 UTC m=+23.770226392" Jul 15 21:01:59.246244 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:59.246193 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal" podStartSLOduration=21.246177079 podStartE2EDuration="21.246177079s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 21:01:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:01:59.24566885 +0000 UTC m=+23.786981400" watchObservedRunningTime="2026-07-15 21:01:59.246177079 +0000 UTC m=+23.787489629" Jul 15 21:01:59.263144 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:59.263087 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-185.ec2.internal" podStartSLOduration=21.263048928 podStartE2EDuration="21.263048928s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 21:01:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:01:59.261821436 +0000 UTC m=+23.803133985" watchObservedRunningTime="2026-07-15 21:01:59.263048928 +0000 UTC m=+23.804361479" Jul 15 21:01:59.318340 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:59.318288 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-l78d9" podStartSLOduration=1.879004366 podStartE2EDuration="22.318270891s" totalImagesPullingTime="20.439266525s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 21:01:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:01:59.317655499 +0000 UTC m=+23.858968050" watchObservedRunningTime="2026-07-15 21:01:59.318270891 +0000 UTC m=+23.859583441" Jul 15 21:01:59.368826 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:59.368037 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-w2rr5" podStartSLOduration=1.875541456 podStartE2EDuration="22.368017714s" totalImagesPullingTime="20.492476258s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 21:01:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:01:59.33815046 +0000 UTC m=+23.879463010" watchObservedRunningTime="2026-07-15 21:01:59.368017714 +0000 UTC m=+23.909330266" Jul 15 21:01:59.368826 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:59.368417 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-5vj8l" podStartSLOduration=1.829357974 podStartE2EDuration="22.368406993s" totalImagesPullingTime="20.539049019s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 21:01:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:01:59.367718923 +0000 UTC m=+23.909031474" watchObservedRunningTime="2026-07-15 21:01:59.368406993 +0000 UTC m=+23.909719559" Jul 15 21:01:59.991155 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:59.991117 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4ctrp" Jul 15 21:01:59.991438 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:59.991117 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dg4dh" Jul 15 21:01:59.991438 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:01:59.991248 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r59ft" Jul 15 21:01:59.991438 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:59.991255 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4ctrp" podUID="3f26570b-5abd-4460-ba74-e59ba65032a1" Jul 15 21:01:59.991438 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:59.991337 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dg4dh" podUID="eeaa6fcf-b320-4851-9bf9-583c963e07ba" Jul 15 21:01:59.991605 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:01:59.991439 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r59ft" podUID="0dafa58d-15c7-46c9-81bd-c3c86bbf8e42" Jul 15 21:02:00.214393 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:00.214326 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m5r2m_0db09c1e-d24d-4519-93c8-c185018c3d24/ovn-acl-logging/0.log" Jul 15 21:02:00.215107 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:00.214666 2579 generic.go:356] "Generic (PLEG): container finished" podID="0db09c1e-d24d-4519-93c8-c185018c3d24" containerID="9f39254813d62efd1bf1990a6ce067f28412a0a7ce91b7e5ef7c6aeceb4a8908" exitCode=1 Jul 15 21:02:00.215107 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:00.214780 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" event={"ID":"0db09c1e-d24d-4519-93c8-c185018c3d24","Type":"ContainerStarted","Data":"7b5a7deffc25eb375b4aefffb47fa9260029dbc6bf2633538d4f5c40c2f84adc"} Jul 15 21:02:00.215107 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:00.214841 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" event={"ID":"0db09c1e-d24d-4519-93c8-c185018c3d24","Type":"ContainerStarted","Data":"a7f4ac4ce8369b9467a45f3ca19f55f7893b28567b9e79494a9a6ec8bdf5ffa7"} Jul 15 21:02:00.215107 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:00.214871 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" event={"ID":"0db09c1e-d24d-4519-93c8-c185018c3d24","Type":"ContainerStarted","Data":"7281cc7f154fb4d75c8ad2476882b4b029747b32d01fe4ff901abeee1ac0ff00"} Jul 15 21:02:00.215107 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:00.214900 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" event={"ID":"0db09c1e-d24d-4519-93c8-c185018c3d24","Type":"ContainerStarted","Data":"4a8a83793e8cdfcafa1669bad996892977f736fdafa07465c9ae755b622993b6"} Jul 15 21:02:00.215107 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:00.214928 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" event={"ID":"0db09c1e-d24d-4519-93c8-c185018c3d24","Type":"ContainerDied","Data":"9f39254813d62efd1bf1990a6ce067f28412a0a7ce91b7e5ef7c6aeceb4a8908"} Jul 15 21:02:00.504289 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:00.504215 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-g65nf" Jul 15 21:02:01.219187 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:01.219145 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lpbxn" event={"ID":"1964ea3b-7a10-404e-b765-29e2636d299b","Type":"ContainerStarted","Data":"eb80ce68f7b9c4bcd85dda3d4cd367d4b9cccc1a69ef94d948056fb5503c59d7"} Jul 15 21:02:01.238267 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:01.238219 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-lpbxn" podStartSLOduration=3.765131706 podStartE2EDuration="24.238204342s" totalImagesPullingTime="20.473072636s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 21:01:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:02:01.237991223 +0000 UTC m=+25.779303773" watchObservedRunningTime="2026-07-15 21:02:01.238204342 +0000 UTC m=+25.779516892" Jul 15 21:02:01.601205 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:01.601175 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-g65nf" Jul 15 21:02:01.601711 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:01.601688 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-g65nf" Jul 15 21:02:01.701219 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:01.701195 2579 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 15 21:02:01.991847 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:01.991716 2579 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-15T21:02:01.701213277Z","UUID":"bcea5fce-dfa2-43e3-a1a8-07dda7af5e62","Handler":null,"Name":"","Endpoint":""} Jul 15 21:02:01.991847 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:01.991845 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4ctrp" Jul 15 21:02:01.992221 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:01.991845 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dg4dh" Jul 15 21:02:01.992221 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:01.991961 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4ctrp" podUID="3f26570b-5abd-4460-ba74-e59ba65032a1" Jul 15 21:02:01.992221 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:01.991845 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r59ft" Jul 15 21:02:01.992221 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:01.992031 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dg4dh" podUID="eeaa6fcf-b320-4851-9bf9-583c963e07ba" Jul 15 21:02:01.992221 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:01.992125 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r59ft" podUID="0dafa58d-15c7-46c9-81bd-c3c86bbf8e42" Jul 15 21:02:01.993851 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:01.993819 2579 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 15 21:02:01.993983 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:01.993868 2579 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 15 21:02:02.223570 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:02.223532 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-72wpg" event={"ID":"47738f2f-70c7-45f4-b2f3-6cd99846e503","Type":"ContainerStarted","Data":"989b7fbcbd0d7833317e77718331c3a519593e67791836926ff27b9bf9147297"} Jul 15 21:02:02.227544 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:02.227371 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m5r2m_0db09c1e-d24d-4519-93c8-c185018c3d24/ovn-acl-logging/0.log" Jul 15 21:02:02.227932 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:02.227903 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" event={"ID":"0db09c1e-d24d-4519-93c8-c185018c3d24","Type":"ContainerStarted","Data":"9f370325f01ddaeb982c81a0014d291db8c5ffad06c53bed20713a359f738895"} Jul 15 21:02:02.229264 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:02.229245 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-g65nf" Jul 15 21:02:03.994334 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:03.994308 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r59ft" Jul 15 21:02:03.994334 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:03.994331 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4ctrp" Jul 15 21:02:03.994983 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:03.994409 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dg4dh" Jul 15 21:02:03.994983 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:03.994419 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4ctrp" podUID="3f26570b-5abd-4460-ba74-e59ba65032a1" Jul 15 21:02:03.994983 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:03.994494 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dg4dh" podUID="eeaa6fcf-b320-4851-9bf9-583c963e07ba" Jul 15 21:02:03.994983 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:03.994548 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r59ft" podUID="0dafa58d-15c7-46c9-81bd-c3c86bbf8e42" Jul 15 21:02:04.235242 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:04.235219 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m5r2m_0db09c1e-d24d-4519-93c8-c185018c3d24/ovn-acl-logging/0.log" Jul 15 21:02:04.235615 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:04.235595 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" event={"ID":"0db09c1e-d24d-4519-93c8-c185018c3d24","Type":"ContainerStarted","Data":"cb6cdca027ba4149fef9643cce7ef2ee190d5165bbdac1a4f66508c6710a842c"} Jul 15 21:02:04.235882 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:04.235854 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:02:04.236010 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:04.235936 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:02:04.236010 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:04.235988 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:02:04.236137 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:04.236095 2579 scope.go:117] "RemoveContainer" containerID="9f39254813d62efd1bf1990a6ce067f28412a0a7ce91b7e5ef7c6aeceb4a8908" Jul 15 21:02:04.237813 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:04.237777 2579 generic.go:356] "Generic (PLEG): container finished" podID="397ae254-d712-41e8-a481-3243dbb511a9" containerID="5432636bff4379e8fadd5fbb51723f6c79cf88f5263a4c68676c39bd64756df1" exitCode=0 Jul 15 21:02:04.237941 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:04.237873 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mstwf" event={"ID":"397ae254-d712-41e8-a481-3243dbb511a9","Type":"ContainerDied","Data":"5432636bff4379e8fadd5fbb51723f6c79cf88f5263a4c68676c39bd64756df1"} Jul 15 21:02:04.251985 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:04.251959 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:02:04.252117 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:04.252096 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:02:05.243504 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:05.243454 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-72wpg" event={"ID":"47738f2f-70c7-45f4-b2f3-6cd99846e503","Type":"ContainerStarted","Data":"8750d90e59e601f4c859c729292ca167600059584c557712e9d7e4e9224cc797"} Jul 15 21:02:05.247038 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:05.247018 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m5r2m_0db09c1e-d24d-4519-93c8-c185018c3d24/ovn-acl-logging/0.log" Jul 15 21:02:05.247350 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:05.247326 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" event={"ID":"0db09c1e-d24d-4519-93c8-c185018c3d24","Type":"ContainerStarted","Data":"1c501653396978acbeea7685cc98767779dca4537c3340394e7751dcb5a0fcc7"} Jul 15 21:02:05.295504 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:05.295456 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-72wpg" podStartSLOduration=2.764281411 podStartE2EDuration="28.295439808s" totalImagesPullingTime="25.531158397s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 21:01:37 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:02:05.293429248 +0000 UTC m=+29.834741799" watchObservedRunningTime="2026-07-15 21:02:05.295439808 +0000 UTC m=+29.836752357" Jul 15 21:02:05.993389 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:05.992833 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4ctrp" Jul 15 21:02:05.993389 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:05.992975 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4ctrp" podUID="3f26570b-5abd-4460-ba74-e59ba65032a1" Jul 15 21:02:05.993715 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:05.993495 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dg4dh" Jul 15 21:02:05.993715 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:05.993612 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dg4dh" podUID="eeaa6fcf-b320-4851-9bf9-583c963e07ba" Jul 15 21:02:05.993715 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:05.993682 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r59ft" Jul 15 21:02:05.993851 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:05.993794 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r59ft" podUID="0dafa58d-15c7-46c9-81bd-c3c86bbf8e42" Jul 15 21:02:06.206221 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:06.206160 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" podStartSLOduration=9.414509717 podStartE2EDuration="30.206141273s" totalImagesPullingTime="20.791631556s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 21:01:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:02:05.367710549 +0000 UTC m=+29.909023099" watchObservedRunningTime="2026-07-15 21:02:06.206141273 +0000 UTC m=+30.747453822" Jul 15 21:02:06.206813 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:06.206795 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4ctrp"] Jul 15 21:02:06.209271 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:06.209246 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-r59ft"] Jul 15 21:02:06.209897 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:06.209871 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-dg4dh"] Jul 15 21:02:06.251713 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:06.251641 2579 generic.go:356] "Generic (PLEG): container finished" podID="397ae254-d712-41e8-a481-3243dbb511a9" containerID="a4ecb288bd289aaa749ce6cc4a3ff1293a1d91c378cd264b39b762250726c072" exitCode=0 Jul 15 21:02:06.252096 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:06.251735 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mstwf" event={"ID":"397ae254-d712-41e8-a481-3243dbb511a9","Type":"ContainerDied","Data":"a4ecb288bd289aaa749ce6cc4a3ff1293a1d91c378cd264b39b762250726c072"} Jul 15 21:02:06.252096 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:06.251768 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4ctrp" Jul 15 21:02:06.252289 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:06.252271 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dg4dh" Jul 15 21:02:06.252424 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:06.252394 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4ctrp" podUID="3f26570b-5abd-4460-ba74-e59ba65032a1" Jul 15 21:02:06.252648 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:06.252480 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dg4dh" podUID="eeaa6fcf-b320-4851-9bf9-583c963e07ba" Jul 15 21:02:06.252648 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:06.252531 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r59ft" Jul 15 21:02:06.252648 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:06.252618 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r59ft" podUID="0dafa58d-15c7-46c9-81bd-c3c86bbf8e42" Jul 15 21:02:07.991569 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:07.991382 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dg4dh" Jul 15 21:02:07.991926 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:07.991382 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4ctrp" Jul 15 21:02:07.991926 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:07.991647 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dg4dh" podUID="eeaa6fcf-b320-4851-9bf9-583c963e07ba" Jul 15 21:02:07.991926 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:07.991753 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4ctrp" podUID="3f26570b-5abd-4460-ba74-e59ba65032a1" Jul 15 21:02:07.991926 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:07.991398 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r59ft" Jul 15 21:02:07.991926 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:07.991836 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r59ft" podUID="0dafa58d-15c7-46c9-81bd-c3c86bbf8e42" Jul 15 21:02:08.258383 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:08.258290 2579 generic.go:356] "Generic (PLEG): container finished" podID="397ae254-d712-41e8-a481-3243dbb511a9" containerID="4e5c2746bbb28a0868d45df9d2bb4f1afbfe794035f27d366c4872579b158f44" exitCode=0 Jul 15 21:02:08.258383 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:08.258347 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mstwf" event={"ID":"397ae254-d712-41e8-a481-3243dbb511a9","Type":"ContainerDied","Data":"4e5c2746bbb28a0868d45df9d2bb4f1afbfe794035f27d366c4872579b158f44"} Jul 15 21:02:09.759699 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:09.759662 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fptjd\" (UniqueName: \"kubernetes.io/projected/eeaa6fcf-b320-4851-9bf9-583c963e07ba-kube-api-access-fptjd\") pod \"network-check-target-dg4dh\" (UID: \"eeaa6fcf-b320-4851-9bf9-583c963e07ba\") " pod="openshift-network-diagnostics/network-check-target-dg4dh" Jul 15 21:02:09.760412 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:09.759816 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 21:02:09.760412 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:09.759835 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 21:02:09.760412 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:09.759844 2579 projected.go:196] Error preparing data for projected volume kube-api-access-fptjd for pod openshift-network-diagnostics/network-check-target-dg4dh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 21:02:09.760412 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:09.759898 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eeaa6fcf-b320-4851-9bf9-583c963e07ba-kube-api-access-fptjd podName:eeaa6fcf-b320-4851-9bf9-583c963e07ba nodeName:}" failed. No retries permitted until 2026-07-15 21:02:41.759880496 +0000 UTC m=+66.301193039 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-fptjd" (UniqueName: "kubernetes.io/projected/eeaa6fcf-b320-4851-9bf9-583c963e07ba-kube-api-access-fptjd") pod "network-check-target-dg4dh" (UID: "eeaa6fcf-b320-4851-9bf9-583c963e07ba") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 21:02:09.961247 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:09.961216 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0dafa58d-15c7-46c9-81bd-c3c86bbf8e42-metrics-certs\") pod \"network-metrics-daemon-r59ft\" (UID: \"0dafa58d-15c7-46c9-81bd-c3c86bbf8e42\") " pod="openshift-multus/network-metrics-daemon-r59ft" Jul 15 21:02:09.961429 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:09.961256 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3f26570b-5abd-4460-ba74-e59ba65032a1-original-pull-secret\") pod \"global-pull-secret-syncer-4ctrp\" (UID: \"3f26570b-5abd-4460-ba74-e59ba65032a1\") " pod="kube-system/global-pull-secret-syncer-4ctrp" Jul 15 21:02:09.961429 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:09.961363 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 21:02:09.961429 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:09.961365 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 21:02:09.961429 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:09.961430 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f26570b-5abd-4460-ba74-e59ba65032a1-original-pull-secret podName:3f26570b-5abd-4460-ba74-e59ba65032a1 nodeName:}" failed. No retries permitted until 2026-07-15 21:02:41.961412073 +0000 UTC m=+66.502724615 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3f26570b-5abd-4460-ba74-e59ba65032a1-original-pull-secret") pod "global-pull-secret-syncer-4ctrp" (UID: "3f26570b-5abd-4460-ba74-e59ba65032a1") : object "kube-system"/"original-pull-secret" not registered Jul 15 21:02:09.961609 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:09.961451 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0dafa58d-15c7-46c9-81bd-c3c86bbf8e42-metrics-certs podName:0dafa58d-15c7-46c9-81bd-c3c86bbf8e42 nodeName:}" failed. No retries permitted until 2026-07-15 21:02:41.961442054 +0000 UTC m=+66.502754587 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0dafa58d-15c7-46c9-81bd-c3c86bbf8e42-metrics-certs") pod "network-metrics-daemon-r59ft" (UID: "0dafa58d-15c7-46c9-81bd-c3c86bbf8e42") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 21:02:09.991515 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:09.991483 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4ctrp" Jul 15 21:02:09.991694 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:09.991500 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dg4dh" Jul 15 21:02:09.991694 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:09.991610 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4ctrp" podUID="3f26570b-5abd-4460-ba74-e59ba65032a1" Jul 15 21:02:09.991694 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:09.991664 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dg4dh" podUID="eeaa6fcf-b320-4851-9bf9-583c963e07ba" Jul 15 21:02:09.991839 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:09.991699 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r59ft" Jul 15 21:02:09.991839 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:09.991813 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r59ft" podUID="0dafa58d-15c7-46c9-81bd-c3c86bbf8e42" Jul 15 21:02:11.775140 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:11.775050 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeReady" Jul 15 21:02:11.775589 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:11.775235 2579 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 15 21:02:11.823663 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:11.823624 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7dc5664c6d-hplh7"] Jul 15 21:02:11.854934 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:11.854892 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-nhh6c"] Jul 15 21:02:11.855196 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:11.855130 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7dc5664c6d-hplh7" Jul 15 21:02:11.862082 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:11.862008 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 15 21:02:11.862082 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:11.862025 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 15 21:02:11.862429 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:11.862329 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-d59m9\"" Jul 15 21:02:11.862429 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:11.862342 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 15 21:02:11.870976 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:11.870952 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-9mhlt"] Jul 15 21:02:11.871253 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:11.871006 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 15 21:02:11.871322 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:11.871288 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nhh6c" Jul 15 21:02:11.874345 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:11.874321 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 15 21:02:11.874470 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:11.874327 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-p4nfw\"" Jul 15 21:02:11.875511 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:11.875491 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 15 21:02:11.887188 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:11.887166 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7dc5664c6d-hplh7"] Jul 15 21:02:11.887291 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:11.887204 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nhh6c"] Jul 15 21:02:11.887291 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:11.887229 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-9mhlt"] Jul 15 21:02:11.887392 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:11.887312 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-9mhlt" Jul 15 21:02:11.890564 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:11.890331 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 15 21:02:11.890564 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:11.890332 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-8p5p2\"" Jul 15 21:02:11.890564 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:11.890366 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 15 21:02:11.890564 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:11.890494 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 15 21:02:11.977705 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:11.977671 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d89392ea-c74a-4ead-b205-afdd71a00a42-registry-certificates\") pod \"image-registry-7dc5664c6d-hplh7\" (UID: \"d89392ea-c74a-4ead-b205-afdd71a00a42\") " pod="openshift-image-registry/image-registry-7dc5664c6d-hplh7" Jul 15 21:02:11.977896 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:11.977721 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d89392ea-c74a-4ead-b205-afdd71a00a42-bound-sa-token\") pod \"image-registry-7dc5664c6d-hplh7\" (UID: \"d89392ea-c74a-4ead-b205-afdd71a00a42\") " pod="openshift-image-registry/image-registry-7dc5664c6d-hplh7" Jul 15 21:02:11.977896 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:11.977745 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d89392ea-c74a-4ead-b205-afdd71a00a42-registry-tls\") pod \"image-registry-7dc5664c6d-hplh7\" (UID: \"d89392ea-c74a-4ead-b205-afdd71a00a42\") " pod="openshift-image-registry/image-registry-7dc5664c6d-hplh7" Jul 15 21:02:11.977896 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:11.977795 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2511b73d-3502-447a-9175-60d60bd9b862-metrics-tls\") pod \"dns-default-nhh6c\" (UID: \"2511b73d-3502-447a-9175-60d60bd9b862\") " pod="openshift-dns/dns-default-nhh6c" Jul 15 21:02:11.977896 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:11.977859 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d89392ea-c74a-4ead-b205-afdd71a00a42-trusted-ca\") pod \"image-registry-7dc5664c6d-hplh7\" (UID: \"d89392ea-c74a-4ead-b205-afdd71a00a42\") " pod="openshift-image-registry/image-registry-7dc5664c6d-hplh7" Jul 15 21:02:11.978146 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:11.977896 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d89392ea-c74a-4ead-b205-afdd71a00a42-installation-pull-secrets\") pod \"image-registry-7dc5664c6d-hplh7\" (UID: \"d89392ea-c74a-4ead-b205-afdd71a00a42\") " pod="openshift-image-registry/image-registry-7dc5664c6d-hplh7" Jul 15 21:02:11.978146 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:11.977932 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ktn5\" (UniqueName: \"kubernetes.io/projected/d89392ea-c74a-4ead-b205-afdd71a00a42-kube-api-access-4ktn5\") pod \"image-registry-7dc5664c6d-hplh7\" (UID: \"d89392ea-c74a-4ead-b205-afdd71a00a42\") " pod="openshift-image-registry/image-registry-7dc5664c6d-hplh7" Jul 15 21:02:11.978146 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:11.977965 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/44d76b49-7290-4217-b4b3-2ec82286aa5c-cert\") pod \"ingress-canary-9mhlt\" (UID: \"44d76b49-7290-4217-b4b3-2ec82286aa5c\") " pod="openshift-ingress-canary/ingress-canary-9mhlt" Jul 15 21:02:11.978146 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:11.978030 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d89392ea-c74a-4ead-b205-afdd71a00a42-image-registry-private-configuration\") pod \"image-registry-7dc5664c6d-hplh7\" (UID: \"d89392ea-c74a-4ead-b205-afdd71a00a42\") " pod="openshift-image-registry/image-registry-7dc5664c6d-hplh7" Jul 15 21:02:11.978146 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:11.978079 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d89392ea-c74a-4ead-b205-afdd71a00a42-ca-trust-extracted\") pod \"image-registry-7dc5664c6d-hplh7\" (UID: \"d89392ea-c74a-4ead-b205-afdd71a00a42\") " pod="openshift-image-registry/image-registry-7dc5664c6d-hplh7" Jul 15 21:02:11.978146 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:11.978118 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2511b73d-3502-447a-9175-60d60bd9b862-config-volume\") pod \"dns-default-nhh6c\" (UID: \"2511b73d-3502-447a-9175-60d60bd9b862\") " pod="openshift-dns/dns-default-nhh6c" Jul 15 21:02:11.978146 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:11.978138 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2511b73d-3502-447a-9175-60d60bd9b862-tmp-dir\") pod \"dns-default-nhh6c\" (UID: \"2511b73d-3502-447a-9175-60d60bd9b862\") " pod="openshift-dns/dns-default-nhh6c" Jul 15 21:02:11.978480 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:11.978173 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqw9r\" (UniqueName: \"kubernetes.io/projected/2511b73d-3502-447a-9175-60d60bd9b862-kube-api-access-vqw9r\") pod \"dns-default-nhh6c\" (UID: \"2511b73d-3502-447a-9175-60d60bd9b862\") " pod="openshift-dns/dns-default-nhh6c" Jul 15 21:02:11.978480 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:11.978205 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84vb7\" (UniqueName: \"kubernetes.io/projected/44d76b49-7290-4217-b4b3-2ec82286aa5c-kube-api-access-84vb7\") pod \"ingress-canary-9mhlt\" (UID: \"44d76b49-7290-4217-b4b3-2ec82286aa5c\") " pod="openshift-ingress-canary/ingress-canary-9mhlt" Jul 15 21:02:11.991865 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:11.991826 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dg4dh" Jul 15 21:02:11.992023 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:11.991826 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r59ft" Jul 15 21:02:11.992109 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:11.992039 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4ctrp" Jul 15 21:02:11.995686 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:11.995663 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jfzc2\"" Jul 15 21:02:11.996023 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:11.996002 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 15 21:02:11.996023 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:11.996019 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 15 21:02:11.996339 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:11.996321 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rwswm\"" Jul 15 21:02:11.997622 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:11.997596 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 15 21:02:11.997711 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:11.997660 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 15 21:02:12.078796 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:12.078708 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d89392ea-c74a-4ead-b205-afdd71a00a42-registry-certificates\") pod \"image-registry-7dc5664c6d-hplh7\" (UID: \"d89392ea-c74a-4ead-b205-afdd71a00a42\") " pod="openshift-image-registry/image-registry-7dc5664c6d-hplh7" Jul 15 21:02:12.078796 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:12.078777 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d89392ea-c74a-4ead-b205-afdd71a00a42-bound-sa-token\") pod \"image-registry-7dc5664c6d-hplh7\" (UID: \"d89392ea-c74a-4ead-b205-afdd71a00a42\") " pod="openshift-image-registry/image-registry-7dc5664c6d-hplh7" Jul 15 21:02:12.079014 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:12.078809 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d89392ea-c74a-4ead-b205-afdd71a00a42-registry-tls\") pod \"image-registry-7dc5664c6d-hplh7\" (UID: \"d89392ea-c74a-4ead-b205-afdd71a00a42\") " pod="openshift-image-registry/image-registry-7dc5664c6d-hplh7" Jul 15 21:02:12.079014 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:12.078834 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2511b73d-3502-447a-9175-60d60bd9b862-metrics-tls\") pod \"dns-default-nhh6c\" (UID: \"2511b73d-3502-447a-9175-60d60bd9b862\") " pod="openshift-dns/dns-default-nhh6c" Jul 15 21:02:12.079014 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:12.078867 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d89392ea-c74a-4ead-b205-afdd71a00a42-trusted-ca\") pod \"image-registry-7dc5664c6d-hplh7\" (UID: \"d89392ea-c74a-4ead-b205-afdd71a00a42\") " pod="openshift-image-registry/image-registry-7dc5664c6d-hplh7" Jul 15 21:02:12.079014 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:12.078891 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d89392ea-c74a-4ead-b205-afdd71a00a42-installation-pull-secrets\") pod \"image-registry-7dc5664c6d-hplh7\" (UID: \"d89392ea-c74a-4ead-b205-afdd71a00a42\") " pod="openshift-image-registry/image-registry-7dc5664c6d-hplh7" Jul 15 21:02:12.079014 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:12.078921 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ktn5\" (UniqueName: \"kubernetes.io/projected/d89392ea-c74a-4ead-b205-afdd71a00a42-kube-api-access-4ktn5\") pod \"image-registry-7dc5664c6d-hplh7\" (UID: \"d89392ea-c74a-4ead-b205-afdd71a00a42\") " pod="openshift-image-registry/image-registry-7dc5664c6d-hplh7" Jul 15 21:02:12.079014 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:12.078944 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/44d76b49-7290-4217-b4b3-2ec82286aa5c-cert\") pod \"ingress-canary-9mhlt\" (UID: \"44d76b49-7290-4217-b4b3-2ec82286aa5c\") " pod="openshift-ingress-canary/ingress-canary-9mhlt" Jul 15 21:02:12.079014 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:12.078970 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d89392ea-c74a-4ead-b205-afdd71a00a42-image-registry-private-configuration\") pod \"image-registry-7dc5664c6d-hplh7\" (UID: \"d89392ea-c74a-4ead-b205-afdd71a00a42\") " pod="openshift-image-registry/image-registry-7dc5664c6d-hplh7" Jul 15 21:02:12.079014 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:12.078987 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d89392ea-c74a-4ead-b205-afdd71a00a42-ca-trust-extracted\") pod \"image-registry-7dc5664c6d-hplh7\" (UID: \"d89392ea-c74a-4ead-b205-afdd71a00a42\") " pod="openshift-image-registry/image-registry-7dc5664c6d-hplh7" Jul 15 21:02:12.079014 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:12.079002 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 21:02:12.079442 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:12.079027 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7dc5664c6d-hplh7: secret "image-registry-tls" not found Jul 15 21:02:12.079442 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:12.079106 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d89392ea-c74a-4ead-b205-afdd71a00a42-registry-tls podName:d89392ea-c74a-4ead-b205-afdd71a00a42 nodeName:}" failed. No retries permitted until 2026-07-15 21:02:12.57908715 +0000 UTC m=+37.120399691 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d89392ea-c74a-4ead-b205-afdd71a00a42-registry-tls") pod "image-registry-7dc5664c6d-hplh7" (UID: "d89392ea-c74a-4ead-b205-afdd71a00a42") : secret "image-registry-tls" not found Jul 15 21:02:12.079442 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:12.079434 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d89392ea-c74a-4ead-b205-afdd71a00a42-registry-certificates\") pod \"image-registry-7dc5664c6d-hplh7\" (UID: \"d89392ea-c74a-4ead-b205-afdd71a00a42\") " pod="openshift-image-registry/image-registry-7dc5664c6d-hplh7" Jul 15 21:02:12.079606 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:12.079010 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2511b73d-3502-447a-9175-60d60bd9b862-config-volume\") pod \"dns-default-nhh6c\" (UID: \"2511b73d-3502-447a-9175-60d60bd9b862\") " pod="openshift-dns/dns-default-nhh6c" Jul 15 21:02:12.079606 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:12.079503 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2511b73d-3502-447a-9175-60d60bd9b862-tmp-dir\") pod \"dns-default-nhh6c\" (UID: \"2511b73d-3502-447a-9175-60d60bd9b862\") " pod="openshift-dns/dns-default-nhh6c" Jul 15 21:02:12.079606 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:12.079556 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqw9r\" (UniqueName: \"kubernetes.io/projected/2511b73d-3502-447a-9175-60d60bd9b862-kube-api-access-vqw9r\") pod \"dns-default-nhh6c\" (UID: \"2511b73d-3502-447a-9175-60d60bd9b862\") " pod="openshift-dns/dns-default-nhh6c" Jul 15 21:02:12.079606 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:12.079578 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2511b73d-3502-447a-9175-60d60bd9b862-config-volume\") pod \"dns-default-nhh6c\" (UID: \"2511b73d-3502-447a-9175-60d60bd9b862\") " pod="openshift-dns/dns-default-nhh6c" Jul 15 21:02:12.079606 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:12.079603 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84vb7\" (UniqueName: \"kubernetes.io/projected/44d76b49-7290-4217-b4b3-2ec82286aa5c-kube-api-access-84vb7\") pod \"ingress-canary-9mhlt\" (UID: \"44d76b49-7290-4217-b4b3-2ec82286aa5c\") " pod="openshift-ingress-canary/ingress-canary-9mhlt" Jul 15 21:02:12.079852 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:12.079665 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 21:02:12.079852 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:12.079714 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2511b73d-3502-447a-9175-60d60bd9b862-metrics-tls podName:2511b73d-3502-447a-9175-60d60bd9b862 nodeName:}" failed. No retries permitted until 2026-07-15 21:02:12.579698458 +0000 UTC m=+37.121011015 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2511b73d-3502-447a-9175-60d60bd9b862-metrics-tls") pod "dns-default-nhh6c" (UID: "2511b73d-3502-447a-9175-60d60bd9b862") : secret "dns-default-metrics-tls" not found Jul 15 21:02:12.080057 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:12.080032 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d89392ea-c74a-4ead-b205-afdd71a00a42-ca-trust-extracted\") pod \"image-registry-7dc5664c6d-hplh7\" (UID: \"d89392ea-c74a-4ead-b205-afdd71a00a42\") " pod="openshift-image-registry/image-registry-7dc5664c6d-hplh7" Jul 15 21:02:12.080202 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:12.080104 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2511b73d-3502-447a-9175-60d60bd9b862-tmp-dir\") pod \"dns-default-nhh6c\" (UID: \"2511b73d-3502-447a-9175-60d60bd9b862\") " pod="openshift-dns/dns-default-nhh6c" Jul 15 21:02:12.080848 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:12.080825 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d89392ea-c74a-4ead-b205-afdd71a00a42-trusted-ca\") pod \"image-registry-7dc5664c6d-hplh7\" (UID: \"d89392ea-c74a-4ead-b205-afdd71a00a42\") " pod="openshift-image-registry/image-registry-7dc5664c6d-hplh7" Jul 15 21:02:12.084555 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:12.084535 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d89392ea-c74a-4ead-b205-afdd71a00a42-installation-pull-secrets\") pod \"image-registry-7dc5664c6d-hplh7\" (UID: \"d89392ea-c74a-4ead-b205-afdd71a00a42\") " pod="openshift-image-registry/image-registry-7dc5664c6d-hplh7" Jul 15 21:02:12.084673 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:12.084606 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d89392ea-c74a-4ead-b205-afdd71a00a42-image-registry-private-configuration\") pod \"image-registry-7dc5664c6d-hplh7\" (UID: \"d89392ea-c74a-4ead-b205-afdd71a00a42\") " pod="openshift-image-registry/image-registry-7dc5664c6d-hplh7" Jul 15 21:02:12.084729 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:12.084690 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/44d76b49-7290-4217-b4b3-2ec82286aa5c-cert\") pod \"ingress-canary-9mhlt\" (UID: \"44d76b49-7290-4217-b4b3-2ec82286aa5c\") " pod="openshift-ingress-canary/ingress-canary-9mhlt" Jul 15 21:02:12.093474 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:12.093417 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqw9r\" (UniqueName: \"kubernetes.io/projected/2511b73d-3502-447a-9175-60d60bd9b862-kube-api-access-vqw9r\") pod \"dns-default-nhh6c\" (UID: \"2511b73d-3502-447a-9175-60d60bd9b862\") " pod="openshift-dns/dns-default-nhh6c" Jul 15 21:02:12.094247 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:12.094228 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d89392ea-c74a-4ead-b205-afdd71a00a42-bound-sa-token\") pod \"image-registry-7dc5664c6d-hplh7\" (UID: \"d89392ea-c74a-4ead-b205-afdd71a00a42\") " pod="openshift-image-registry/image-registry-7dc5664c6d-hplh7" Jul 15 21:02:12.094353 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:12.094325 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ktn5\" (UniqueName: \"kubernetes.io/projected/d89392ea-c74a-4ead-b205-afdd71a00a42-kube-api-access-4ktn5\") pod \"image-registry-7dc5664c6d-hplh7\" (UID: \"d89392ea-c74a-4ead-b205-afdd71a00a42\") " pod="openshift-image-registry/image-registry-7dc5664c6d-hplh7" Jul 15 21:02:12.094438 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:12.094412 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84vb7\" (UniqueName: \"kubernetes.io/projected/44d76b49-7290-4217-b4b3-2ec82286aa5c-kube-api-access-84vb7\") pod \"ingress-canary-9mhlt\" (UID: \"44d76b49-7290-4217-b4b3-2ec82286aa5c\") " pod="openshift-ingress-canary/ingress-canary-9mhlt" Jul 15 21:02:12.197394 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:12.197355 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-9mhlt" Jul 15 21:02:12.473789 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:12.473599 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-9mhlt"] Jul 15 21:02:12.529125 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:02:12.529097 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44d76b49_7290_4217_b4b3_2ec82286aa5c.slice/crio-82219aba9d537dcf0a0750b15228d06bed5d4d6bb1b3fce6626f632784048db4 WatchSource:0}: Error finding container 82219aba9d537dcf0a0750b15228d06bed5d4d6bb1b3fce6626f632784048db4: Status 404 returned error can't find the container with id 82219aba9d537dcf0a0750b15228d06bed5d4d6bb1b3fce6626f632784048db4 Jul 15 21:02:12.536233 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:12.536207 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:02:12.583132 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:12.583036 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d89392ea-c74a-4ead-b205-afdd71a00a42-registry-tls\") pod \"image-registry-7dc5664c6d-hplh7\" (UID: \"d89392ea-c74a-4ead-b205-afdd71a00a42\") " pod="openshift-image-registry/image-registry-7dc5664c6d-hplh7" Jul 15 21:02:12.583259 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:12.583190 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 21:02:12.583259 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:12.583215 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7dc5664c6d-hplh7: secret "image-registry-tls" not found Jul 15 21:02:12.583363 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:12.583270 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d89392ea-c74a-4ead-b205-afdd71a00a42-registry-tls podName:d89392ea-c74a-4ead-b205-afdd71a00a42 nodeName:}" failed. No retries permitted until 2026-07-15 21:02:13.58325501 +0000 UTC m=+38.124567553 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d89392ea-c74a-4ead-b205-afdd71a00a42-registry-tls") pod "image-registry-7dc5664c6d-hplh7" (UID: "d89392ea-c74a-4ead-b205-afdd71a00a42") : secret "image-registry-tls" not found Jul 15 21:02:12.583363 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:12.583195 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2511b73d-3502-447a-9175-60d60bd9b862-metrics-tls\") pod \"dns-default-nhh6c\" (UID: \"2511b73d-3502-447a-9175-60d60bd9b862\") " pod="openshift-dns/dns-default-nhh6c" Jul 15 21:02:12.583363 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:12.583296 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 21:02:12.583512 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:12.583402 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2511b73d-3502-447a-9175-60d60bd9b862-metrics-tls podName:2511b73d-3502-447a-9175-60d60bd9b862 nodeName:}" failed. No retries permitted until 2026-07-15 21:02:13.583389758 +0000 UTC m=+38.124702286 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2511b73d-3502-447a-9175-60d60bd9b862-metrics-tls") pod "dns-default-nhh6c" (UID: "2511b73d-3502-447a-9175-60d60bd9b862") : secret "dns-default-metrics-tls" not found Jul 15 21:02:13.272606 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:13.272565 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-9mhlt" event={"ID":"44d76b49-7290-4217-b4b3-2ec82286aa5c","Type":"ContainerStarted","Data":"82219aba9d537dcf0a0750b15228d06bed5d4d6bb1b3fce6626f632784048db4"} Jul 15 21:02:13.275817 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:13.275784 2579 generic.go:356] "Generic (PLEG): container finished" podID="397ae254-d712-41e8-a481-3243dbb511a9" containerID="fa0fa4976d3564532369f439fb2e25d83aa617bacfc47dadf928b61284d09158" exitCode=0 Jul 15 21:02:13.275939 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:13.275831 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mstwf" event={"ID":"397ae254-d712-41e8-a481-3243dbb511a9","Type":"ContainerDied","Data":"fa0fa4976d3564532369f439fb2e25d83aa617bacfc47dadf928b61284d09158"} Jul 15 21:02:13.590277 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:13.590185 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d89392ea-c74a-4ead-b205-afdd71a00a42-registry-tls\") pod \"image-registry-7dc5664c6d-hplh7\" (UID: \"d89392ea-c74a-4ead-b205-afdd71a00a42\") " pod="openshift-image-registry/image-registry-7dc5664c6d-hplh7" Jul 15 21:02:13.590277 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:13.590220 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2511b73d-3502-447a-9175-60d60bd9b862-metrics-tls\") pod \"dns-default-nhh6c\" (UID: \"2511b73d-3502-447a-9175-60d60bd9b862\") " pod="openshift-dns/dns-default-nhh6c" Jul 15 21:02:13.590486 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:13.590341 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 21:02:13.590486 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:13.590349 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 21:02:13.590486 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:13.590370 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7dc5664c6d-hplh7: secret "image-registry-tls" not found Jul 15 21:02:13.590486 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:13.590400 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2511b73d-3502-447a-9175-60d60bd9b862-metrics-tls podName:2511b73d-3502-447a-9175-60d60bd9b862 nodeName:}" failed. No retries permitted until 2026-07-15 21:02:15.590386124 +0000 UTC m=+40.131698654 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2511b73d-3502-447a-9175-60d60bd9b862-metrics-tls") pod "dns-default-nhh6c" (UID: "2511b73d-3502-447a-9175-60d60bd9b862") : secret "dns-default-metrics-tls" not found Jul 15 21:02:13.590486 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:13.590414 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d89392ea-c74a-4ead-b205-afdd71a00a42-registry-tls podName:d89392ea-c74a-4ead-b205-afdd71a00a42 nodeName:}" failed. No retries permitted until 2026-07-15 21:02:15.590408679 +0000 UTC m=+40.131721206 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d89392ea-c74a-4ead-b205-afdd71a00a42-registry-tls") pod "image-registry-7dc5664c6d-hplh7" (UID: "d89392ea-c74a-4ead-b205-afdd71a00a42") : secret "image-registry-tls" not found Jul 15 21:02:13.890445 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:13.890408 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:02:13.986362 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:13.986321 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:02:14.281317 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:14.281219 2579 generic.go:356] "Generic (PLEG): container finished" podID="397ae254-d712-41e8-a481-3243dbb511a9" containerID="0522655ab46fcb9e4bab19d06ce3decd3dddfeb6f4a69d2a43810b15e6b4bd6d" exitCode=0 Jul 15 21:02:14.281317 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:14.281283 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mstwf" event={"ID":"397ae254-d712-41e8-a481-3243dbb511a9","Type":"ContainerDied","Data":"0522655ab46fcb9e4bab19d06ce3decd3dddfeb6f4a69d2a43810b15e6b4bd6d"} Jul 15 21:02:15.284867 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:15.284833 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-9mhlt" event={"ID":"44d76b49-7290-4217-b4b3-2ec82286aa5c","Type":"ContainerStarted","Data":"c18c366bf2818c94ecdf5369bfb16fa05c787566e6577e2dfd38508062b6006a"} Jul 15 21:02:15.288260 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:15.288229 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mstwf" event={"ID":"397ae254-d712-41e8-a481-3243dbb511a9","Type":"ContainerStarted","Data":"58b021fdfe25e09d2d027ad92e9bb301bb5759f3272c77782543d33022c924fe"} Jul 15 21:02:15.305472 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:15.305429 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-9mhlt" podStartSLOduration=1.6638528959999999 podStartE2EDuration="4.305414515s" totalImagesPullingTime="2.641561619s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:02:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:02:15.305126301 +0000 UTC m=+39.846438856" watchObservedRunningTime="2026-07-15 21:02:15.305414515 +0000 UTC m=+39.846727065" Jul 15 21:02:15.339164 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:15.339116 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-mstwf" podStartSLOduration=6.666572899 podStartE2EDuration="38.33909923s" totalImagesPullingTime="31.672526331s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 21:01:37 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:02:15.33877525 +0000 UTC m=+39.880087801" watchObservedRunningTime="2026-07-15 21:02:15.33909923 +0000 UTC m=+39.880411779" Jul 15 21:02:15.466598 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:15.466497 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:02:15.542178 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:15.542135 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:02:15.607353 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:15.607308 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d89392ea-c74a-4ead-b205-afdd71a00a42-registry-tls\") pod \"image-registry-7dc5664c6d-hplh7\" (UID: \"d89392ea-c74a-4ead-b205-afdd71a00a42\") " pod="openshift-image-registry/image-registry-7dc5664c6d-hplh7" Jul 15 21:02:15.607353 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:15.607343 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2511b73d-3502-447a-9175-60d60bd9b862-metrics-tls\") pod \"dns-default-nhh6c\" (UID: \"2511b73d-3502-447a-9175-60d60bd9b862\") " pod="openshift-dns/dns-default-nhh6c" Jul 15 21:02:15.607607 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:15.607451 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 21:02:15.607607 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:15.607457 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 21:02:15.607607 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:15.607474 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7dc5664c6d-hplh7: secret "image-registry-tls" not found Jul 15 21:02:15.607607 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:15.607503 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2511b73d-3502-447a-9175-60d60bd9b862-metrics-tls podName:2511b73d-3502-447a-9175-60d60bd9b862 nodeName:}" failed. No retries permitted until 2026-07-15 21:02:19.607491079 +0000 UTC m=+44.148803610 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2511b73d-3502-447a-9175-60d60bd9b862-metrics-tls") pod "dns-default-nhh6c" (UID: "2511b73d-3502-447a-9175-60d60bd9b862") : secret "dns-default-metrics-tls" not found Jul 15 21:02:15.607607 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:15.607523 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d89392ea-c74a-4ead-b205-afdd71a00a42-registry-tls podName:d89392ea-c74a-4ead-b205-afdd71a00a42 nodeName:}" failed. No retries permitted until 2026-07-15 21:02:19.607511124 +0000 UTC m=+44.148823652 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d89392ea-c74a-4ead-b205-afdd71a00a42-registry-tls") pod "image-registry-7dc5664c6d-hplh7" (UID: "d89392ea-c74a-4ead-b205-afdd71a00a42") : secret "image-registry-tls" not found Jul 15 21:02:16.893343 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:16.893307 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:02:16.958819 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:16.958786 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:02:19.024642 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:19.024604 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:02:19.126984 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:19.126935 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:02:19.478273 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:19.478238 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:02:19.550566 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:19.550537 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:02:19.632669 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:19.632627 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d89392ea-c74a-4ead-b205-afdd71a00a42-registry-tls\") pod \"image-registry-7dc5664c6d-hplh7\" (UID: \"d89392ea-c74a-4ead-b205-afdd71a00a42\") " pod="openshift-image-registry/image-registry-7dc5664c6d-hplh7" Jul 15 21:02:19.632669 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:19.632668 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2511b73d-3502-447a-9175-60d60bd9b862-metrics-tls\") pod \"dns-default-nhh6c\" (UID: \"2511b73d-3502-447a-9175-60d60bd9b862\") " pod="openshift-dns/dns-default-nhh6c" Jul 15 21:02:19.632886 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:19.632789 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 21:02:19.632886 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:19.632806 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7dc5664c6d-hplh7: secret "image-registry-tls" not found Jul 15 21:02:19.632886 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:19.632828 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 21:02:19.632886 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:19.632871 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d89392ea-c74a-4ead-b205-afdd71a00a42-registry-tls podName:d89392ea-c74a-4ead-b205-afdd71a00a42 nodeName:}" failed. No retries permitted until 2026-07-15 21:02:27.632852073 +0000 UTC m=+52.174164602 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d89392ea-c74a-4ead-b205-afdd71a00a42-registry-tls") pod "image-registry-7dc5664c6d-hplh7" (UID: "d89392ea-c74a-4ead-b205-afdd71a00a42") : secret "image-registry-tls" not found Jul 15 21:02:19.632886 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:19.632885 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2511b73d-3502-447a-9175-60d60bd9b862-metrics-tls podName:2511b73d-3502-447a-9175-60d60bd9b862 nodeName:}" failed. No retries permitted until 2026-07-15 21:02:27.63287935 +0000 UTC m=+52.174191878 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2511b73d-3502-447a-9175-60d60bd9b862-metrics-tls") pod "dns-default-nhh6c" (UID: "2511b73d-3502-447a-9175-60d60bd9b862") : secret "dns-default-metrics-tls" not found Jul 15 21:02:23.776615 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:23.776583 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:02:23.849139 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:23.849108 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:02:27.692011 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:27.691973 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d89392ea-c74a-4ead-b205-afdd71a00a42-registry-tls\") pod \"image-registry-7dc5664c6d-hplh7\" (UID: \"d89392ea-c74a-4ead-b205-afdd71a00a42\") " pod="openshift-image-registry/image-registry-7dc5664c6d-hplh7" Jul 15 21:02:27.692011 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:27.692012 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2511b73d-3502-447a-9175-60d60bd9b862-metrics-tls\") pod \"dns-default-nhh6c\" (UID: \"2511b73d-3502-447a-9175-60d60bd9b862\") " pod="openshift-dns/dns-default-nhh6c" Jul 15 21:02:27.692457 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:27.692146 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 21:02:27.692457 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:27.692163 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 21:02:27.692457 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:27.692223 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2511b73d-3502-447a-9175-60d60bd9b862-metrics-tls podName:2511b73d-3502-447a-9175-60d60bd9b862 nodeName:}" failed. No retries permitted until 2026-07-15 21:02:43.692209452 +0000 UTC m=+68.233521983 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2511b73d-3502-447a-9175-60d60bd9b862-metrics-tls") pod "dns-default-nhh6c" (UID: "2511b73d-3502-447a-9175-60d60bd9b862") : secret "dns-default-metrics-tls" not found Jul 15 21:02:27.692457 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:27.692164 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7dc5664c6d-hplh7: secret "image-registry-tls" not found Jul 15 21:02:27.692457 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:27.692287 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d89392ea-c74a-4ead-b205-afdd71a00a42-registry-tls podName:d89392ea-c74a-4ead-b205-afdd71a00a42 nodeName:}" failed. No retries permitted until 2026-07-15 21:02:43.692274261 +0000 UTC m=+68.233586789 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d89392ea-c74a-4ead-b205-afdd71a00a42-registry-tls") pod "image-registry-7dc5664c6d-hplh7" (UID: "d89392ea-c74a-4ead-b205-afdd71a00a42") : secret "image-registry-tls" not found Jul 15 21:02:33.402851 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:33.402819 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:02:33.501310 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:33.501273 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:02:34.460725 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:34.460696 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:02:34.546859 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:34.546818 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:02:34.627704 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:34.627653 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:02:36.261720 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:36.261690 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-m5r2m" Jul 15 21:02:36.421215 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:36.421177 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:02:36.522253 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:36.522156 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:02:40.560245 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:40.560209 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:02:40.646602 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:40.646568 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:02:41.792579 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:41.792527 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fptjd\" (UniqueName: \"kubernetes.io/projected/eeaa6fcf-b320-4851-9bf9-583c963e07ba-kube-api-access-fptjd\") pod \"network-check-target-dg4dh\" (UID: \"eeaa6fcf-b320-4851-9bf9-583c963e07ba\") " pod="openshift-network-diagnostics/network-check-target-dg4dh" Jul 15 21:02:41.795943 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:41.795918 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 15 21:02:41.806656 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:41.806636 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 15 21:02:41.818395 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:41.818369 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fptjd\" (UniqueName: \"kubernetes.io/projected/eeaa6fcf-b320-4851-9bf9-583c963e07ba-kube-api-access-fptjd\") pod \"network-check-target-dg4dh\" (UID: \"eeaa6fcf-b320-4851-9bf9-583c963e07ba\") " pod="openshift-network-diagnostics/network-check-target-dg4dh" Jul 15 21:02:41.993779 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:41.993747 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0dafa58d-15c7-46c9-81bd-c3c86bbf8e42-metrics-certs\") pod \"network-metrics-daemon-r59ft\" (UID: \"0dafa58d-15c7-46c9-81bd-c3c86bbf8e42\") " pod="openshift-multus/network-metrics-daemon-r59ft" Jul 15 21:02:41.993779 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:41.993785 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3f26570b-5abd-4460-ba74-e59ba65032a1-original-pull-secret\") pod \"global-pull-secret-syncer-4ctrp\" (UID: \"3f26570b-5abd-4460-ba74-e59ba65032a1\") " pod="kube-system/global-pull-secret-syncer-4ctrp" Jul 15 21:02:41.997422 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:41.997397 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 15 21:02:41.997546 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:41.997428 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 15 21:02:42.004472 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:42.004454 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 15 21:02:42.004545 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:42.004534 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0dafa58d-15c7-46c9-81bd-c3c86bbf8e42-metrics-certs podName:0dafa58d-15c7-46c9-81bd-c3c86bbf8e42 nodeName:}" failed. No retries permitted until 2026-07-15 21:03:46.004513178 +0000 UTC m=+130.545825717 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0dafa58d-15c7-46c9-81bd-c3c86bbf8e42-metrics-certs") pod "network-metrics-daemon-r59ft" (UID: "0dafa58d-15c7-46c9-81bd-c3c86bbf8e42") : secret "metrics-daemon-secret" not found Jul 15 21:02:42.006664 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:42.006643 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3f26570b-5abd-4460-ba74-e59ba65032a1-original-pull-secret\") pod \"global-pull-secret-syncer-4ctrp\" (UID: \"3f26570b-5abd-4460-ba74-e59ba65032a1\") " pod="kube-system/global-pull-secret-syncer-4ctrp" Jul 15 21:02:42.006824 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:42.006811 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jfzc2\"" Jul 15 21:02:42.013888 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:42.013869 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dg4dh" Jul 15 21:02:42.016160 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:42.016145 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4ctrp" Jul 15 21:02:42.181136 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:42.181108 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4ctrp"] Jul 15 21:02:42.185778 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:42.185748 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-dg4dh"] Jul 15 21:02:42.188761 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:02:42.188735 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeeaa6fcf_b320_4851_9bf9_583c963e07ba.slice/crio-1175523dd65b208fa576787188155790ada4f005c7563723cc517804a6b60262 WatchSource:0}: Error finding container 1175523dd65b208fa576787188155790ada4f005c7563723cc517804a6b60262: Status 404 returned error can't find the container with id 1175523dd65b208fa576787188155790ada4f005c7563723cc517804a6b60262 Jul 15 21:02:42.189298 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:02:42.189279 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f26570b_5abd_4460_ba74_e59ba65032a1.slice/crio-62f3934e0bf5a6fa52a238ebeea7c3d463eeb47759bc48e745cbe3f5b48351de WatchSource:0}: Error finding container 62f3934e0bf5a6fa52a238ebeea7c3d463eeb47759bc48e745cbe3f5b48351de: Status 404 returned error can't find the container with id 62f3934e0bf5a6fa52a238ebeea7c3d463eeb47759bc48e745cbe3f5b48351de Jul 15 21:02:42.351050 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:42.350959 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4ctrp" event={"ID":"3f26570b-5abd-4460-ba74-e59ba65032a1","Type":"ContainerStarted","Data":"62f3934e0bf5a6fa52a238ebeea7c3d463eeb47759bc48e745cbe3f5b48351de"} Jul 15 21:02:42.352159 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:42.352138 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-dg4dh" event={"ID":"eeaa6fcf-b320-4851-9bf9-583c963e07ba","Type":"ContainerStarted","Data":"1175523dd65b208fa576787188155790ada4f005c7563723cc517804a6b60262"} Jul 15 21:02:43.710530 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:43.710488 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d89392ea-c74a-4ead-b205-afdd71a00a42-registry-tls\") pod \"image-registry-7dc5664c6d-hplh7\" (UID: \"d89392ea-c74a-4ead-b205-afdd71a00a42\") " pod="openshift-image-registry/image-registry-7dc5664c6d-hplh7" Jul 15 21:02:43.710924 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:43.710538 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2511b73d-3502-447a-9175-60d60bd9b862-metrics-tls\") pod \"dns-default-nhh6c\" (UID: \"2511b73d-3502-447a-9175-60d60bd9b862\") " pod="openshift-dns/dns-default-nhh6c" Jul 15 21:02:43.710924 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:43.710729 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 21:02:43.710924 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:43.710790 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2511b73d-3502-447a-9175-60d60bd9b862-metrics-tls podName:2511b73d-3502-447a-9175-60d60bd9b862 nodeName:}" failed. No retries permitted until 2026-07-15 21:03:15.710770732 +0000 UTC m=+100.252083265 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2511b73d-3502-447a-9175-60d60bd9b862-metrics-tls") pod "dns-default-nhh6c" (UID: "2511b73d-3502-447a-9175-60d60bd9b862") : secret "dns-default-metrics-tls" not found Jul 15 21:02:43.711187 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:43.711164 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 21:02:43.711187 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:43.711183 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7dc5664c6d-hplh7: secret "image-registry-tls" not found Jul 15 21:02:43.711389 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:43.711228 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d89392ea-c74a-4ead-b205-afdd71a00a42-registry-tls podName:d89392ea-c74a-4ead-b205-afdd71a00a42 nodeName:}" failed. No retries permitted until 2026-07-15 21:03:15.711212701 +0000 UTC m=+100.252525243 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d89392ea-c74a-4ead-b205-afdd71a00a42-registry-tls") pod "image-registry-7dc5664c6d-hplh7" (UID: "d89392ea-c74a-4ead-b205-afdd71a00a42") : secret "image-registry-tls" not found Jul 15 21:02:45.165542 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:45.165502 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:02:45.235780 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:45.235739 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:02:46.153967 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:46.153906 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-p85nr"] Jul 15 21:02:46.167432 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:46.167407 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-p85nr" Jul 15 21:02:46.180386 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:46.180365 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 15 21:02:46.180520 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:46.180433 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 15 21:02:46.180953 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:46.180939 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 15 21:02:46.181212 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:46.181197 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 15 21:02:46.181212 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:46.181206 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-vhbfp\"" Jul 15 21:02:46.212411 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:46.212320 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-p85nr"] Jul 15 21:02:46.229015 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:46.228965 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/835f5980-1870-4052-af02-3e3a4653501c-config\") pod \"service-ca-operator-7695fc6c8c-p85nr\" (UID: \"835f5980-1870-4052-af02-3e3a4653501c\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-p85nr" Jul 15 21:02:46.229200 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:46.229027 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/835f5980-1870-4052-af02-3e3a4653501c-serving-cert\") pod \"service-ca-operator-7695fc6c8c-p85nr\" (UID: \"835f5980-1870-4052-af02-3e3a4653501c\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-p85nr" Jul 15 21:02:46.229200 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:46.229121 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndksb\" (UniqueName: \"kubernetes.io/projected/835f5980-1870-4052-af02-3e3a4653501c-kube-api-access-ndksb\") pod \"service-ca-operator-7695fc6c8c-p85nr\" (UID: \"835f5980-1870-4052-af02-3e3a4653501c\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-p85nr" Jul 15 21:02:46.330262 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:46.330234 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/835f5980-1870-4052-af02-3e3a4653501c-serving-cert\") pod \"service-ca-operator-7695fc6c8c-p85nr\" (UID: \"835f5980-1870-4052-af02-3e3a4653501c\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-p85nr" Jul 15 21:02:46.330434 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:46.330283 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndksb\" (UniqueName: \"kubernetes.io/projected/835f5980-1870-4052-af02-3e3a4653501c-kube-api-access-ndksb\") pod \"service-ca-operator-7695fc6c8c-p85nr\" (UID: \"835f5980-1870-4052-af02-3e3a4653501c\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-p85nr" Jul 15 21:02:46.330434 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:46.330360 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/835f5980-1870-4052-af02-3e3a4653501c-config\") pod \"service-ca-operator-7695fc6c8c-p85nr\" (UID: \"835f5980-1870-4052-af02-3e3a4653501c\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-p85nr" Jul 15 21:02:46.331010 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:46.330981 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/835f5980-1870-4052-af02-3e3a4653501c-config\") pod \"service-ca-operator-7695fc6c8c-p85nr\" (UID: \"835f5980-1870-4052-af02-3e3a4653501c\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-p85nr" Jul 15 21:02:46.332649 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:46.332624 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/835f5980-1870-4052-af02-3e3a4653501c-serving-cert\") pod \"service-ca-operator-7695fc6c8c-p85nr\" (UID: \"835f5980-1870-4052-af02-3e3a4653501c\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-p85nr" Jul 15 21:02:46.352598 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:46.352567 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndksb\" (UniqueName: \"kubernetes.io/projected/835f5980-1870-4052-af02-3e3a4653501c-kube-api-access-ndksb\") pod \"service-ca-operator-7695fc6c8c-p85nr\" (UID: \"835f5980-1870-4052-af02-3e3a4653501c\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-p85nr" Jul 15 21:02:46.478660 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:46.478569 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-p85nr" Jul 15 21:02:46.498343 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:46.498318 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:02:46.569445 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:46.569391 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:02:47.032670 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:47.032644 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-p85nr"] Jul 15 21:02:47.035349 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:02:47.035326 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod835f5980_1870_4052_af02_3e3a4653501c.slice/crio-da4208a72c886c9568d93e84d52c3d4e49d12fc138b65abfc5806090d07937e6 WatchSource:0}: Error finding container da4208a72c886c9568d93e84d52c3d4e49d12fc138b65abfc5806090d07937e6: Status 404 returned error can't find the container with id da4208a72c886c9568d93e84d52c3d4e49d12fc138b65abfc5806090d07937e6 Jul 15 21:02:47.367615 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:47.367584 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4ctrp" event={"ID":"3f26570b-5abd-4460-ba74-e59ba65032a1","Type":"ContainerStarted","Data":"f2cd3bd85e5e0fb9292a6ff51d47786dd910e2cbf26abd350b4b61fddc3dbadb"} Jul 15 21:02:47.369015 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:47.368992 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-dg4dh" event={"ID":"eeaa6fcf-b320-4851-9bf9-583c963e07ba","Type":"ContainerStarted","Data":"dd56c002de56d8ba0a00836f555188aa07c1d15976f8eda7314a9277000b9f33"} Jul 15 21:02:47.369150 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:47.369047 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-dg4dh" Jul 15 21:02:47.370103 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:47.370083 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-p85nr" event={"ID":"835f5980-1870-4052-af02-3e3a4653501c","Type":"ContainerStarted","Data":"da4208a72c886c9568d93e84d52c3d4e49d12fc138b65abfc5806090d07937e6"} Jul 15 21:02:47.389259 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:47.389219 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-4ctrp" podStartSLOduration=65.668068108 podStartE2EDuration="1m10.389201897s" totalImagesPullingTime="4.721133789s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 21:01:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:02:47.388523421 +0000 UTC m=+71.929835971" watchObservedRunningTime="2026-07-15 21:02:47.389201897 +0000 UTC m=+71.930514446" Jul 15 21:02:47.411882 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:47.411842 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-dg4dh" podStartSLOduration=66.701761619 podStartE2EDuration="1m11.411832343s" totalImagesPullingTime="4.710070724s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:01:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:02:47.411319078 +0000 UTC m=+71.952631640" watchObservedRunningTime="2026-07-15 21:02:47.411832343 +0000 UTC m=+71.953144870" Jul 15 21:02:49.655410 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:49.655371 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:02:49.754040 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:49.753996 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:02:50.382716 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:50.382679 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-p85nr" event={"ID":"835f5980-1870-4052-af02-3e3a4653501c","Type":"ContainerStarted","Data":"10ee1ccd2139578251eb730ee2013584f29af870fd0e6290039dc24a3d2199c1"} Jul 15 21:02:50.404273 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:50.404215 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-p85nr" podStartSLOduration=1.725877885 podStartE2EDuration="4.404199302s" totalImagesPullingTime="2.678321417s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:02:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:02:50.40140834 +0000 UTC m=+74.942720889" watchObservedRunningTime="2026-07-15 21:02:50.404199302 +0000 UTC m=+74.945511903" Jul 15 21:02:51.118847 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:51.118811 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:02:51.194566 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:51.194531 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:02:52.159792 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:52.159757 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-mmxvl"] Jul 15 21:02:52.161937 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:52.161918 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mmxvl" Jul 15 21:02:52.165780 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:52.165761 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 15 21:02:52.166378 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:52.166352 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 15 21:02:52.167078 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:52.167044 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 15 21:02:52.167197 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:52.167096 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-s5tt8\"" Jul 15 21:02:52.167769 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:52.167751 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 15 21:02:52.186726 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:52.186701 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mmxvl"] Jul 15 21:02:52.274818 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:52.274783 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gmrt\" (UniqueName: \"kubernetes.io/projected/50d5a72a-1db9-492b-ab5c-21c12a0efca2-kube-api-access-5gmrt\") pod \"insights-runtime-extractor-mmxvl\" (UID: \"50d5a72a-1db9-492b-ab5c-21c12a0efca2\") " pod="openshift-insights/insights-runtime-extractor-mmxvl" Jul 15 21:02:52.274818 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:52.274822 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/50d5a72a-1db9-492b-ab5c-21c12a0efca2-crio-socket\") pod \"insights-runtime-extractor-mmxvl\" (UID: \"50d5a72a-1db9-492b-ab5c-21c12a0efca2\") " pod="openshift-insights/insights-runtime-extractor-mmxvl" Jul 15 21:02:52.275027 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:52.274840 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/50d5a72a-1db9-492b-ab5c-21c12a0efca2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mmxvl\" (UID: \"50d5a72a-1db9-492b-ab5c-21c12a0efca2\") " pod="openshift-insights/insights-runtime-extractor-mmxvl" Jul 15 21:02:52.275027 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:52.274888 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/50d5a72a-1db9-492b-ab5c-21c12a0efca2-data-volume\") pod \"insights-runtime-extractor-mmxvl\" (UID: \"50d5a72a-1db9-492b-ab5c-21c12a0efca2\") " pod="openshift-insights/insights-runtime-extractor-mmxvl" Jul 15 21:02:52.275027 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:52.274939 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/50d5a72a-1db9-492b-ab5c-21c12a0efca2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mmxvl\" (UID: \"50d5a72a-1db9-492b-ab5c-21c12a0efca2\") " pod="openshift-insights/insights-runtime-extractor-mmxvl" Jul 15 21:02:52.375481 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:52.375450 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/50d5a72a-1db9-492b-ab5c-21c12a0efca2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mmxvl\" (UID: \"50d5a72a-1db9-492b-ab5c-21c12a0efca2\") " pod="openshift-insights/insights-runtime-extractor-mmxvl" Jul 15 21:02:52.375636 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:52.375502 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gmrt\" (UniqueName: \"kubernetes.io/projected/50d5a72a-1db9-492b-ab5c-21c12a0efca2-kube-api-access-5gmrt\") pod \"insights-runtime-extractor-mmxvl\" (UID: \"50d5a72a-1db9-492b-ab5c-21c12a0efca2\") " pod="openshift-insights/insights-runtime-extractor-mmxvl" Jul 15 21:02:52.375636 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:52.375525 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/50d5a72a-1db9-492b-ab5c-21c12a0efca2-crio-socket\") pod \"insights-runtime-extractor-mmxvl\" (UID: \"50d5a72a-1db9-492b-ab5c-21c12a0efca2\") " pod="openshift-insights/insights-runtime-extractor-mmxvl" Jul 15 21:02:52.375636 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:52.375545 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/50d5a72a-1db9-492b-ab5c-21c12a0efca2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mmxvl\" (UID: \"50d5a72a-1db9-492b-ab5c-21c12a0efca2\") " pod="openshift-insights/insights-runtime-extractor-mmxvl" Jul 15 21:02:52.375636 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:52.375564 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/50d5a72a-1db9-492b-ab5c-21c12a0efca2-data-volume\") pod \"insights-runtime-extractor-mmxvl\" (UID: \"50d5a72a-1db9-492b-ab5c-21c12a0efca2\") " pod="openshift-insights/insights-runtime-extractor-mmxvl" Jul 15 21:02:52.375636 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:52.375621 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 21:02:52.375824 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:52.375645 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 21:02:52.375824 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:52.375691 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50d5a72a-1db9-492b-ab5c-21c12a0efca2-insights-runtime-extractor-tls podName:50d5a72a-1db9-492b-ab5c-21c12a0efca2 nodeName:}" failed. No retries permitted until 2026-07-15 21:02:52.875671506 +0000 UTC m=+77.416984040 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/50d5a72a-1db9-492b-ab5c-21c12a0efca2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mmxvl" (UID: "50d5a72a-1db9-492b-ab5c-21c12a0efca2") : secret "insights-runtime-extractor-tls" not found Jul 15 21:02:52.375824 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:52.375720 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/50d5a72a-1db9-492b-ab5c-21c12a0efca2-kube-rbac-proxy-cm podName:50d5a72a-1db9-492b-ab5c-21c12a0efca2 nodeName:}" failed. No retries permitted until 2026-07-15 21:02:52.875701049 +0000 UTC m=+77.417013604 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/50d5a72a-1db9-492b-ab5c-21c12a0efca2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mmxvl" (UID: "50d5a72a-1db9-492b-ab5c-21c12a0efca2") : configmap "kube-rbac-proxy" not found Jul 15 21:02:52.375824 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:52.375643 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/50d5a72a-1db9-492b-ab5c-21c12a0efca2-crio-socket\") pod \"insights-runtime-extractor-mmxvl\" (UID: \"50d5a72a-1db9-492b-ab5c-21c12a0efca2\") " pod="openshift-insights/insights-runtime-extractor-mmxvl" Jul 15 21:02:52.375965 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:52.375877 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/50d5a72a-1db9-492b-ab5c-21c12a0efca2-data-volume\") pod \"insights-runtime-extractor-mmxvl\" (UID: \"50d5a72a-1db9-492b-ab5c-21c12a0efca2\") " pod="openshift-insights/insights-runtime-extractor-mmxvl" Jul 15 21:02:52.384693 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:52.384657 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gmrt\" (UniqueName: \"kubernetes.io/projected/50d5a72a-1db9-492b-ab5c-21c12a0efca2-kube-api-access-5gmrt\") pod \"insights-runtime-extractor-mmxvl\" (UID: \"50d5a72a-1db9-492b-ab5c-21c12a0efca2\") " pod="openshift-insights/insights-runtime-extractor-mmxvl" Jul 15 21:02:52.610661 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:52.610628 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:02:52.714077 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:52.714010 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:02:52.784730 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:52.784699 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-974pc"] Jul 15 21:02:52.786863 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:52.786847 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-974pc" Jul 15 21:02:52.791571 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:52.791543 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-s94cg\"" Jul 15 21:02:52.791571 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:52.791568 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 15 21:02:52.791816 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:52.791803 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 15 21:02:52.803027 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:52.802999 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-974pc"] Jul 15 21:02:52.829147 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:52.829117 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:02:52.878525 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:52.878448 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/50d5a72a-1db9-492b-ab5c-21c12a0efca2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mmxvl\" (UID: \"50d5a72a-1db9-492b-ab5c-21c12a0efca2\") " pod="openshift-insights/insights-runtime-extractor-mmxvl" Jul 15 21:02:52.878525 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:52.878515 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/50d5a72a-1db9-492b-ab5c-21c12a0efca2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mmxvl\" (UID: \"50d5a72a-1db9-492b-ab5c-21c12a0efca2\") " pod="openshift-insights/insights-runtime-extractor-mmxvl" Jul 15 21:02:52.878686 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:52.878550 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9z2rj\" (UniqueName: \"kubernetes.io/projected/92770f0f-ea66-4ff1-906b-32fff3b4447f-kube-api-access-9z2rj\") pod \"migrator-7f59b54f6c-974pc\" (UID: \"92770f0f-ea66-4ff1-906b-32fff3b4447f\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-974pc" Jul 15 21:02:52.878686 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:52.878571 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 21:02:52.878686 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:52.878635 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/50d5a72a-1db9-492b-ab5c-21c12a0efca2-kube-rbac-proxy-cm podName:50d5a72a-1db9-492b-ab5c-21c12a0efca2 nodeName:}" failed. No retries permitted until 2026-07-15 21:02:53.87862101 +0000 UTC m=+78.419933538 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/50d5a72a-1db9-492b-ab5c-21c12a0efca2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mmxvl" (UID: "50d5a72a-1db9-492b-ab5c-21c12a0efca2") : configmap "kube-rbac-proxy" not found Jul 15 21:02:52.878686 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:52.878657 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 21:02:52.878875 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:52.878713 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50d5a72a-1db9-492b-ab5c-21c12a0efca2-insights-runtime-extractor-tls podName:50d5a72a-1db9-492b-ab5c-21c12a0efca2 nodeName:}" failed. No retries permitted until 2026-07-15 21:02:53.878697746 +0000 UTC m=+78.420010274 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/50d5a72a-1db9-492b-ab5c-21c12a0efca2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mmxvl" (UID: "50d5a72a-1db9-492b-ab5c-21c12a0efca2") : secret "insights-runtime-extractor-tls" not found Jul 15 21:02:52.922837 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:52.922800 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:02:52.979757 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:52.979724 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9z2rj\" (UniqueName: \"kubernetes.io/projected/92770f0f-ea66-4ff1-906b-32fff3b4447f-kube-api-access-9z2rj\") pod \"migrator-7f59b54f6c-974pc\" (UID: \"92770f0f-ea66-4ff1-906b-32fff3b4447f\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-974pc" Jul 15 21:02:52.989381 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:52.989349 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9z2rj\" (UniqueName: \"kubernetes.io/projected/92770f0f-ea66-4ff1-906b-32fff3b4447f-kube-api-access-9z2rj\") pod \"migrator-7f59b54f6c-974pc\" (UID: \"92770f0f-ea66-4ff1-906b-32fff3b4447f\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-974pc" Jul 15 21:02:53.096001 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:53.095962 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-974pc" Jul 15 21:02:53.219810 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:53.219786 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-974pc"] Jul 15 21:02:53.222256 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:02:53.222232 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92770f0f_ea66_4ff1_906b_32fff3b4447f.slice/crio-c12101a184d4c0a5eb8b4138fa26e4fe6025a5e722608d6778a0a89b016ef390 WatchSource:0}: Error finding container c12101a184d4c0a5eb8b4138fa26e4fe6025a5e722608d6778a0a89b016ef390: Status 404 returned error can't find the container with id c12101a184d4c0a5eb8b4138fa26e4fe6025a5e722608d6778a0a89b016ef390 Jul 15 21:02:53.390949 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:53.390914 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-974pc" event={"ID":"92770f0f-ea66-4ff1-906b-32fff3b4447f","Type":"ContainerStarted","Data":"c12101a184d4c0a5eb8b4138fa26e4fe6025a5e722608d6778a0a89b016ef390"} Jul 15 21:02:53.397361 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:53.397288 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-q9qt7"] Jul 15 21:02:53.399749 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:53.399730 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-q9qt7" Jul 15 21:02:53.417281 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:53.417258 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 15 21:02:53.417385 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:53.417296 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 15 21:02:53.417497 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:53.417484 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 15 21:02:53.418706 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:53.418693 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 15 21:02:53.422117 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:53.422099 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-j7dlp\"" Jul 15 21:02:53.432255 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:53.432223 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-q9qt7"] Jul 15 21:02:53.484332 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:53.484302 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/53c27246-99aa-4570-8f65-0d0f7a0fde09-signing-key\") pod \"service-ca-79f68b9f4f-q9qt7\" (UID: \"53c27246-99aa-4570-8f65-0d0f7a0fde09\") " pod="openshift-service-ca/service-ca-79f68b9f4f-q9qt7" Jul 15 21:02:53.484481 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:53.484367 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zz7m5\" (UniqueName: \"kubernetes.io/projected/53c27246-99aa-4570-8f65-0d0f7a0fde09-kube-api-access-zz7m5\") pod \"service-ca-79f68b9f4f-q9qt7\" (UID: \"53c27246-99aa-4570-8f65-0d0f7a0fde09\") " pod="openshift-service-ca/service-ca-79f68b9f4f-q9qt7" Jul 15 21:02:53.484481 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:53.484411 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/53c27246-99aa-4570-8f65-0d0f7a0fde09-signing-cabundle\") pod \"service-ca-79f68b9f4f-q9qt7\" (UID: \"53c27246-99aa-4570-8f65-0d0f7a0fde09\") " pod="openshift-service-ca/service-ca-79f68b9f4f-q9qt7" Jul 15 21:02:53.497509 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:53.497477 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:02:53.585220 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:53.585189 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zz7m5\" (UniqueName: \"kubernetes.io/projected/53c27246-99aa-4570-8f65-0d0f7a0fde09-kube-api-access-zz7m5\") pod \"service-ca-79f68b9f4f-q9qt7\" (UID: \"53c27246-99aa-4570-8f65-0d0f7a0fde09\") " pod="openshift-service-ca/service-ca-79f68b9f4f-q9qt7" Jul 15 21:02:53.585386 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:53.585249 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/53c27246-99aa-4570-8f65-0d0f7a0fde09-signing-cabundle\") pod \"service-ca-79f68b9f4f-q9qt7\" (UID: \"53c27246-99aa-4570-8f65-0d0f7a0fde09\") " pod="openshift-service-ca/service-ca-79f68b9f4f-q9qt7" Jul 15 21:02:53.585433 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:53.585409 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/53c27246-99aa-4570-8f65-0d0f7a0fde09-signing-key\") pod \"service-ca-79f68b9f4f-q9qt7\" (UID: \"53c27246-99aa-4570-8f65-0d0f7a0fde09\") " pod="openshift-service-ca/service-ca-79f68b9f4f-q9qt7" Jul 15 21:02:53.589339 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:53.586421 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/53c27246-99aa-4570-8f65-0d0f7a0fde09-signing-cabundle\") pod \"service-ca-79f68b9f4f-q9qt7\" (UID: \"53c27246-99aa-4570-8f65-0d0f7a0fde09\") " pod="openshift-service-ca/service-ca-79f68b9f4f-q9qt7" Jul 15 21:02:53.589339 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:53.588856 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/53c27246-99aa-4570-8f65-0d0f7a0fde09-signing-key\") pod \"service-ca-79f68b9f4f-q9qt7\" (UID: \"53c27246-99aa-4570-8f65-0d0f7a0fde09\") " pod="openshift-service-ca/service-ca-79f68b9f4f-q9qt7" Jul 15 21:02:53.591191 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:53.591163 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:02:53.593774 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:53.593752 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zz7m5\" (UniqueName: \"kubernetes.io/projected/53c27246-99aa-4570-8f65-0d0f7a0fde09-kube-api-access-zz7m5\") pod \"service-ca-79f68b9f4f-q9qt7\" (UID: \"53c27246-99aa-4570-8f65-0d0f7a0fde09\") " pod="openshift-service-ca/service-ca-79f68b9f4f-q9qt7" Jul 15 21:02:53.708117 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:53.708017 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-q9qt7" Jul 15 21:02:53.829592 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:53.829560 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-q9qt7"] Jul 15 21:02:53.832683 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:02:53.832652 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53c27246_99aa_4570_8f65_0d0f7a0fde09.slice/crio-b42856e0fa100f730214fb07d0706edfde570bb658f3620b00b1240f12f3ea1c WatchSource:0}: Error finding container b42856e0fa100f730214fb07d0706edfde570bb658f3620b00b1240f12f3ea1c: Status 404 returned error can't find the container with id b42856e0fa100f730214fb07d0706edfde570bb658f3620b00b1240f12f3ea1c Jul 15 21:02:53.887494 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:53.887473 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/50d5a72a-1db9-492b-ab5c-21c12a0efca2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mmxvl\" (UID: \"50d5a72a-1db9-492b-ab5c-21c12a0efca2\") " pod="openshift-insights/insights-runtime-extractor-mmxvl" Jul 15 21:02:53.887585 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:53.887533 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/50d5a72a-1db9-492b-ab5c-21c12a0efca2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mmxvl\" (UID: \"50d5a72a-1db9-492b-ab5c-21c12a0efca2\") " pod="openshift-insights/insights-runtime-extractor-mmxvl" Jul 15 21:02:53.887649 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:53.887635 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 21:02:53.887697 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:53.887688 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/50d5a72a-1db9-492b-ab5c-21c12a0efca2-kube-rbac-proxy-cm podName:50d5a72a-1db9-492b-ab5c-21c12a0efca2 nodeName:}" failed. No retries permitted until 2026-07-15 21:02:55.887674563 +0000 UTC m=+80.428987091 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/50d5a72a-1db9-492b-ab5c-21c12a0efca2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mmxvl" (UID: "50d5a72a-1db9-492b-ab5c-21c12a0efca2") : configmap "kube-rbac-proxy" not found Jul 15 21:02:53.887736 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:53.887636 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 21:02:53.887736 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:53.887723 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50d5a72a-1db9-492b-ab5c-21c12a0efca2-insights-runtime-extractor-tls podName:50d5a72a-1db9-492b-ab5c-21c12a0efca2 nodeName:}" failed. No retries permitted until 2026-07-15 21:02:55.887714474 +0000 UTC m=+80.429027016 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/50d5a72a-1db9-492b-ab5c-21c12a0efca2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mmxvl" (UID: "50d5a72a-1db9-492b-ab5c-21c12a0efca2") : secret "insights-runtime-extractor-tls" not found Jul 15 21:02:54.175695 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:54.175672 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jw6xs_05324e48-e436-4e6a-9b35-5d0c661a82a4/dns-node-resolver/0.log" Jul 15 21:02:54.398203 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:54.398163 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-q9qt7" event={"ID":"53c27246-99aa-4570-8f65-0d0f7a0fde09","Type":"ContainerStarted","Data":"6ea592b49482fb9609b46e53c7fbcbc5d2c9c7ced59e57012eac8944c83f1b32"} Jul 15 21:02:54.398616 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:54.398215 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-q9qt7" event={"ID":"53c27246-99aa-4570-8f65-0d0f7a0fde09","Type":"ContainerStarted","Data":"b42856e0fa100f730214fb07d0706edfde570bb658f3620b00b1240f12f3ea1c"} Jul 15 21:02:54.406412 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:54.406382 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:02:55.384626 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:55.384597 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-w2rr5_e24afff0-0ef0-4463-92fc-848462106a1b/node-ca/0.log" Jul 15 21:02:55.650771 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:55.650742 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:02:55.761892 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:55.761858 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:02:55.781823 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:55.781784 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-9mhlt_44d76b49-7290-4217-b4b3-2ec82286aa5c/serve-healthcheck-canary/0.log" Jul 15 21:02:55.904694 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:55.903853 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/50d5a72a-1db9-492b-ab5c-21c12a0efca2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mmxvl\" (UID: \"50d5a72a-1db9-492b-ab5c-21c12a0efca2\") " pod="openshift-insights/insights-runtime-extractor-mmxvl" Jul 15 21:02:55.904694 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:55.903942 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/50d5a72a-1db9-492b-ab5c-21c12a0efca2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mmxvl\" (UID: \"50d5a72a-1db9-492b-ab5c-21c12a0efca2\") " pod="openshift-insights/insights-runtime-extractor-mmxvl" Jul 15 21:02:55.904694 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:55.904091 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 21:02:55.904694 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:55.904148 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50d5a72a-1db9-492b-ab5c-21c12a0efca2-insights-runtime-extractor-tls podName:50d5a72a-1db9-492b-ab5c-21c12a0efca2 nodeName:}" failed. No retries permitted until 2026-07-15 21:02:59.904131419 +0000 UTC m=+84.445443953 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/50d5a72a-1db9-492b-ab5c-21c12a0efca2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mmxvl" (UID: "50d5a72a-1db9-492b-ab5c-21c12a0efca2") : secret "insights-runtime-extractor-tls" not found Jul 15 21:02:55.904694 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:55.904551 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 21:02:55.904694 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:55.904596 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/50d5a72a-1db9-492b-ab5c-21c12a0efca2-kube-rbac-proxy-cm podName:50d5a72a-1db9-492b-ab5c-21c12a0efca2 nodeName:}" failed. No retries permitted until 2026-07-15 21:02:59.904583002 +0000 UTC m=+84.445895538 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/50d5a72a-1db9-492b-ab5c-21c12a0efca2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mmxvl" (UID: "50d5a72a-1db9-492b-ab5c-21c12a0efca2") : configmap "kube-rbac-proxy" not found Jul 15 21:02:56.406086 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:56.406032 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-974pc" event={"ID":"92770f0f-ea66-4ff1-906b-32fff3b4447f","Type":"ContainerStarted","Data":"f3214f9e3f776189a5da2068dd0e6c765676b418850810099ba91a10f33d129e"} Jul 15 21:02:56.406289 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:56.406137 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-974pc" event={"ID":"92770f0f-ea66-4ff1-906b-32fff3b4447f","Type":"ContainerStarted","Data":"b838df383a29b00a6eef4faa63b31b4a2fb923750185018b29afc4c4736ea978"} Jul 15 21:02:56.436146 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:56.436094 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-974pc" podStartSLOduration=2.144856663 podStartE2EDuration="4.436057592s" totalImagesPullingTime="2.291200929s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:02:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:02:56.435362624 +0000 UTC m=+80.976675174" watchObservedRunningTime="2026-07-15 21:02:56.436057592 +0000 UTC m=+80.977370142" Jul 15 21:02:56.436425 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:56.436406 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-q9qt7" podStartSLOduration=3.436401685 podStartE2EDuration="3.436401685s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:02:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:02:54.424733831 +0000 UTC m=+78.966046381" watchObservedRunningTime="2026-07-15 21:02:56.436401685 +0000 UTC m=+80.977714244" Jul 15 21:02:56.815431 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:56.815349 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:02:56.911205 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:56.911170 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:02:57.126828 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:57.126785 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:02:57.560646 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:57.560559 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:02:57.718531 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:57.718495 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:02:57.927955 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:57.927921 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:02:59.061379 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:59.061338 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:02:59.160638 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:59.160595 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:02:59.324162 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:59.324053 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:02:59.547905 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:59.547867 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:02:59.722678 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:59.722645 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:02:59.805027 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:59.804998 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:02:59.936333 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:59.936297 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/50d5a72a-1db9-492b-ab5c-21c12a0efca2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mmxvl\" (UID: \"50d5a72a-1db9-492b-ab5c-21c12a0efca2\") " pod="openshift-insights/insights-runtime-extractor-mmxvl" Jul 15 21:02:59.936499 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:02:59.936363 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/50d5a72a-1db9-492b-ab5c-21c12a0efca2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mmxvl\" (UID: \"50d5a72a-1db9-492b-ab5c-21c12a0efca2\") " pod="openshift-insights/insights-runtime-extractor-mmxvl" Jul 15 21:02:59.936499 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:59.936441 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 21:02:59.936499 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:59.936474 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 21:02:59.936499 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:59.936500 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/50d5a72a-1db9-492b-ab5c-21c12a0efca2-kube-rbac-proxy-cm podName:50d5a72a-1db9-492b-ab5c-21c12a0efca2 nodeName:}" failed. No retries permitted until 2026-07-15 21:03:07.936487463 +0000 UTC m=+92.477799992 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/50d5a72a-1db9-492b-ab5c-21c12a0efca2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mmxvl" (UID: "50d5a72a-1db9-492b-ab5c-21c12a0efca2") : configmap "kube-rbac-proxy" not found Jul 15 21:02:59.936636 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:02:59.936540 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50d5a72a-1db9-492b-ab5c-21c12a0efca2-insights-runtime-extractor-tls podName:50d5a72a-1db9-492b-ab5c-21c12a0efca2 nodeName:}" failed. No retries permitted until 2026-07-15 21:03:07.936521816 +0000 UTC m=+92.477834355 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/50d5a72a-1db9-492b-ab5c-21c12a0efca2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mmxvl" (UID: "50d5a72a-1db9-492b-ab5c-21c12a0efca2") : secret "insights-runtime-extractor-tls" not found Jul 15 21:03:00.178635 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:00.178561 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:00.526245 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:00.526148 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:01.792036 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:01.791998 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:01.890595 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:01.890545 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:01.957441 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:01.957406 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:02.315468 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:02.315431 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:02.515328 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:02.515280 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:03.108993 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:03.108962 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:03.542031 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:03.541937 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:03.916232 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:03.916194 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:04.119460 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:04.119426 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:04.684598 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:04.684565 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:05.543214 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:05.543181 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:05.609989 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:05.609951 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:05.934121 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:05.934092 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:06.293976 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:06.293889 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:06.372451 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:06.372419 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:08.001394 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:08.001366 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/50d5a72a-1db9-492b-ab5c-21c12a0efca2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mmxvl\" (UID: \"50d5a72a-1db9-492b-ab5c-21c12a0efca2\") " pod="openshift-insights/insights-runtime-extractor-mmxvl" Jul 15 21:03:08.001791 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:08.001424 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/50d5a72a-1db9-492b-ab5c-21c12a0efca2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mmxvl\" (UID: \"50d5a72a-1db9-492b-ab5c-21c12a0efca2\") " pod="openshift-insights/insights-runtime-extractor-mmxvl" Jul 15 21:03:08.001879 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:08.001860 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/50d5a72a-1db9-492b-ab5c-21c12a0efca2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mmxvl\" (UID: \"50d5a72a-1db9-492b-ab5c-21c12a0efca2\") " pod="openshift-insights/insights-runtime-extractor-mmxvl" Jul 15 21:03:08.003633 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:08.003617 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/50d5a72a-1db9-492b-ab5c-21c12a0efca2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mmxvl\" (UID: \"50d5a72a-1db9-492b-ab5c-21c12a0efca2\") " pod="openshift-insights/insights-runtime-extractor-mmxvl" Jul 15 21:03:08.070964 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:08.070926 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mmxvl" Jul 15 21:03:08.204053 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:08.204020 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mmxvl"] Jul 15 21:03:08.207051 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:03:08.207026 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50d5a72a_1db9_492b_ab5c_21c12a0efca2.slice/crio-9433bea8f03bbfde888d727d17f41b4a34b6b213201a21128f61434c0c32331c WatchSource:0}: Error finding container 9433bea8f03bbfde888d727d17f41b4a34b6b213201a21128f61434c0c32331c: Status 404 returned error can't find the container with id 9433bea8f03bbfde888d727d17f41b4a34b6b213201a21128f61434c0c32331c Jul 15 21:03:08.442994 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:08.442959 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mmxvl" event={"ID":"50d5a72a-1db9-492b-ab5c-21c12a0efca2","Type":"ContainerStarted","Data":"9433bea8f03bbfde888d727d17f41b4a34b6b213201a21128f61434c0c32331c"} Jul 15 21:03:08.792613 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:08.792501 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:08.894569 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:08.894473 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:13.464591 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:13.464504 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mmxvl" event={"ID":"50d5a72a-1db9-492b-ab5c-21c12a0efca2","Type":"ContainerStarted","Data":"a239d8cfed7ef05b71f76c356392c6bbd36435175703d96907b23a7c9805cbe9"} Jul 15 21:03:15.764139 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:15.764090 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d89392ea-c74a-4ead-b205-afdd71a00a42-registry-tls\") pod \"image-registry-7dc5664c6d-hplh7\" (UID: \"d89392ea-c74a-4ead-b205-afdd71a00a42\") " pod="openshift-image-registry/image-registry-7dc5664c6d-hplh7" Jul 15 21:03:15.764604 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:15.764149 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2511b73d-3502-447a-9175-60d60bd9b862-metrics-tls\") pod \"dns-default-nhh6c\" (UID: \"2511b73d-3502-447a-9175-60d60bd9b862\") " pod="openshift-dns/dns-default-nhh6c" Jul 15 21:03:15.766724 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:15.766694 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2511b73d-3502-447a-9175-60d60bd9b862-metrics-tls\") pod \"dns-default-nhh6c\" (UID: \"2511b73d-3502-447a-9175-60d60bd9b862\") " pod="openshift-dns/dns-default-nhh6c" Jul 15 21:03:15.766901 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:15.766874 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d89392ea-c74a-4ead-b205-afdd71a00a42-registry-tls\") pod \"image-registry-7dc5664c6d-hplh7\" (UID: \"d89392ea-c74a-4ead-b205-afdd71a00a42\") " pod="openshift-image-registry/image-registry-7dc5664c6d-hplh7" Jul 15 21:03:15.770916 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:15.770894 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-d59m9\"" Jul 15 21:03:15.777827 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:15.777801 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7dc5664c6d-hplh7" Jul 15 21:03:15.783976 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:15.783955 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-p4nfw\"" Jul 15 21:03:15.791788 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:15.791772 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nhh6c" Jul 15 21:03:16.757200 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:16.757171 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d7c8ff795-4vzdw"] Jul 15 21:03:16.795110 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:16.795080 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d7c8ff795-4vzdw" Jul 15 21:03:16.807985 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:16.807963 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 15 21:03:16.808114 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:16.808013 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 15 21:03:16.808598 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:16.808584 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 15 21:03:16.809325 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:16.809310 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 15 21:03:16.809371 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:16.809319 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 15 21:03:16.809371 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:16.809339 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 15 21:03:16.809371 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:16.809357 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 15 21:03:16.833017 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:16.832992 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d7c8ff795-4vzdw"] Jul 15 21:03:16.932326 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:16.932296 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:16.974435 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:16.974404 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/24e84f99-b6d6-463c-b41a-e4d75e251fc2-ca\") pod \"cluster-proxy-proxy-agent-6d7c8ff795-4vzdw\" (UID: \"24e84f99-b6d6-463c-b41a-e4d75e251fc2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d7c8ff795-4vzdw" Jul 15 21:03:16.974435 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:16.974437 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/24e84f99-b6d6-463c-b41a-e4d75e251fc2-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6d7c8ff795-4vzdw\" (UID: \"24e84f99-b6d6-463c-b41a-e4d75e251fc2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d7c8ff795-4vzdw" Jul 15 21:03:16.974629 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:16.974489 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/24e84f99-b6d6-463c-b41a-e4d75e251fc2-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6d7c8ff795-4vzdw\" (UID: \"24e84f99-b6d6-463c-b41a-e4d75e251fc2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d7c8ff795-4vzdw" Jul 15 21:03:16.974629 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:16.974546 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jj5j\" (UniqueName: \"kubernetes.io/projected/24e84f99-b6d6-463c-b41a-e4d75e251fc2-kube-api-access-8jj5j\") pod \"cluster-proxy-proxy-agent-6d7c8ff795-4vzdw\" (UID: \"24e84f99-b6d6-463c-b41a-e4d75e251fc2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d7c8ff795-4vzdw" Jul 15 21:03:16.974629 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:16.974577 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/24e84f99-b6d6-463c-b41a-e4d75e251fc2-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6d7c8ff795-4vzdw\" (UID: \"24e84f99-b6d6-463c-b41a-e4d75e251fc2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d7c8ff795-4vzdw" Jul 15 21:03:16.974629 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:16.974608 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/24e84f99-b6d6-463c-b41a-e4d75e251fc2-hub\") pod \"cluster-proxy-proxy-agent-6d7c8ff795-4vzdw\" (UID: \"24e84f99-b6d6-463c-b41a-e4d75e251fc2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d7c8ff795-4vzdw" Jul 15 21:03:17.075509 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:17.075476 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jj5j\" (UniqueName: \"kubernetes.io/projected/24e84f99-b6d6-463c-b41a-e4d75e251fc2-kube-api-access-8jj5j\") pod \"cluster-proxy-proxy-agent-6d7c8ff795-4vzdw\" (UID: \"24e84f99-b6d6-463c-b41a-e4d75e251fc2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d7c8ff795-4vzdw" Jul 15 21:03:17.075665 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:17.075517 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/24e84f99-b6d6-463c-b41a-e4d75e251fc2-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6d7c8ff795-4vzdw\" (UID: \"24e84f99-b6d6-463c-b41a-e4d75e251fc2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d7c8ff795-4vzdw" Jul 15 21:03:17.075665 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:17.075550 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/24e84f99-b6d6-463c-b41a-e4d75e251fc2-hub\") pod \"cluster-proxy-proxy-agent-6d7c8ff795-4vzdw\" (UID: \"24e84f99-b6d6-463c-b41a-e4d75e251fc2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d7c8ff795-4vzdw" Jul 15 21:03:17.075665 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:17.075569 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/24e84f99-b6d6-463c-b41a-e4d75e251fc2-ca\") pod \"cluster-proxy-proxy-agent-6d7c8ff795-4vzdw\" (UID: \"24e84f99-b6d6-463c-b41a-e4d75e251fc2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d7c8ff795-4vzdw" Jul 15 21:03:17.075665 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:17.075589 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/24e84f99-b6d6-463c-b41a-e4d75e251fc2-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6d7c8ff795-4vzdw\" (UID: \"24e84f99-b6d6-463c-b41a-e4d75e251fc2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d7c8ff795-4vzdw" Jul 15 21:03:17.075883 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:17.075799 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/24e84f99-b6d6-463c-b41a-e4d75e251fc2-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6d7c8ff795-4vzdw\" (UID: \"24e84f99-b6d6-463c-b41a-e4d75e251fc2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d7c8ff795-4vzdw" Jul 15 21:03:17.076440 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:17.076414 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/24e84f99-b6d6-463c-b41a-e4d75e251fc2-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6d7c8ff795-4vzdw\" (UID: \"24e84f99-b6d6-463c-b41a-e4d75e251fc2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d7c8ff795-4vzdw" Jul 15 21:03:17.078428 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:17.078370 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/24e84f99-b6d6-463c-b41a-e4d75e251fc2-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6d7c8ff795-4vzdw\" (UID: \"24e84f99-b6d6-463c-b41a-e4d75e251fc2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d7c8ff795-4vzdw" Jul 15 21:03:17.078575 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:17.078554 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/24e84f99-b6d6-463c-b41a-e4d75e251fc2-ca\") pod \"cluster-proxy-proxy-agent-6d7c8ff795-4vzdw\" (UID: \"24e84f99-b6d6-463c-b41a-e4d75e251fc2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d7c8ff795-4vzdw" Jul 15 21:03:17.079172 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:17.079153 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/24e84f99-b6d6-463c-b41a-e4d75e251fc2-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6d7c8ff795-4vzdw\" (UID: \"24e84f99-b6d6-463c-b41a-e4d75e251fc2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d7c8ff795-4vzdw" Jul 15 21:03:17.079258 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:17.079236 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/24e84f99-b6d6-463c-b41a-e4d75e251fc2-hub\") pod \"cluster-proxy-proxy-agent-6d7c8ff795-4vzdw\" (UID: \"24e84f99-b6d6-463c-b41a-e4d75e251fc2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d7c8ff795-4vzdw" Jul 15 21:03:17.110505 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:17.110477 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jj5j\" (UniqueName: \"kubernetes.io/projected/24e84f99-b6d6-463c-b41a-e4d75e251fc2-kube-api-access-8jj5j\") pod \"cluster-proxy-proxy-agent-6d7c8ff795-4vzdw\" (UID: \"24e84f99-b6d6-463c-b41a-e4d75e251fc2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d7c8ff795-4vzdw" Jul 15 21:03:17.114231 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:17.114185 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d7c8ff795-4vzdw" Jul 15 21:03:17.120205 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:17.120178 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:17.151499 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:17.151475 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7dc5664c6d-hplh7"] Jul 15 21:03:17.154241 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:03:17.154212 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd89392ea_c74a_4ead_b205_afdd71a00a42.slice/crio-8539c56ab0ad3cddcf2bd7d56bf91cfbc42e4867cf44aa0a7cbc2d2e002970dc WatchSource:0}: Error finding container 8539c56ab0ad3cddcf2bd7d56bf91cfbc42e4867cf44aa0a7cbc2d2e002970dc: Status 404 returned error can't find the container with id 8539c56ab0ad3cddcf2bd7d56bf91cfbc42e4867cf44aa0a7cbc2d2e002970dc Jul 15 21:03:17.174077 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:17.174015 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nhh6c"] Jul 15 21:03:17.281932 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:17.281858 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d7c8ff795-4vzdw"] Jul 15 21:03:17.358525 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:03:17.358377 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2511b73d_3502_447a_9175_60d60bd9b862.slice/crio-7c905f52cae14144abfeeaffaa6648b8c0173033b2ea7f8ba885c87fcdc2797d WatchSource:0}: Error finding container 7c905f52cae14144abfeeaffaa6648b8c0173033b2ea7f8ba885c87fcdc2797d: Status 404 returned error can't find the container with id 7c905f52cae14144abfeeaffaa6648b8c0173033b2ea7f8ba885c87fcdc2797d Jul 15 21:03:17.359047 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:03:17.359023 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod24e84f99_b6d6_463c_b41a_e4d75e251fc2.slice/crio-7ff43db29525ee483bffed1051cca297db2b04d1d8b093e462a22c4ac17a36e4 WatchSource:0}: Error finding container 7ff43db29525ee483bffed1051cca297db2b04d1d8b093e462a22c4ac17a36e4: Status 404 returned error can't find the container with id 7ff43db29525ee483bffed1051cca297db2b04d1d8b093e462a22c4ac17a36e4 Jul 15 21:03:17.478058 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:17.478027 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d7c8ff795-4vzdw" event={"ID":"24e84f99-b6d6-463c-b41a-e4d75e251fc2","Type":"ContainerStarted","Data":"7ff43db29525ee483bffed1051cca297db2b04d1d8b093e462a22c4ac17a36e4"} Jul 15 21:03:17.479092 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:17.479042 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nhh6c" event={"ID":"2511b73d-3502-447a-9175-60d60bd9b862","Type":"ContainerStarted","Data":"7c905f52cae14144abfeeaffaa6648b8c0173033b2ea7f8ba885c87fcdc2797d"} Jul 15 21:03:17.480352 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:17.480333 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7dc5664c6d-hplh7" event={"ID":"d89392ea-c74a-4ead-b205-afdd71a00a42","Type":"ContainerStarted","Data":"9796de36976bf14e4458e866fed9e3dcd5ea5974955d9cc9d2d4400f4467c4a4"} Jul 15 21:03:17.480434 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:17.480368 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7dc5664c6d-hplh7" event={"ID":"d89392ea-c74a-4ead-b205-afdd71a00a42","Type":"ContainerStarted","Data":"8539c56ab0ad3cddcf2bd7d56bf91cfbc42e4867cf44aa0a7cbc2d2e002970dc"} Jul 15 21:03:18.377646 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:18.376821 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-dg4dh" Jul 15 21:03:18.492306 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:18.492274 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mmxvl" event={"ID":"50d5a72a-1db9-492b-ab5c-21c12a0efca2","Type":"ContainerStarted","Data":"f408e572041a8c48d7d07b909509e4eefd9210b8821e408c35ca380b6cbb3305"} Jul 15 21:03:18.492478 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:18.492366 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7dc5664c6d-hplh7" Jul 15 21:03:18.520793 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:18.519408 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7dc5664c6d-hplh7" podStartSLOduration=101.519386724 podStartE2EDuration="1m41.519386724s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:01:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:03:18.519229376 +0000 UTC m=+103.060541921" watchObservedRunningTime="2026-07-15 21:03:18.519386724 +0000 UTC m=+103.060699275" Jul 15 21:03:19.696465 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:19.696354 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:19.812200 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:19.812163 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:23.514195 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:23.514158 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d7c8ff795-4vzdw" event={"ID":"24e84f99-b6d6-463c-b41a-e4d75e251fc2","Type":"ContainerStarted","Data":"804ae06a94919d5563fa06cd9c8f2a0138edca0b4ab0a08095f320cb2a6f3e09"} Jul 15 21:03:23.514648 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:23.514224 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d7c8ff795-4vzdw" event={"ID":"24e84f99-b6d6-463c-b41a-e4d75e251fc2","Type":"ContainerStarted","Data":"566e7fc59ca7eb120489ecf53de598d9679fc38ce3cb2660c5c54afc9fd4ade5"} Jul 15 21:03:23.514648 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:23.514243 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d7c8ff795-4vzdw" event={"ID":"24e84f99-b6d6-463c-b41a-e4d75e251fc2","Type":"ContainerStarted","Data":"cbb1238b06ee597c8feeb9e0e0d9e2f6049fcb44d6529733585607bac6efb352"} Jul 15 21:03:23.515994 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:23.515970 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mmxvl" event={"ID":"50d5a72a-1db9-492b-ab5c-21c12a0efca2","Type":"ContainerStarted","Data":"1df40e757ab6458f78202479f0027393425010e2e03be08a849484b49d419c2d"} Jul 15 21:03:23.517530 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:23.517509 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nhh6c" event={"ID":"2511b73d-3502-447a-9175-60d60bd9b862","Type":"ContainerStarted","Data":"1f17147becf998d3af062d2e766a6fd059bb51852c8743194bb1925d7e399a53"} Jul 15 21:03:23.517636 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:23.517550 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nhh6c" event={"ID":"2511b73d-3502-447a-9175-60d60bd9b862","Type":"ContainerStarted","Data":"b298503a24c9b046734cadc34a371e2557e426fbedb69681ef9ed042a315fcbe"} Jul 15 21:03:23.517636 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:23.517576 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-nhh6c" Jul 15 21:03:23.542347 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:23.542294 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d7c8ff795-4vzdw" podStartSLOduration=1.81796548 podStartE2EDuration="7.542278033s" totalImagesPullingTime="5.724312553s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:03:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:03:23.54201629 +0000 UTC m=+108.083328840" watchObservedRunningTime="2026-07-15 21:03:23.542278033 +0000 UTC m=+108.083590582" Jul 15 21:03:23.564550 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:23.564505 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-nhh6c" podStartSLOduration=66.859730585 podStartE2EDuration="1m12.56448909s" totalImagesPullingTime="5.704758505s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:02:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:03:23.563646207 +0000 UTC m=+108.104958756" watchObservedRunningTime="2026-07-15 21:03:23.56448909 +0000 UTC m=+108.105801643" Jul 15 21:03:23.589371 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:23.589310 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-mmxvl" podStartSLOduration=16.876973187 podStartE2EDuration="31.589292916s" totalImagesPullingTime="14.712319729s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 21:02:52 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:03:23.588584849 +0000 UTC m=+108.129897399" watchObservedRunningTime="2026-07-15 21:03:23.589292916 +0000 UTC m=+108.130605465" Jul 15 21:03:23.731080 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:23.731029 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:23.824372 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:23.824285 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:23.925055 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:23.925018 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:24.019791 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:24.019756 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:25.783851 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:25.783815 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:25.886146 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:25.886108 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:26.954430 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:26.954396 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:27.039867 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:27.039834 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:27.111209 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:27.111178 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:27.828197 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:27.828161 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:27.900404 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:27.900372 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:28.943613 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:28.943572 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:30.539008 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:30.538971 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:30.656348 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:30.656315 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:30.795220 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:30.795136 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:30.964677 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:30.964647 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:31.137316 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:31.137281 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:31.231084 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:31.231032 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:31.455569 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:31.455491 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-5ngtg"] Jul 15 21:03:31.460025 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:31.460002 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5ngtg" Jul 15 21:03:31.464739 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:31.464716 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 15 21:03:31.466141 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:31.466113 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 15 21:03:31.466141 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:31.466131 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 15 21:03:31.466288 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:31.466135 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-lhkbt\"" Jul 15 21:03:31.466288 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:31.466221 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 15 21:03:31.466288 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:31.466261 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 15 21:03:31.466844 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:31.466596 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 15 21:03:31.483230 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:31.483203 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9331cdf3-604a-475b-a036-afe088bdea64-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5ngtg\" (UID: \"9331cdf3-604a-475b-a036-afe088bdea64\") " pod="openshift-monitoring/node-exporter-5ngtg" Jul 15 21:03:31.483362 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:31.483248 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9331cdf3-604a-475b-a036-afe088bdea64-root\") pod \"node-exporter-5ngtg\" (UID: \"9331cdf3-604a-475b-a036-afe088bdea64\") " pod="openshift-monitoring/node-exporter-5ngtg" Jul 15 21:03:31.483362 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:31.483268 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9331cdf3-604a-475b-a036-afe088bdea64-node-exporter-textfile\") pod \"node-exporter-5ngtg\" (UID: \"9331cdf3-604a-475b-a036-afe088bdea64\") " pod="openshift-monitoring/node-exporter-5ngtg" Jul 15 21:03:31.483362 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:31.483310 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9331cdf3-604a-475b-a036-afe088bdea64-node-exporter-wtmp\") pod \"node-exporter-5ngtg\" (UID: \"9331cdf3-604a-475b-a036-afe088bdea64\") " pod="openshift-monitoring/node-exporter-5ngtg" Jul 15 21:03:31.483362 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:31.483327 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9331cdf3-604a-475b-a036-afe088bdea64-node-exporter-tls\") pod \"node-exporter-5ngtg\" (UID: \"9331cdf3-604a-475b-a036-afe088bdea64\") " pod="openshift-monitoring/node-exporter-5ngtg" Jul 15 21:03:31.483362 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:31.483350 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9331cdf3-604a-475b-a036-afe088bdea64-sys\") pod \"node-exporter-5ngtg\" (UID: \"9331cdf3-604a-475b-a036-afe088bdea64\") " pod="openshift-monitoring/node-exporter-5ngtg" Jul 15 21:03:31.483627 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:31.483384 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdx4w\" (UniqueName: \"kubernetes.io/projected/9331cdf3-604a-475b-a036-afe088bdea64-kube-api-access-fdx4w\") pod \"node-exporter-5ngtg\" (UID: \"9331cdf3-604a-475b-a036-afe088bdea64\") " pod="openshift-monitoring/node-exporter-5ngtg" Jul 15 21:03:31.483627 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:31.483426 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9331cdf3-604a-475b-a036-afe088bdea64-metrics-client-ca\") pod \"node-exporter-5ngtg\" (UID: \"9331cdf3-604a-475b-a036-afe088bdea64\") " pod="openshift-monitoring/node-exporter-5ngtg" Jul 15 21:03:31.483627 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:31.483470 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9331cdf3-604a-475b-a036-afe088bdea64-node-exporter-accelerators-collector-config\") pod \"node-exporter-5ngtg\" (UID: \"9331cdf3-604a-475b-a036-afe088bdea64\") " pod="openshift-monitoring/node-exporter-5ngtg" Jul 15 21:03:31.584500 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:31.584461 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9331cdf3-604a-475b-a036-afe088bdea64-node-exporter-textfile\") pod \"node-exporter-5ngtg\" (UID: \"9331cdf3-604a-475b-a036-afe088bdea64\") " pod="openshift-monitoring/node-exporter-5ngtg" Jul 15 21:03:31.584500 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:31.584504 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9331cdf3-604a-475b-a036-afe088bdea64-node-exporter-wtmp\") pod \"node-exporter-5ngtg\" (UID: \"9331cdf3-604a-475b-a036-afe088bdea64\") " pod="openshift-monitoring/node-exporter-5ngtg" Jul 15 21:03:31.585054 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:31.584532 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9331cdf3-604a-475b-a036-afe088bdea64-node-exporter-tls\") pod \"node-exporter-5ngtg\" (UID: \"9331cdf3-604a-475b-a036-afe088bdea64\") " pod="openshift-monitoring/node-exporter-5ngtg" Jul 15 21:03:31.585054 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:31.584567 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9331cdf3-604a-475b-a036-afe088bdea64-sys\") pod \"node-exporter-5ngtg\" (UID: \"9331cdf3-604a-475b-a036-afe088bdea64\") " pod="openshift-monitoring/node-exporter-5ngtg" Jul 15 21:03:31.585054 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:31.584588 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdx4w\" (UniqueName: \"kubernetes.io/projected/9331cdf3-604a-475b-a036-afe088bdea64-kube-api-access-fdx4w\") pod \"node-exporter-5ngtg\" (UID: \"9331cdf3-604a-475b-a036-afe088bdea64\") " pod="openshift-monitoring/node-exporter-5ngtg" Jul 15 21:03:31.585054 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:31.584615 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9331cdf3-604a-475b-a036-afe088bdea64-metrics-client-ca\") pod \"node-exporter-5ngtg\" (UID: \"9331cdf3-604a-475b-a036-afe088bdea64\") " pod="openshift-monitoring/node-exporter-5ngtg" Jul 15 21:03:31.585054 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:31.584665 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9331cdf3-604a-475b-a036-afe088bdea64-sys\") pod \"node-exporter-5ngtg\" (UID: \"9331cdf3-604a-475b-a036-afe088bdea64\") " pod="openshift-monitoring/node-exporter-5ngtg" Jul 15 21:03:31.585054 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:31.584665 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9331cdf3-604a-475b-a036-afe088bdea64-node-exporter-wtmp\") pod \"node-exporter-5ngtg\" (UID: \"9331cdf3-604a-475b-a036-afe088bdea64\") " pod="openshift-monitoring/node-exporter-5ngtg" Jul 15 21:03:31.585054 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:31.584759 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9331cdf3-604a-475b-a036-afe088bdea64-node-exporter-accelerators-collector-config\") pod \"node-exporter-5ngtg\" (UID: \"9331cdf3-604a-475b-a036-afe088bdea64\") " pod="openshift-monitoring/node-exporter-5ngtg" Jul 15 21:03:31.585054 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:31.584816 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9331cdf3-604a-475b-a036-afe088bdea64-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5ngtg\" (UID: \"9331cdf3-604a-475b-a036-afe088bdea64\") " pod="openshift-monitoring/node-exporter-5ngtg" Jul 15 21:03:31.585054 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:31.584827 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9331cdf3-604a-475b-a036-afe088bdea64-node-exporter-textfile\") pod \"node-exporter-5ngtg\" (UID: \"9331cdf3-604a-475b-a036-afe088bdea64\") " pod="openshift-monitoring/node-exporter-5ngtg" Jul 15 21:03:31.585054 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:31.584880 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9331cdf3-604a-475b-a036-afe088bdea64-root\") pod \"node-exporter-5ngtg\" (UID: \"9331cdf3-604a-475b-a036-afe088bdea64\") " pod="openshift-monitoring/node-exporter-5ngtg" Jul 15 21:03:31.585054 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:31.584952 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9331cdf3-604a-475b-a036-afe088bdea64-root\") pod \"node-exporter-5ngtg\" (UID: \"9331cdf3-604a-475b-a036-afe088bdea64\") " pod="openshift-monitoring/node-exporter-5ngtg" Jul 15 21:03:31.585518 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:31.585274 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9331cdf3-604a-475b-a036-afe088bdea64-node-exporter-accelerators-collector-config\") pod \"node-exporter-5ngtg\" (UID: \"9331cdf3-604a-475b-a036-afe088bdea64\") " pod="openshift-monitoring/node-exporter-5ngtg" Jul 15 21:03:31.585518 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:31.585325 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9331cdf3-604a-475b-a036-afe088bdea64-metrics-client-ca\") pod \"node-exporter-5ngtg\" (UID: \"9331cdf3-604a-475b-a036-afe088bdea64\") " pod="openshift-monitoring/node-exporter-5ngtg" Jul 15 21:03:31.586922 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:31.586903 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9331cdf3-604a-475b-a036-afe088bdea64-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5ngtg\" (UID: \"9331cdf3-604a-475b-a036-afe088bdea64\") " pod="openshift-monitoring/node-exporter-5ngtg" Jul 15 21:03:31.587031 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:31.586998 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9331cdf3-604a-475b-a036-afe088bdea64-node-exporter-tls\") pod \"node-exporter-5ngtg\" (UID: \"9331cdf3-604a-475b-a036-afe088bdea64\") " pod="openshift-monitoring/node-exporter-5ngtg" Jul 15 21:03:31.600599 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:31.600572 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdx4w\" (UniqueName: \"kubernetes.io/projected/9331cdf3-604a-475b-a036-afe088bdea64-kube-api-access-fdx4w\") pod \"node-exporter-5ngtg\" (UID: \"9331cdf3-604a-475b-a036-afe088bdea64\") " pod="openshift-monitoring/node-exporter-5ngtg" Jul 15 21:03:31.768901 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:31.768806 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5ngtg" Jul 15 21:03:31.780245 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:03:31.780212 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9331cdf3_604a_475b_a036_afe088bdea64.slice/crio-0daaa5269e9dc159298db4764ab7b3e3d6507bc5b28022c4edf29e15d95a0621 WatchSource:0}: Error finding container 0daaa5269e9dc159298db4764ab7b3e3d6507bc5b28022c4edf29e15d95a0621: Status 404 returned error can't find the container with id 0daaa5269e9dc159298db4764ab7b3e3d6507bc5b28022c4edf29e15d95a0621 Jul 15 21:03:31.873916 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:31.873880 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:31.977390 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:31.977340 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:32.092025 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:32.091930 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:32.194992 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:32.194956 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:32.325311 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:32.325279 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:32.425270 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:32.425216 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:32.525489 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:32.525452 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:32.550568 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:32.550538 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5ngtg" event={"ID":"9331cdf3-604a-475b-a036-afe088bdea64","Type":"ContainerStarted","Data":"0daaa5269e9dc159298db4764ab7b3e3d6507bc5b28022c4edf29e15d95a0621"} Jul 15 21:03:32.662864 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:32.662827 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:33.193551 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:33.193511 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:33.301879 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:33.301843 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:33.524031 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:33.523956 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-nhh6c" Jul 15 21:03:33.556159 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:33.556129 2579 generic.go:356] "Generic (PLEG): container finished" podID="9331cdf3-604a-475b-a036-afe088bdea64" containerID="721479394bfc90dd1bdbca1aed7de14cfcae4a22352689c85afb0b5bd3e4b351" exitCode=0 Jul 15 21:03:33.556351 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:33.556173 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5ngtg" event={"ID":"9331cdf3-604a-475b-a036-afe088bdea64","Type":"ContainerDied","Data":"721479394bfc90dd1bdbca1aed7de14cfcae4a22352689c85afb0b5bd3e4b351"} Jul 15 21:03:34.453642 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:34.453600 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:34.541003 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:34.540970 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:34.562281 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:34.562248 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5ngtg" event={"ID":"9331cdf3-604a-475b-a036-afe088bdea64","Type":"ContainerStarted","Data":"b13e39729c366a22d1b0944bf5039dbbce4415ed1483a66a8008a32b970eb9af"} Jul 15 21:03:34.562402 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:34.562309 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5ngtg" event={"ID":"9331cdf3-604a-475b-a036-afe088bdea64","Type":"ContainerStarted","Data":"3fe67c883b5e666a4e73a308e47bd59eef2b07069caae59e8782a13054986013"} Jul 15 21:03:34.593597 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:34.593539 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-5ngtg" podStartSLOduration=2.511291447 podStartE2EDuration="3.593515082s" totalImagesPullingTime="1.018103702s" totalInitContainerRuntime="64.119933ms" isStatefulPod=true podCreationTimestamp="2026-07-15 21:03:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:03:34.593082495 +0000 UTC m=+119.134395035" watchObservedRunningTime="2026-07-15 21:03:34.593515082 +0000 UTC m=+119.134827631" Jul 15 21:03:34.776516 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:34.776426 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:34.878579 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:34.878541 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:36.172454 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:36.172422 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:36.271334 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:36.271300 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:36.378869 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:36.378837 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:36.946462 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:36.946406 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:37.066784 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:37.066747 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:37.846550 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:37.846518 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:38.785091 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:38.785032 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:38.881256 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:38.881221 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:39.501145 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:39.501108 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7dc5664c6d-hplh7" Jul 15 21:03:39.601198 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:39.601165 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7dc5664c6d-hplh7"] Jul 15 21:03:39.744573 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:39.744540 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:39.839786 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:39.839683 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:39.941283 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:39.941247 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:42.751624 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:42.751594 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:42.853033 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:42.852983 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:42.923421 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:42.923387 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:46.097018 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:46.096978 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0dafa58d-15c7-46c9-81bd-c3c86bbf8e42-metrics-certs\") pod \"network-metrics-daemon-r59ft\" (UID: \"0dafa58d-15c7-46c9-81bd-c3c86bbf8e42\") " pod="openshift-multus/network-metrics-daemon-r59ft" Jul 15 21:03:46.099231 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:46.099208 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0dafa58d-15c7-46c9-81bd-c3c86bbf8e42-metrics-certs\") pod \"network-metrics-daemon-r59ft\" (UID: \"0dafa58d-15c7-46c9-81bd-c3c86bbf8e42\") " pod="openshift-multus/network-metrics-daemon-r59ft" Jul 15 21:03:46.213313 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:46.213283 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rwswm\"" Jul 15 21:03:46.221010 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:46.220989 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r59ft" Jul 15 21:03:46.405537 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:46.405514 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-r59ft"] Jul 15 21:03:46.407995 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:03:46.407962 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0dafa58d_15c7_46c9_81bd_c3c86bbf8e42.slice/crio-d9096fd729dee8d050782e662719a2f6dae93b86302d200c87babc1ebd74215b WatchSource:0}: Error finding container d9096fd729dee8d050782e662719a2f6dae93b86302d200c87babc1ebd74215b: Status 404 returned error can't find the container with id d9096fd729dee8d050782e662719a2f6dae93b86302d200c87babc1ebd74215b Jul 15 21:03:46.603453 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:46.603411 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-r59ft" event={"ID":"0dafa58d-15c7-46c9-81bd-c3c86bbf8e42","Type":"ContainerStarted","Data":"d9096fd729dee8d050782e662719a2f6dae93b86302d200c87babc1ebd74215b"} Jul 15 21:03:48.611929 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:48.611890 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-r59ft" event={"ID":"0dafa58d-15c7-46c9-81bd-c3c86bbf8e42","Type":"ContainerStarted","Data":"8a972d79d7dcca7d02259e7c67720b2338a7683b1974d70babf70d096de6765d"} Jul 15 21:03:48.612411 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:48.611944 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-r59ft" event={"ID":"0dafa58d-15c7-46c9-81bd-c3c86bbf8e42","Type":"ContainerStarted","Data":"13970b37a9fa0f0d4b22fff27be34b427b477b617918ac1a2b9f3f88c8c53047"} Jul 15 21:03:48.662713 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:48.662645 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-r59ft" podStartSLOduration=130.483432275 podStartE2EDuration="2m11.662622829s" totalImagesPullingTime="1.179190554s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:01:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:03:48.662176143 +0000 UTC m=+133.203488694" watchObservedRunningTime="2026-07-15 21:03:48.662622829 +0000 UTC m=+133.203935379" Jul 15 21:03:48.771952 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:48.771922 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:48.872234 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:48.872153 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:48.946727 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:48.946689 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:51.735712 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:51.735678 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:51.822919 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:51.822876 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:52.849256 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:52.849219 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:52.920455 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:52.920421 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:53.774410 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:53.774377 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:53.875377 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:53.875348 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:55.634485 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:55.634450 2579 generic.go:356] "Generic (PLEG): container finished" podID="835f5980-1870-4052-af02-3e3a4653501c" containerID="10ee1ccd2139578251eb730ee2013584f29af870fd0e6290039dc24a3d2199c1" exitCode=0 Jul 15 21:03:55.634853 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:55.634507 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-p85nr" event={"ID":"835f5980-1870-4052-af02-3e3a4653501c","Type":"ContainerDied","Data":"10ee1ccd2139578251eb730ee2013584f29af870fd0e6290039dc24a3d2199c1"} Jul 15 21:03:55.634853 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:55.634793 2579 scope.go:117] "RemoveContainer" containerID="10ee1ccd2139578251eb730ee2013584f29af870fd0e6290039dc24a3d2199c1" Jul 15 21:03:55.814419 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:55.814382 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:03:56.639620 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:56.639588 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-p85nr" event={"ID":"835f5980-1870-4052-af02-3e3a4653501c","Type":"ContainerStarted","Data":"1a9a93b02453c30c35ebe61fd0e63b395c5fd285d4c16bf4a769508b556b97ba"} Jul 15 21:03:59.984189 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:03:59.984152 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:04:00.084323 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:00.084267 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:04:00.624493 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:00.624453 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:04:00.700371 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:00.700338 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:04:02.101094 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:02.101052 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:04:02.204086 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:02.204036 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:04:02.273461 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:02.273425 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:04:04.621326 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:04.621280 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7dc5664c6d-hplh7" podUID="d89392ea-c74a-4ead-b205-afdd71a00a42" containerName="registry" containerID="cri-o://9796de36976bf14e4458e866fed9e3dcd5ea5974955d9cc9d2d4400f4467c4a4" gracePeriod=30 Jul 15 21:04:04.862846 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:04.862821 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7dc5664c6d-hplh7" Jul 15 21:04:04.938702 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:04.938622 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d89392ea-c74a-4ead-b205-afdd71a00a42-image-registry-private-configuration\") pod \"d89392ea-c74a-4ead-b205-afdd71a00a42\" (UID: \"d89392ea-c74a-4ead-b205-afdd71a00a42\") " Jul 15 21:04:04.938702 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:04.938661 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d89392ea-c74a-4ead-b205-afdd71a00a42-registry-tls\") pod \"d89392ea-c74a-4ead-b205-afdd71a00a42\" (UID: \"d89392ea-c74a-4ead-b205-afdd71a00a42\") " Jul 15 21:04:04.938702 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:04.938698 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d89392ea-c74a-4ead-b205-afdd71a00a42-ca-trust-extracted\") pod \"d89392ea-c74a-4ead-b205-afdd71a00a42\" (UID: \"d89392ea-c74a-4ead-b205-afdd71a00a42\") " Jul 15 21:04:04.938958 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:04.938732 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d89392ea-c74a-4ead-b205-afdd71a00a42-trusted-ca\") pod \"d89392ea-c74a-4ead-b205-afdd71a00a42\" (UID: \"d89392ea-c74a-4ead-b205-afdd71a00a42\") " Jul 15 21:04:04.938958 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:04.938754 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d89392ea-c74a-4ead-b205-afdd71a00a42-registry-certificates\") pod \"d89392ea-c74a-4ead-b205-afdd71a00a42\" (UID: \"d89392ea-c74a-4ead-b205-afdd71a00a42\") " Jul 15 21:04:04.938958 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:04.938797 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4ktn5\" (UniqueName: \"kubernetes.io/projected/d89392ea-c74a-4ead-b205-afdd71a00a42-kube-api-access-4ktn5\") pod \"d89392ea-c74a-4ead-b205-afdd71a00a42\" (UID: \"d89392ea-c74a-4ead-b205-afdd71a00a42\") " Jul 15 21:04:04.938958 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:04.938826 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d89392ea-c74a-4ead-b205-afdd71a00a42-bound-sa-token\") pod \"d89392ea-c74a-4ead-b205-afdd71a00a42\" (UID: \"d89392ea-c74a-4ead-b205-afdd71a00a42\") " Jul 15 21:04:04.938958 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:04.938843 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d89392ea-c74a-4ead-b205-afdd71a00a42-installation-pull-secrets\") pod \"d89392ea-c74a-4ead-b205-afdd71a00a42\" (UID: \"d89392ea-c74a-4ead-b205-afdd71a00a42\") " Jul 15 21:04:04.939297 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:04.939256 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d89392ea-c74a-4ead-b205-afdd71a00a42-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "d89392ea-c74a-4ead-b205-afdd71a00a42" (UID: "d89392ea-c74a-4ead-b205-afdd71a00a42"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 21:04:04.939421 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:04.939289 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d89392ea-c74a-4ead-b205-afdd71a00a42-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "d89392ea-c74a-4ead-b205-afdd71a00a42" (UID: "d89392ea-c74a-4ead-b205-afdd71a00a42"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 21:04:04.941413 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:04.941386 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d89392ea-c74a-4ead-b205-afdd71a00a42-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "d89392ea-c74a-4ead-b205-afdd71a00a42" (UID: "d89392ea-c74a-4ead-b205-afdd71a00a42"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:04:04.941519 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:04.941390 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d89392ea-c74a-4ead-b205-afdd71a00a42-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "d89392ea-c74a-4ead-b205-afdd71a00a42" (UID: "d89392ea-c74a-4ead-b205-afdd71a00a42"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 21:04:04.941519 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:04.941441 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d89392ea-c74a-4ead-b205-afdd71a00a42-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "d89392ea-c74a-4ead-b205-afdd71a00a42" (UID: "d89392ea-c74a-4ead-b205-afdd71a00a42"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:04:04.941597 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:04.941512 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d89392ea-c74a-4ead-b205-afdd71a00a42-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "d89392ea-c74a-4ead-b205-afdd71a00a42" (UID: "d89392ea-c74a-4ead-b205-afdd71a00a42"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 21:04:04.941638 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:04.941612 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d89392ea-c74a-4ead-b205-afdd71a00a42-kube-api-access-4ktn5" (OuterVolumeSpecName: "kube-api-access-4ktn5") pod "d89392ea-c74a-4ead-b205-afdd71a00a42" (UID: "d89392ea-c74a-4ead-b205-afdd71a00a42"). InnerVolumeSpecName "kube-api-access-4ktn5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:04:04.947590 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:04.947565 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d89392ea-c74a-4ead-b205-afdd71a00a42-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "d89392ea-c74a-4ead-b205-afdd71a00a42" (UID: "d89392ea-c74a-4ead-b205-afdd71a00a42"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:04:05.039905 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:05.039851 2579 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d89392ea-c74a-4ead-b205-afdd71a00a42-registry-certificates\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:04:05.039905 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:05.039896 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4ktn5\" (UniqueName: \"kubernetes.io/projected/d89392ea-c74a-4ead-b205-afdd71a00a42-kube-api-access-4ktn5\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:04:05.039905 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:05.039906 2579 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d89392ea-c74a-4ead-b205-afdd71a00a42-bound-sa-token\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:04:05.039905 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:05.039915 2579 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d89392ea-c74a-4ead-b205-afdd71a00a42-installation-pull-secrets\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:04:05.040206 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:05.039926 2579 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d89392ea-c74a-4ead-b205-afdd71a00a42-image-registry-private-configuration\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:04:05.040206 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:05.039936 2579 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d89392ea-c74a-4ead-b205-afdd71a00a42-registry-tls\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:04:05.040206 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:05.039944 2579 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d89392ea-c74a-4ead-b205-afdd71a00a42-ca-trust-extracted\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:04:05.040206 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:05.039951 2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d89392ea-c74a-4ead-b205-afdd71a00a42-trusted-ca\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:04:05.671727 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:05.671688 2579 generic.go:356] "Generic (PLEG): container finished" podID="d89392ea-c74a-4ead-b205-afdd71a00a42" containerID="9796de36976bf14e4458e866fed9e3dcd5ea5974955d9cc9d2d4400f4467c4a4" exitCode=0 Jul 15 21:04:05.672134 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:05.671751 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7dc5664c6d-hplh7" Jul 15 21:04:05.672134 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:05.671777 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7dc5664c6d-hplh7" event={"ID":"d89392ea-c74a-4ead-b205-afdd71a00a42","Type":"ContainerDied","Data":"9796de36976bf14e4458e866fed9e3dcd5ea5974955d9cc9d2d4400f4467c4a4"} Jul 15 21:04:05.672134 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:05.671863 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7dc5664c6d-hplh7" event={"ID":"d89392ea-c74a-4ead-b205-afdd71a00a42","Type":"ContainerDied","Data":"8539c56ab0ad3cddcf2bd7d56bf91cfbc42e4867cf44aa0a7cbc2d2e002970dc"} Jul 15 21:04:05.672134 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:05.671890 2579 scope.go:117] "RemoveContainer" containerID="9796de36976bf14e4458e866fed9e3dcd5ea5974955d9cc9d2d4400f4467c4a4" Jul 15 21:04:05.681702 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:05.681680 2579 scope.go:117] "RemoveContainer" containerID="9796de36976bf14e4458e866fed9e3dcd5ea5974955d9cc9d2d4400f4467c4a4" Jul 15 21:04:05.682022 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:04:05.682000 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9796de36976bf14e4458e866fed9e3dcd5ea5974955d9cc9d2d4400f4467c4a4\": container with ID starting with 9796de36976bf14e4458e866fed9e3dcd5ea5974955d9cc9d2d4400f4467c4a4 not found: ID does not exist" containerID="9796de36976bf14e4458e866fed9e3dcd5ea5974955d9cc9d2d4400f4467c4a4" Jul 15 21:04:05.682112 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:05.682031 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9796de36976bf14e4458e866fed9e3dcd5ea5974955d9cc9d2d4400f4467c4a4"} err="failed to get container status \"9796de36976bf14e4458e866fed9e3dcd5ea5974955d9cc9d2d4400f4467c4a4\": rpc error: code = NotFound desc = could not find container \"9796de36976bf14e4458e866fed9e3dcd5ea5974955d9cc9d2d4400f4467c4a4\": container with ID starting with 9796de36976bf14e4458e866fed9e3dcd5ea5974955d9cc9d2d4400f4467c4a4 not found: ID does not exist" Jul 15 21:04:05.695284 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:05.695256 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7dc5664c6d-hplh7"] Jul 15 21:04:05.699777 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:05.699751 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7dc5664c6d-hplh7"] Jul 15 21:04:05.996432 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:05.996359 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d89392ea-c74a-4ead-b205-afdd71a00a42" path="/var/lib/kubelet/pods/d89392ea-c74a-4ead-b205-afdd71a00a42/volumes" Jul 15 21:04:08.784689 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:08.784652 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:04:08.880556 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:08.880522 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:04:23.772874 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:23.772821 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:04:23.848789 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:23.848755 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:04:50.787251 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:50.787215 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:04:50.905879 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:50.905845 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:04:55.134517 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:55.134485 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:04:55.917604 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:55.917573 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5494675598-c64bz"] Jul 15 21:04:55.917916 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:55.917899 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d89392ea-c74a-4ead-b205-afdd71a00a42" containerName="registry" Jul 15 21:04:55.918002 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:55.917919 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d89392ea-c74a-4ead-b205-afdd71a00a42" containerName="registry" Jul 15 21:04:55.918002 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:55.917999 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d89392ea-c74a-4ead-b205-afdd71a00a42" containerName="registry" Jul 15 21:04:55.920496 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:55.920473 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5494675598-c64bz" Jul 15 21:04:55.931714 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:55.931696 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 15 21:04:55.931838 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:55.931825 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 15 21:04:55.931977 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:55.931961 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 15 21:04:55.932275 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:55.932262 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 15 21:04:55.932628 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:55.932616 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-brbgh\"" Jul 15 21:04:55.932697 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:55.932685 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 15 21:04:55.944013 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:55.943989 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5494675598-c64bz"] Jul 15 21:04:55.944692 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:55.944668 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 15 21:04:56.100970 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:56.100934 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/111ca5e6-b48c-44ea-a3ae-52f1a9572b11-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5494675598-c64bz\" (UID: \"111ca5e6-b48c-44ea-a3ae-52f1a9572b11\") " pod="openshift-monitoring/telemeter-client-5494675598-c64bz" Jul 15 21:04:56.100970 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:56.100970 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4phpm\" (UniqueName: \"kubernetes.io/projected/111ca5e6-b48c-44ea-a3ae-52f1a9572b11-kube-api-access-4phpm\") pod \"telemeter-client-5494675598-c64bz\" (UID: \"111ca5e6-b48c-44ea-a3ae-52f1a9572b11\") " pod="openshift-monitoring/telemeter-client-5494675598-c64bz" Jul 15 21:04:56.101268 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:56.100990 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/111ca5e6-b48c-44ea-a3ae-52f1a9572b11-metrics-client-ca\") pod \"telemeter-client-5494675598-c64bz\" (UID: \"111ca5e6-b48c-44ea-a3ae-52f1a9572b11\") " pod="openshift-monitoring/telemeter-client-5494675598-c64bz" Jul 15 21:04:56.101268 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:56.101014 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/111ca5e6-b48c-44ea-a3ae-52f1a9572b11-telemeter-client-tls\") pod \"telemeter-client-5494675598-c64bz\" (UID: \"111ca5e6-b48c-44ea-a3ae-52f1a9572b11\") " pod="openshift-monitoring/telemeter-client-5494675598-c64bz" Jul 15 21:04:56.101268 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:56.101036 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/111ca5e6-b48c-44ea-a3ae-52f1a9572b11-secret-telemeter-client\") pod \"telemeter-client-5494675598-c64bz\" (UID: \"111ca5e6-b48c-44ea-a3ae-52f1a9572b11\") " pod="openshift-monitoring/telemeter-client-5494675598-c64bz" Jul 15 21:04:56.101268 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:56.101090 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/111ca5e6-b48c-44ea-a3ae-52f1a9572b11-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5494675598-c64bz\" (UID: \"111ca5e6-b48c-44ea-a3ae-52f1a9572b11\") " pod="openshift-monitoring/telemeter-client-5494675598-c64bz" Jul 15 21:04:56.101268 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:56.101133 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/111ca5e6-b48c-44ea-a3ae-52f1a9572b11-serving-certs-ca-bundle\") pod \"telemeter-client-5494675598-c64bz\" (UID: \"111ca5e6-b48c-44ea-a3ae-52f1a9572b11\") " pod="openshift-monitoring/telemeter-client-5494675598-c64bz" Jul 15 21:04:56.101268 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:56.101177 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/111ca5e6-b48c-44ea-a3ae-52f1a9572b11-federate-client-tls\") pod \"telemeter-client-5494675598-c64bz\" (UID: \"111ca5e6-b48c-44ea-a3ae-52f1a9572b11\") " pod="openshift-monitoring/telemeter-client-5494675598-c64bz" Jul 15 21:04:56.202387 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:56.202302 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/111ca5e6-b48c-44ea-a3ae-52f1a9572b11-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5494675598-c64bz\" (UID: \"111ca5e6-b48c-44ea-a3ae-52f1a9572b11\") " pod="openshift-monitoring/telemeter-client-5494675598-c64bz" Jul 15 21:04:56.202387 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:56.202339 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4phpm\" (UniqueName: \"kubernetes.io/projected/111ca5e6-b48c-44ea-a3ae-52f1a9572b11-kube-api-access-4phpm\") pod \"telemeter-client-5494675598-c64bz\" (UID: \"111ca5e6-b48c-44ea-a3ae-52f1a9572b11\") " pod="openshift-monitoring/telemeter-client-5494675598-c64bz" Jul 15 21:04:56.202387 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:56.202361 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/111ca5e6-b48c-44ea-a3ae-52f1a9572b11-metrics-client-ca\") pod \"telemeter-client-5494675598-c64bz\" (UID: \"111ca5e6-b48c-44ea-a3ae-52f1a9572b11\") " pod="openshift-monitoring/telemeter-client-5494675598-c64bz" Jul 15 21:04:56.202877 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:56.202418 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/111ca5e6-b48c-44ea-a3ae-52f1a9572b11-telemeter-client-tls\") pod \"telemeter-client-5494675598-c64bz\" (UID: \"111ca5e6-b48c-44ea-a3ae-52f1a9572b11\") " pod="openshift-monitoring/telemeter-client-5494675598-c64bz" Jul 15 21:04:56.202877 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:56.202461 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/111ca5e6-b48c-44ea-a3ae-52f1a9572b11-secret-telemeter-client\") pod \"telemeter-client-5494675598-c64bz\" (UID: \"111ca5e6-b48c-44ea-a3ae-52f1a9572b11\") " pod="openshift-monitoring/telemeter-client-5494675598-c64bz" Jul 15 21:04:56.202877 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:56.202497 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/111ca5e6-b48c-44ea-a3ae-52f1a9572b11-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5494675598-c64bz\" (UID: \"111ca5e6-b48c-44ea-a3ae-52f1a9572b11\") " pod="openshift-monitoring/telemeter-client-5494675598-c64bz" Jul 15 21:04:56.202877 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:56.202515 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/111ca5e6-b48c-44ea-a3ae-52f1a9572b11-serving-certs-ca-bundle\") pod \"telemeter-client-5494675598-c64bz\" (UID: \"111ca5e6-b48c-44ea-a3ae-52f1a9572b11\") " pod="openshift-monitoring/telemeter-client-5494675598-c64bz" Jul 15 21:04:56.202877 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:56.202556 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/111ca5e6-b48c-44ea-a3ae-52f1a9572b11-federate-client-tls\") pod \"telemeter-client-5494675598-c64bz\" (UID: \"111ca5e6-b48c-44ea-a3ae-52f1a9572b11\") " pod="openshift-monitoring/telemeter-client-5494675598-c64bz" Jul 15 21:04:56.203202 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:56.203172 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/111ca5e6-b48c-44ea-a3ae-52f1a9572b11-metrics-client-ca\") pod \"telemeter-client-5494675598-c64bz\" (UID: \"111ca5e6-b48c-44ea-a3ae-52f1a9572b11\") " pod="openshift-monitoring/telemeter-client-5494675598-c64bz" Jul 15 21:04:56.203202 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:56.203179 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/111ca5e6-b48c-44ea-a3ae-52f1a9572b11-serving-certs-ca-bundle\") pod \"telemeter-client-5494675598-c64bz\" (UID: \"111ca5e6-b48c-44ea-a3ae-52f1a9572b11\") " pod="openshift-monitoring/telemeter-client-5494675598-c64bz" Jul 15 21:04:56.203809 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:56.203782 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/111ca5e6-b48c-44ea-a3ae-52f1a9572b11-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5494675598-c64bz\" (UID: \"111ca5e6-b48c-44ea-a3ae-52f1a9572b11\") " pod="openshift-monitoring/telemeter-client-5494675598-c64bz" Jul 15 21:04:56.205361 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:56.205340 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/111ca5e6-b48c-44ea-a3ae-52f1a9572b11-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5494675598-c64bz\" (UID: \"111ca5e6-b48c-44ea-a3ae-52f1a9572b11\") " pod="openshift-monitoring/telemeter-client-5494675598-c64bz" Jul 15 21:04:56.205567 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:56.205548 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/111ca5e6-b48c-44ea-a3ae-52f1a9572b11-secret-telemeter-client\") pod \"telemeter-client-5494675598-c64bz\" (UID: \"111ca5e6-b48c-44ea-a3ae-52f1a9572b11\") " pod="openshift-monitoring/telemeter-client-5494675598-c64bz" Jul 15 21:04:56.205629 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:56.205570 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/111ca5e6-b48c-44ea-a3ae-52f1a9572b11-federate-client-tls\") pod \"telemeter-client-5494675598-c64bz\" (UID: \"111ca5e6-b48c-44ea-a3ae-52f1a9572b11\") " pod="openshift-monitoring/telemeter-client-5494675598-c64bz" Jul 15 21:04:56.205629 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:56.205605 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/111ca5e6-b48c-44ea-a3ae-52f1a9572b11-telemeter-client-tls\") pod \"telemeter-client-5494675598-c64bz\" (UID: \"111ca5e6-b48c-44ea-a3ae-52f1a9572b11\") " pod="openshift-monitoring/telemeter-client-5494675598-c64bz" Jul 15 21:04:56.213111 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:56.213090 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4phpm\" (UniqueName: \"kubernetes.io/projected/111ca5e6-b48c-44ea-a3ae-52f1a9572b11-kube-api-access-4phpm\") pod \"telemeter-client-5494675598-c64bz\" (UID: \"111ca5e6-b48c-44ea-a3ae-52f1a9572b11\") " pod="openshift-monitoring/telemeter-client-5494675598-c64bz" Jul 15 21:04:56.229028 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:56.229004 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5494675598-c64bz" Jul 15 21:04:56.362345 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:56.362313 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5494675598-c64bz"] Jul 15 21:04:56.365295 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:04:56.365255 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod111ca5e6_b48c_44ea_a3ae_52f1a9572b11.slice/crio-2491202c80bd82e76fd9cc22012f0ae4e10105e2ab346beb07507ed6b71fe652 WatchSource:0}: Error finding container 2491202c80bd82e76fd9cc22012f0ae4e10105e2ab346beb07507ed6b71fe652: Status 404 returned error can't find the container with id 2491202c80bd82e76fd9cc22012f0ae4e10105e2ab346beb07507ed6b71fe652 Jul 15 21:04:56.843255 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:56.843217 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5494675598-c64bz" event={"ID":"111ca5e6-b48c-44ea-a3ae-52f1a9572b11","Type":"ContainerStarted","Data":"2491202c80bd82e76fd9cc22012f0ae4e10105e2ab346beb07507ed6b71fe652"} Jul 15 21:04:58.852384 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:58.852352 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5494675598-c64bz" event={"ID":"111ca5e6-b48c-44ea-a3ae-52f1a9572b11","Type":"ContainerStarted","Data":"534145940ad3278953d46b0f63168443bd4acfbf8dbcc07ab8dbae7f9fbf54b2"} Jul 15 21:04:59.859371 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:59.859338 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5494675598-c64bz" event={"ID":"111ca5e6-b48c-44ea-a3ae-52f1a9572b11","Type":"ContainerStarted","Data":"65d6d2655c4da1cbf13899d64dd0bdb489ac17187fdbbceb030be358776ea7a7"} Jul 15 21:04:59.859727 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:59.859400 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5494675598-c64bz" event={"ID":"111ca5e6-b48c-44ea-a3ae-52f1a9572b11","Type":"ContainerStarted","Data":"2207fe9874fd266a984825bfeb8494f7d309198a07ffe4bfee33cf423284bdf9"} Jul 15 21:04:59.887194 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:04:59.887147 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5494675598-c64bz" podStartSLOduration=2.034352627 podStartE2EDuration="4.887131724s" totalImagesPullingTime="2.852779097s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:04:55 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:04:59.885404153 +0000 UTC m=+204.426716704" watchObservedRunningTime="2026-07-15 21:04:59.887131724 +0000 UTC m=+204.428444300" Jul 15 21:05:00.699352 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:05:00.699317 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:05:00.778097 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:05:00.778035 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:05:11.493658 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:05:11.493621 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:05:11.619763 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:05:11.619729 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:05:28.435650 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:05:28.435617 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:05:43.074333 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:05:43.074297 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:05:47.409916 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:05:47.409872 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:06:06.129615 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:06:06.129527 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:06:06.190022 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:06:06.189962 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:06:07.351326 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:06:07.351288 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:06:07.451434 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:06:07.451401 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:06:14.301141 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:06:14.301103 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:06:18.934801 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:06:18.934739 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:06:18.985883 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:06:18.985849 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:06:23.776025 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:06:23.775985 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:06:23.876513 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:06:23.876485 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:06:35.898777 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:06:35.898749 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m5r2m_0db09c1e-d24d-4519-93c8-c185018c3d24/ovn-acl-logging/0.log" Jul 15 21:06:35.901652 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:06:35.901621 2579 kubelet.go:1648] "Image garbage collection succeeded" Jul 15 21:06:35.906304 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:06:35.906278 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m5r2m_0db09c1e-d24d-4519-93c8-c185018c3d24/ovn-acl-logging/0.log" Jul 15 21:06:43.582540 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:06:43.582503 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:06:43.682737 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:06:43.682701 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:07:01.926450 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:01.926418 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bq8q"] Jul 15 21:07:01.929483 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:01.929466 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bq8q" Jul 15 21:07:01.932975 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:01.932954 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-kj26j\"" Jul 15 21:07:01.933163 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:01.933144 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 15 21:07:01.934534 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:01.934509 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 15 21:07:01.938983 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:01.938958 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bq8q"] Jul 15 21:07:02.060704 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:02.060659 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b3130208-ff25-4624-a8f8-2d14658857a9-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bq8q\" (UID: \"b3130208-ff25-4624-a8f8-2d14658857a9\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bq8q" Jul 15 21:07:02.060704 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:02.060698 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b3130208-ff25-4624-a8f8-2d14658857a9-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bq8q\" (UID: \"b3130208-ff25-4624-a8f8-2d14658857a9\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bq8q" Jul 15 21:07:02.060927 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:02.060740 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8jd8\" (UniqueName: \"kubernetes.io/projected/b3130208-ff25-4624-a8f8-2d14658857a9-kube-api-access-h8jd8\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bq8q\" (UID: \"b3130208-ff25-4624-a8f8-2d14658857a9\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bq8q" Jul 15 21:07:02.161939 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:02.161901 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b3130208-ff25-4624-a8f8-2d14658857a9-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bq8q\" (UID: \"b3130208-ff25-4624-a8f8-2d14658857a9\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bq8q" Jul 15 21:07:02.161939 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:02.161941 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b3130208-ff25-4624-a8f8-2d14658857a9-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bq8q\" (UID: \"b3130208-ff25-4624-a8f8-2d14658857a9\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bq8q" Jul 15 21:07:02.162187 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:02.161973 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8jd8\" (UniqueName: \"kubernetes.io/projected/b3130208-ff25-4624-a8f8-2d14658857a9-kube-api-access-h8jd8\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bq8q\" (UID: \"b3130208-ff25-4624-a8f8-2d14658857a9\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bq8q" Jul 15 21:07:02.162333 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:02.162311 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b3130208-ff25-4624-a8f8-2d14658857a9-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bq8q\" (UID: \"b3130208-ff25-4624-a8f8-2d14658857a9\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bq8q" Jul 15 21:07:02.162402 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:02.162362 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b3130208-ff25-4624-a8f8-2d14658857a9-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bq8q\" (UID: \"b3130208-ff25-4624-a8f8-2d14658857a9\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bq8q" Jul 15 21:07:02.171639 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:02.171613 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8jd8\" (UniqueName: \"kubernetes.io/projected/b3130208-ff25-4624-a8f8-2d14658857a9-kube-api-access-h8jd8\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bq8q\" (UID: \"b3130208-ff25-4624-a8f8-2d14658857a9\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bq8q" Jul 15 21:07:02.238364 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:02.238268 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bq8q" Jul 15 21:07:02.364813 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:02.364791 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bq8q"] Jul 15 21:07:02.367498 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:07:02.367475 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3130208_ff25_4624_a8f8_2d14658857a9.slice/crio-586dcff000d0f233ee7dc9f0199cbb908ac9154d513802f98778e8e5dfbd09db WatchSource:0}: Error finding container 586dcff000d0f233ee7dc9f0199cbb908ac9154d513802f98778e8e5dfbd09db: Status 404 returned error can't find the container with id 586dcff000d0f233ee7dc9f0199cbb908ac9154d513802f98778e8e5dfbd09db Jul 15 21:07:02.369833 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:02.369816 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 21:07:03.272710 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:03.272675 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bq8q" event={"ID":"b3130208-ff25-4624-a8f8-2d14658857a9","Type":"ContainerStarted","Data":"586dcff000d0f233ee7dc9f0199cbb908ac9154d513802f98778e8e5dfbd09db"} Jul 15 21:07:09.296687 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:09.296593 2579 generic.go:356] "Generic (PLEG): container finished" podID="b3130208-ff25-4624-a8f8-2d14658857a9" containerID="f6c6629b56f7f11a2bebf7deea572804442d71fdb5d6f98e011a14d032932b5a" exitCode=0 Jul 15 21:07:09.296687 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:09.296643 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bq8q" event={"ID":"b3130208-ff25-4624-a8f8-2d14658857a9","Type":"ContainerDied","Data":"f6c6629b56f7f11a2bebf7deea572804442d71fdb5d6f98e011a14d032932b5a"} Jul 15 21:07:12.308942 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:12.308906 2579 generic.go:356] "Generic (PLEG): container finished" podID="b3130208-ff25-4624-a8f8-2d14658857a9" containerID="c4fe300b5df2a6df3640e1fb916c19abd859981932946f636d1cb1633a4003db" exitCode=0 Jul 15 21:07:12.309332 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:12.308988 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bq8q" event={"ID":"b3130208-ff25-4624-a8f8-2d14658857a9","Type":"ContainerDied","Data":"c4fe300b5df2a6df3640e1fb916c19abd859981932946f636d1cb1633a4003db"} Jul 15 21:07:19.338120 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:19.338085 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bq8q" event={"ID":"b3130208-ff25-4624-a8f8-2d14658857a9","Type":"ContainerStarted","Data":"9445843c2e2065e6839b8c954381fe2d5d502a96f6a530cb46c0b454ed215a44"} Jul 15 21:07:19.357164 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:19.357097 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bq8q" podStartSLOduration=2.8053624839999998 podStartE2EDuration="18.35707641s" totalImagesPullingTime="15.551713926s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:07:01 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:07:19.356352531 +0000 UTC m=+343.897665084" watchObservedRunningTime="2026-07-15 21:07:19.35707641 +0000 UTC m=+343.898388951" Jul 15 21:07:20.343816 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:20.343785 2579 generic.go:356] "Generic (PLEG): container finished" podID="b3130208-ff25-4624-a8f8-2d14658857a9" containerID="9445843c2e2065e6839b8c954381fe2d5d502a96f6a530cb46c0b454ed215a44" exitCode=0 Jul 15 21:07:20.344228 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:20.343873 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bq8q" event={"ID":"b3130208-ff25-4624-a8f8-2d14658857a9","Type":"ContainerDied","Data":"9445843c2e2065e6839b8c954381fe2d5d502a96f6a530cb46c0b454ed215a44"} Jul 15 21:07:21.469146 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:21.469123 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bq8q" Jul 15 21:07:21.620400 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:21.620369 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h8jd8\" (UniqueName: \"kubernetes.io/projected/b3130208-ff25-4624-a8f8-2d14658857a9-kube-api-access-h8jd8\") pod \"b3130208-ff25-4624-a8f8-2d14658857a9\" (UID: \"b3130208-ff25-4624-a8f8-2d14658857a9\") " Jul 15 21:07:21.620592 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:21.620418 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b3130208-ff25-4624-a8f8-2d14658857a9-util\") pod \"b3130208-ff25-4624-a8f8-2d14658857a9\" (UID: \"b3130208-ff25-4624-a8f8-2d14658857a9\") " Jul 15 21:07:21.620592 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:21.620442 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b3130208-ff25-4624-a8f8-2d14658857a9-bundle\") pod \"b3130208-ff25-4624-a8f8-2d14658857a9\" (UID: \"b3130208-ff25-4624-a8f8-2d14658857a9\") " Jul 15 21:07:21.621145 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:21.621119 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3130208-ff25-4624-a8f8-2d14658857a9-bundle" (OuterVolumeSpecName: "bundle") pod "b3130208-ff25-4624-a8f8-2d14658857a9" (UID: "b3130208-ff25-4624-a8f8-2d14658857a9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:07:21.622624 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:21.622602 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3130208-ff25-4624-a8f8-2d14658857a9-kube-api-access-h8jd8" (OuterVolumeSpecName: "kube-api-access-h8jd8") pod "b3130208-ff25-4624-a8f8-2d14658857a9" (UID: "b3130208-ff25-4624-a8f8-2d14658857a9"). InnerVolumeSpecName "kube-api-access-h8jd8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:07:21.625205 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:21.625182 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3130208-ff25-4624-a8f8-2d14658857a9-util" (OuterVolumeSpecName: "util") pod "b3130208-ff25-4624-a8f8-2d14658857a9" (UID: "b3130208-ff25-4624-a8f8-2d14658857a9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:07:21.721176 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:21.721147 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h8jd8\" (UniqueName: \"kubernetes.io/projected/b3130208-ff25-4624-a8f8-2d14658857a9-kube-api-access-h8jd8\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:07:21.721176 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:21.721172 2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b3130208-ff25-4624-a8f8-2d14658857a9-util\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:07:21.721176 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:21.721181 2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b3130208-ff25-4624-a8f8-2d14658857a9-bundle\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:07:22.352481 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:22.352411 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bq8q" Jul 15 21:07:22.352613 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:22.352411 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bq8q" event={"ID":"b3130208-ff25-4624-a8f8-2d14658857a9","Type":"ContainerDied","Data":"586dcff000d0f233ee7dc9f0199cbb908ac9154d513802f98778e8e5dfbd09db"} Jul 15 21:07:22.352613 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:22.352528 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="586dcff000d0f233ee7dc9f0199cbb908ac9154d513802f98778e8e5dfbd09db" Jul 15 21:07:28.147232 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:28.147180 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:07:29.761690 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:29.761649 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:07:31.737842 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:31.737770 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:07:31.836560 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:31.836527 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:07:33.721676 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:33.721636 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:07:33.948261 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:33.948228 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:07:34.218002 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:34.217949 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:07:34.333374 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:34.333336 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:07:34.665710 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:34.665679 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx26zw"] Jul 15 21:07:34.665958 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:34.665946 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3130208-ff25-4624-a8f8-2d14658857a9" containerName="util" Jul 15 21:07:34.666001 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:34.665960 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3130208-ff25-4624-a8f8-2d14658857a9" containerName="util" Jul 15 21:07:34.666001 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:34.665967 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3130208-ff25-4624-a8f8-2d14658857a9" containerName="pull" Jul 15 21:07:34.666001 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:34.665974 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3130208-ff25-4624-a8f8-2d14658857a9" containerName="pull" Jul 15 21:07:34.666001 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:34.665982 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3130208-ff25-4624-a8f8-2d14658857a9" containerName="extract" Jul 15 21:07:34.666001 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:34.665988 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3130208-ff25-4624-a8f8-2d14658857a9" containerName="extract" Jul 15 21:07:34.666172 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:34.666025 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3130208-ff25-4624-a8f8-2d14658857a9" containerName="extract" Jul 15 21:07:34.667887 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:34.667866 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx26zw" Jul 15 21:07:34.672423 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:34.672401 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 15 21:07:34.674123 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:34.674101 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 15 21:07:34.678093 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:34.678076 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-kj26j\"" Jul 15 21:07:34.689339 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:34.689313 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx26zw"] Jul 15 21:07:34.720790 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:34.720752 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/90f92d66-0569-457b-a34f-3e8d543c6d39-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx26zw\" (UID: \"90f92d66-0569-457b-a34f-3e8d543c6d39\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx26zw" Jul 15 21:07:34.720967 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:34.720799 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/90f92d66-0569-457b-a34f-3e8d543c6d39-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx26zw\" (UID: \"90f92d66-0569-457b-a34f-3e8d543c6d39\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx26zw" Jul 15 21:07:34.720967 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:34.720869 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drtcb\" (UniqueName: \"kubernetes.io/projected/90f92d66-0569-457b-a34f-3e8d543c6d39-kube-api-access-drtcb\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx26zw\" (UID: \"90f92d66-0569-457b-a34f-3e8d543c6d39\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx26zw" Jul 15 21:07:34.821261 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:34.821221 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/90f92d66-0569-457b-a34f-3e8d543c6d39-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx26zw\" (UID: \"90f92d66-0569-457b-a34f-3e8d543c6d39\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx26zw" Jul 15 21:07:34.821707 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:34.821277 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/90f92d66-0569-457b-a34f-3e8d543c6d39-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx26zw\" (UID: \"90f92d66-0569-457b-a34f-3e8d543c6d39\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx26zw" Jul 15 21:07:34.821707 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:34.821332 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drtcb\" (UniqueName: \"kubernetes.io/projected/90f92d66-0569-457b-a34f-3e8d543c6d39-kube-api-access-drtcb\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx26zw\" (UID: \"90f92d66-0569-457b-a34f-3e8d543c6d39\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx26zw" Jul 15 21:07:34.821707 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:34.821597 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/90f92d66-0569-457b-a34f-3e8d543c6d39-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx26zw\" (UID: \"90f92d66-0569-457b-a34f-3e8d543c6d39\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx26zw" Jul 15 21:07:34.821707 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:34.821631 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/90f92d66-0569-457b-a34f-3e8d543c6d39-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx26zw\" (UID: \"90f92d66-0569-457b-a34f-3e8d543c6d39\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx26zw" Jul 15 21:07:34.837728 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:34.837696 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drtcb\" (UniqueName: \"kubernetes.io/projected/90f92d66-0569-457b-a34f-3e8d543c6d39-kube-api-access-drtcb\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx26zw\" (UID: \"90f92d66-0569-457b-a34f-3e8d543c6d39\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx26zw" Jul 15 21:07:34.976241 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:34.976129 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx26zw" Jul 15 21:07:35.133905 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:35.133877 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx26zw"] Jul 15 21:07:35.136435 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:07:35.136408 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90f92d66_0569_457b_a34f_3e8d543c6d39.slice/crio-4303495769ee6e60d2b26d66863bb7926634ae9df16b187181345b11ea741667 WatchSource:0}: Error finding container 4303495769ee6e60d2b26d66863bb7926634ae9df16b187181345b11ea741667: Status 404 returned error can't find the container with id 4303495769ee6e60d2b26d66863bb7926634ae9df16b187181345b11ea741667 Jul 15 21:07:35.396952 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:35.396917 2579 generic.go:356] "Generic (PLEG): container finished" podID="90f92d66-0569-457b-a34f-3e8d543c6d39" containerID="4d4a11bb66f0ee0145c0ff418affeeb859fb4a926a2907e09ed51f82e3788119" exitCode=0 Jul 15 21:07:35.397162 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:35.396982 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx26zw" event={"ID":"90f92d66-0569-457b-a34f-3e8d543c6d39","Type":"ContainerDied","Data":"4d4a11bb66f0ee0145c0ff418affeeb859fb4a926a2907e09ed51f82e3788119"} Jul 15 21:07:35.397162 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:35.397036 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx26zw" event={"ID":"90f92d66-0569-457b-a34f-3e8d543c6d39","Type":"ContainerStarted","Data":"4303495769ee6e60d2b26d66863bb7926634ae9df16b187181345b11ea741667"} Jul 15 21:07:35.981489 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:35.981452 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:07:36.177838 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:36.177788 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:07:38.409845 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:38.409807 2579 generic.go:356] "Generic (PLEG): container finished" podID="90f92d66-0569-457b-a34f-3e8d543c6d39" containerID="8d65605aeed17d21411cd87897a49a11f1e6ce2d5ccc10438eaf16cbe52bde3d" exitCode=0 Jul 15 21:07:38.410320 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:38.409891 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx26zw" event={"ID":"90f92d66-0569-457b-a34f-3e8d543c6d39","Type":"ContainerDied","Data":"8d65605aeed17d21411cd87897a49a11f1e6ce2d5ccc10438eaf16cbe52bde3d"} Jul 15 21:07:38.725364 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:38.725239 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:07:38.790734 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:38.790701 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:07:39.415927 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:39.415894 2579 generic.go:356] "Generic (PLEG): container finished" podID="90f92d66-0569-457b-a34f-3e8d543c6d39" containerID="324d57a57ef36c64831de6dff22f485e9a91d1d4ca56b8059d5942ce66794e2f" exitCode=0 Jul 15 21:07:39.416330 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:39.415954 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx26zw" event={"ID":"90f92d66-0569-457b-a34f-3e8d543c6d39","Type":"ContainerDied","Data":"324d57a57ef36c64831de6dff22f485e9a91d1d4ca56b8059d5942ce66794e2f"} Jul 15 21:07:40.544021 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:40.543997 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx26zw" Jul 15 21:07:40.571019 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:40.570991 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/90f92d66-0569-457b-a34f-3e8d543c6d39-util\") pod \"90f92d66-0569-457b-a34f-3e8d543c6d39\" (UID: \"90f92d66-0569-457b-a34f-3e8d543c6d39\") " Jul 15 21:07:40.571179 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:40.571031 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/90f92d66-0569-457b-a34f-3e8d543c6d39-bundle\") pod \"90f92d66-0569-457b-a34f-3e8d543c6d39\" (UID: \"90f92d66-0569-457b-a34f-3e8d543c6d39\") " Jul 15 21:07:40.571179 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:40.571086 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-drtcb\" (UniqueName: \"kubernetes.io/projected/90f92d66-0569-457b-a34f-3e8d543c6d39-kube-api-access-drtcb\") pod \"90f92d66-0569-457b-a34f-3e8d543c6d39\" (UID: \"90f92d66-0569-457b-a34f-3e8d543c6d39\") " Jul 15 21:07:40.571455 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:40.571429 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90f92d66-0569-457b-a34f-3e8d543c6d39-bundle" (OuterVolumeSpecName: "bundle") pod "90f92d66-0569-457b-a34f-3e8d543c6d39" (UID: "90f92d66-0569-457b-a34f-3e8d543c6d39"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:07:40.573331 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:40.573287 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90f92d66-0569-457b-a34f-3e8d543c6d39-kube-api-access-drtcb" (OuterVolumeSpecName: "kube-api-access-drtcb") pod "90f92d66-0569-457b-a34f-3e8d543c6d39" (UID: "90f92d66-0569-457b-a34f-3e8d543c6d39"). InnerVolumeSpecName "kube-api-access-drtcb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:07:40.576564 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:40.576532 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90f92d66-0569-457b-a34f-3e8d543c6d39-util" (OuterVolumeSpecName: "util") pod "90f92d66-0569-457b-a34f-3e8d543c6d39" (UID: "90f92d66-0569-457b-a34f-3e8d543c6d39"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:07:40.672414 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:40.672319 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-drtcb\" (UniqueName: \"kubernetes.io/projected/90f92d66-0569-457b-a34f-3e8d543c6d39-kube-api-access-drtcb\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:07:40.672414 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:40.672351 2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/90f92d66-0569-457b-a34f-3e8d543c6d39-util\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:07:40.672414 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:40.672362 2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/90f92d66-0569-457b-a34f-3e8d543c6d39-bundle\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:07:41.424698 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:41.424652 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx26zw" event={"ID":"90f92d66-0569-457b-a34f-3e8d543c6d39","Type":"ContainerDied","Data":"4303495769ee6e60d2b26d66863bb7926634ae9df16b187181345b11ea741667"} Jul 15 21:07:41.424698 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:41.424696 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx26zw" Jul 15 21:07:41.424940 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:41.424701 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4303495769ee6e60d2b26d66863bb7926634ae9df16b187181345b11ea741667" Jul 15 21:07:43.171367 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:43.171332 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:07:45.507988 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:45.507955 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-jkp58"] Jul 15 21:07:45.508367 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:45.508227 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90f92d66-0569-457b-a34f-3e8d543c6d39" containerName="util" Jul 15 21:07:45.508367 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:45.508238 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="90f92d66-0569-457b-a34f-3e8d543c6d39" containerName="util" Jul 15 21:07:45.508367 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:45.508246 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90f92d66-0569-457b-a34f-3e8d543c6d39" containerName="pull" Jul 15 21:07:45.508367 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:45.508251 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="90f92d66-0569-457b-a34f-3e8d543c6d39" containerName="pull" Jul 15 21:07:45.508367 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:45.508258 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90f92d66-0569-457b-a34f-3e8d543c6d39" containerName="extract" Jul 15 21:07:45.508367 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:45.508264 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="90f92d66-0569-457b-a34f-3e8d543c6d39" containerName="extract" Jul 15 21:07:45.508367 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:45.508317 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="90f92d66-0569-457b-a34f-3e8d543c6d39" containerName="extract" Jul 15 21:07:45.510143 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:45.510127 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-jkp58" Jul 15 21:07:45.513128 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:45.513109 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 15 21:07:45.513254 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:45.513109 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 15 21:07:45.514500 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:45.514480 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-l2nkk\"" Jul 15 21:07:45.518527 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:45.518507 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-jkp58"] Jul 15 21:07:45.606729 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:45.606694 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b7682475-5146-4dc5-badc-1cf1def70ebe-bound-sa-token\") pod \"cert-manager-55b4d954d5-jkp58\" (UID: \"b7682475-5146-4dc5-badc-1cf1def70ebe\") " pod="cert-manager/cert-manager-55b4d954d5-jkp58" Jul 15 21:07:45.606729 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:45.606738 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwr5h\" (UniqueName: \"kubernetes.io/projected/b7682475-5146-4dc5-badc-1cf1def70ebe-kube-api-access-pwr5h\") pod \"cert-manager-55b4d954d5-jkp58\" (UID: \"b7682475-5146-4dc5-badc-1cf1def70ebe\") " pod="cert-manager/cert-manager-55b4d954d5-jkp58" Jul 15 21:07:45.708040 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:45.708001 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b7682475-5146-4dc5-badc-1cf1def70ebe-bound-sa-token\") pod \"cert-manager-55b4d954d5-jkp58\" (UID: \"b7682475-5146-4dc5-badc-1cf1def70ebe\") " pod="cert-manager/cert-manager-55b4d954d5-jkp58" Jul 15 21:07:45.708237 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:45.708044 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwr5h\" (UniqueName: \"kubernetes.io/projected/b7682475-5146-4dc5-badc-1cf1def70ebe-kube-api-access-pwr5h\") pod \"cert-manager-55b4d954d5-jkp58\" (UID: \"b7682475-5146-4dc5-badc-1cf1def70ebe\") " pod="cert-manager/cert-manager-55b4d954d5-jkp58" Jul 15 21:07:45.716812 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:45.716778 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwr5h\" (UniqueName: \"kubernetes.io/projected/b7682475-5146-4dc5-badc-1cf1def70ebe-kube-api-access-pwr5h\") pod \"cert-manager-55b4d954d5-jkp58\" (UID: \"b7682475-5146-4dc5-badc-1cf1def70ebe\") " pod="cert-manager/cert-manager-55b4d954d5-jkp58" Jul 15 21:07:45.716913 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:45.716823 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b7682475-5146-4dc5-badc-1cf1def70ebe-bound-sa-token\") pod \"cert-manager-55b4d954d5-jkp58\" (UID: \"b7682475-5146-4dc5-badc-1cf1def70ebe\") " pod="cert-manager/cert-manager-55b4d954d5-jkp58" Jul 15 21:07:45.820132 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:45.820008 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-jkp58" Jul 15 21:07:45.949573 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:45.949547 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-jkp58"] Jul 15 21:07:45.951953 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:07:45.951924 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb7682475_5146_4dc5_badc_1cf1def70ebe.slice/crio-732d8311f8ad2b1d3d336278f88eded13aa13a6f7c1a26d4ede4f85cf0008064 WatchSource:0}: Error finding container 732d8311f8ad2b1d3d336278f88eded13aa13a6f7c1a26d4ede4f85cf0008064: Status 404 returned error can't find the container with id 732d8311f8ad2b1d3d336278f88eded13aa13a6f7c1a26d4ede4f85cf0008064 Jul 15 21:07:46.446604 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:46.446563 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-jkp58" event={"ID":"b7682475-5146-4dc5-badc-1cf1def70ebe","Type":"ContainerStarted","Data":"732d8311f8ad2b1d3d336278f88eded13aa13a6f7c1a26d4ede4f85cf0008064"} Jul 15 21:07:46.446785 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:46.446654 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:07:46.539149 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:46.539115 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:07:48.278929 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:48.278889 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8mrng"] Jul 15 21:07:48.281460 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:48.281436 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8mrng" Jul 15 21:07:48.284949 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:48.284926 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 15 21:07:48.285348 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:48.285328 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 15 21:07:48.285860 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:48.285840 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-nvt2c\"" Jul 15 21:07:48.298353 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:48.298329 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8mrng"] Jul 15 21:07:48.327731 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:48.327688 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrcck\" (UniqueName: \"kubernetes.io/projected/f91d181a-4185-4d11-acbd-d91ae771a9d1-kube-api-access-vrcck\") pod \"openshift-lws-operator-fd8ccff4c-8mrng\" (UID: \"f91d181a-4185-4d11-acbd-d91ae771a9d1\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8mrng" Jul 15 21:07:48.327898 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:48.327746 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f91d181a-4185-4d11-acbd-d91ae771a9d1-tmp\") pod \"openshift-lws-operator-fd8ccff4c-8mrng\" (UID: \"f91d181a-4185-4d11-acbd-d91ae771a9d1\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8mrng" Jul 15 21:07:48.428590 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:48.428517 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrcck\" (UniqueName: \"kubernetes.io/projected/f91d181a-4185-4d11-acbd-d91ae771a9d1-kube-api-access-vrcck\") pod \"openshift-lws-operator-fd8ccff4c-8mrng\" (UID: \"f91d181a-4185-4d11-acbd-d91ae771a9d1\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8mrng" Jul 15 21:07:48.428779 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:48.428616 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f91d181a-4185-4d11-acbd-d91ae771a9d1-tmp\") pod \"openshift-lws-operator-fd8ccff4c-8mrng\" (UID: \"f91d181a-4185-4d11-acbd-d91ae771a9d1\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8mrng" Jul 15 21:07:48.429044 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:48.429024 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f91d181a-4185-4d11-acbd-d91ae771a9d1-tmp\") pod \"openshift-lws-operator-fd8ccff4c-8mrng\" (UID: \"f91d181a-4185-4d11-acbd-d91ae771a9d1\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8mrng" Jul 15 21:07:48.445684 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:48.445650 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrcck\" (UniqueName: \"kubernetes.io/projected/f91d181a-4185-4d11-acbd-d91ae771a9d1-kube-api-access-vrcck\") pod \"openshift-lws-operator-fd8ccff4c-8mrng\" (UID: \"f91d181a-4185-4d11-acbd-d91ae771a9d1\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8mrng" Jul 15 21:07:48.593108 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:48.592994 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8mrng" Jul 15 21:07:48.880457 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:48.880434 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8mrng"] Jul 15 21:07:48.882973 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:07:48.882944 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf91d181a_4185_4d11_acbd_d91ae771a9d1.slice/crio-6f9087e718b64a819bcde050a2f78808e12f17bddf633ca8aea97bd71c9237f9 WatchSource:0}: Error finding container 6f9087e718b64a819bcde050a2f78808e12f17bddf633ca8aea97bd71c9237f9: Status 404 returned error can't find the container with id 6f9087e718b64a819bcde050a2f78808e12f17bddf633ca8aea97bd71c9237f9 Jul 15 21:07:49.458903 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:49.458860 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-jkp58" event={"ID":"b7682475-5146-4dc5-badc-1cf1def70ebe","Type":"ContainerStarted","Data":"784285cb819cdb1171d59cea2dafd5c7afbce06e3d37342d2d16d55ee399837e"} Jul 15 21:07:49.460188 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:49.460164 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8mrng" event={"ID":"f91d181a-4185-4d11-acbd-d91ae771a9d1","Type":"ContainerStarted","Data":"6f9087e718b64a819bcde050a2f78808e12f17bddf633ca8aea97bd71c9237f9"} Jul 15 21:07:49.478326 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:49.478274 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-jkp58" podStartSLOduration=1.6330866400000001 podStartE2EDuration="4.478257295s" totalImagesPullingTime="2.845170655s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:07:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:07:49.476827352 +0000 UTC m=+374.018139902" watchObservedRunningTime="2026-07-15 21:07:49.478257295 +0000 UTC m=+374.019569845" Jul 15 21:07:51.469610 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:51.469521 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8mrng" event={"ID":"f91d181a-4185-4d11-acbd-d91ae771a9d1","Type":"ContainerStarted","Data":"953868e4b5b1c5897573c98c5d8882b42e37ca60d0ad87809b37b5bdd708e430"} Jul 15 21:07:51.487221 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:51.487170 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8mrng" podStartSLOduration=1.215450429 podStartE2EDuration="3.487154052s" totalImagesPullingTime="2.271703623s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:07:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:07:51.485951066 +0000 UTC m=+376.027263611" watchObservedRunningTime="2026-07-15 21:07:51.487154052 +0000 UTC m=+376.028466611" Jul 15 21:07:52.241876 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:52.241843 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:07:52.347548 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:52.347509 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:07:54.068930 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:54.068896 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnmrb"] Jul 15 21:07:54.071210 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:54.071194 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnmrb" Jul 15 21:07:54.076190 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:54.076166 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-kj26j\"" Jul 15 21:07:54.076543 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:54.076518 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 15 21:07:54.077714 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:54.077691 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 15 21:07:54.081326 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:54.081303 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnmrb"] Jul 15 21:07:54.174579 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:54.174543 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a211b55a-db07-40fb-9b78-16c37fa9e47a-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnmrb\" (UID: \"a211b55a-db07-40fb-9b78-16c37fa9e47a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnmrb" Jul 15 21:07:54.174747 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:54.174593 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a211b55a-db07-40fb-9b78-16c37fa9e47a-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnmrb\" (UID: \"a211b55a-db07-40fb-9b78-16c37fa9e47a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnmrb" Jul 15 21:07:54.174747 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:54.174651 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7hs7\" (UniqueName: \"kubernetes.io/projected/a211b55a-db07-40fb-9b78-16c37fa9e47a-kube-api-access-j7hs7\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnmrb\" (UID: \"a211b55a-db07-40fb-9b78-16c37fa9e47a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnmrb" Jul 15 21:07:54.275802 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:54.275768 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a211b55a-db07-40fb-9b78-16c37fa9e47a-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnmrb\" (UID: \"a211b55a-db07-40fb-9b78-16c37fa9e47a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnmrb" Jul 15 21:07:54.275802 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:54.275815 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a211b55a-db07-40fb-9b78-16c37fa9e47a-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnmrb\" (UID: \"a211b55a-db07-40fb-9b78-16c37fa9e47a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnmrb" Jul 15 21:07:54.276006 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:54.275836 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7hs7\" (UniqueName: \"kubernetes.io/projected/a211b55a-db07-40fb-9b78-16c37fa9e47a-kube-api-access-j7hs7\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnmrb\" (UID: \"a211b55a-db07-40fb-9b78-16c37fa9e47a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnmrb" Jul 15 21:07:54.276274 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:54.276252 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a211b55a-db07-40fb-9b78-16c37fa9e47a-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnmrb\" (UID: \"a211b55a-db07-40fb-9b78-16c37fa9e47a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnmrb" Jul 15 21:07:54.276321 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:54.276263 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a211b55a-db07-40fb-9b78-16c37fa9e47a-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnmrb\" (UID: \"a211b55a-db07-40fb-9b78-16c37fa9e47a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnmrb" Jul 15 21:07:54.281033 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:54.280998 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:07:54.284912 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:54.284889 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7hs7\" (UniqueName: \"kubernetes.io/projected/a211b55a-db07-40fb-9b78-16c37fa9e47a-kube-api-access-j7hs7\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnmrb\" (UID: \"a211b55a-db07-40fb-9b78-16c37fa9e47a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnmrb" Jul 15 21:07:54.381468 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:54.381432 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnmrb" Jul 15 21:07:54.507468 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:54.507445 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnmrb"] Jul 15 21:07:54.509925 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:07:54.509892 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda211b55a_db07_40fb_9b78_16c37fa9e47a.slice/crio-8528c0a639104bf202ef054f42d72027a136a728af8380f7363a3cb9f46c601f WatchSource:0}: Error finding container 8528c0a639104bf202ef054f42d72027a136a728af8380f7363a3cb9f46c601f: Status 404 returned error can't find the container with id 8528c0a639104bf202ef054f42d72027a136a728af8380f7363a3cb9f46c601f Jul 15 21:07:55.487665 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:55.487634 2579 generic.go:356] "Generic (PLEG): container finished" podID="a211b55a-db07-40fb-9b78-16c37fa9e47a" containerID="d9dd5221e7f8c847c96e2da2d4960ef889d2e156ae61b0a46ab35bc314c9fe09" exitCode=0 Jul 15 21:07:55.488011 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:55.487689 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnmrb" event={"ID":"a211b55a-db07-40fb-9b78-16c37fa9e47a","Type":"ContainerDied","Data":"d9dd5221e7f8c847c96e2da2d4960ef889d2e156ae61b0a46ab35bc314c9fe09"} Jul 15 21:07:55.488011 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:55.487756 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnmrb" event={"ID":"a211b55a-db07-40fb-9b78-16c37fa9e47a","Type":"ContainerStarted","Data":"8528c0a639104bf202ef054f42d72027a136a728af8380f7363a3cb9f46c601f"} Jul 15 21:07:56.492901 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:56.492865 2579 generic.go:356] "Generic (PLEG): container finished" podID="a211b55a-db07-40fb-9b78-16c37fa9e47a" containerID="8a11a5c5576141ad6bde664e9ae81ae6d33c0eda29a48bcdc0b3b37ad4b378ce" exitCode=0 Jul 15 21:07:56.493310 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:56.492919 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnmrb" event={"ID":"a211b55a-db07-40fb-9b78-16c37fa9e47a","Type":"ContainerDied","Data":"8a11a5c5576141ad6bde664e9ae81ae6d33c0eda29a48bcdc0b3b37ad4b378ce"} Jul 15 21:07:57.499024 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:57.498989 2579 generic.go:356] "Generic (PLEG): container finished" podID="a211b55a-db07-40fb-9b78-16c37fa9e47a" containerID="27eee1c1c6df348ffbebb1d111b3682cad6fde6fdf8d5131b198d863ad357c75" exitCode=0 Jul 15 21:07:57.499420 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:57.499090 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnmrb" event={"ID":"a211b55a-db07-40fb-9b78-16c37fa9e47a","Type":"ContainerDied","Data":"27eee1c1c6df348ffbebb1d111b3682cad6fde6fdf8d5131b198d863ad357c75"} Jul 15 21:07:58.628941 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:58.628913 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnmrb" Jul 15 21:07:58.704345 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:58.704312 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a211b55a-db07-40fb-9b78-16c37fa9e47a-util\") pod \"a211b55a-db07-40fb-9b78-16c37fa9e47a\" (UID: \"a211b55a-db07-40fb-9b78-16c37fa9e47a\") " Jul 15 21:07:58.704506 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:58.704368 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a211b55a-db07-40fb-9b78-16c37fa9e47a-bundle\") pod \"a211b55a-db07-40fb-9b78-16c37fa9e47a\" (UID: \"a211b55a-db07-40fb-9b78-16c37fa9e47a\") " Jul 15 21:07:58.704506 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:58.704394 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j7hs7\" (UniqueName: \"kubernetes.io/projected/a211b55a-db07-40fb-9b78-16c37fa9e47a-kube-api-access-j7hs7\") pod \"a211b55a-db07-40fb-9b78-16c37fa9e47a\" (UID: \"a211b55a-db07-40fb-9b78-16c37fa9e47a\") " Jul 15 21:07:58.705033 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:58.704998 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a211b55a-db07-40fb-9b78-16c37fa9e47a-bundle" (OuterVolumeSpecName: "bundle") pod "a211b55a-db07-40fb-9b78-16c37fa9e47a" (UID: "a211b55a-db07-40fb-9b78-16c37fa9e47a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:07:58.706550 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:58.706522 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a211b55a-db07-40fb-9b78-16c37fa9e47a-kube-api-access-j7hs7" (OuterVolumeSpecName: "kube-api-access-j7hs7") pod "a211b55a-db07-40fb-9b78-16c37fa9e47a" (UID: "a211b55a-db07-40fb-9b78-16c37fa9e47a"). InnerVolumeSpecName "kube-api-access-j7hs7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:07:58.709773 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:58.709750 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a211b55a-db07-40fb-9b78-16c37fa9e47a-util" (OuterVolumeSpecName: "util") pod "a211b55a-db07-40fb-9b78-16c37fa9e47a" (UID: "a211b55a-db07-40fb-9b78-16c37fa9e47a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:07:58.805633 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:58.805549 2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a211b55a-db07-40fb-9b78-16c37fa9e47a-util\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:07:58.805633 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:58.805578 2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a211b55a-db07-40fb-9b78-16c37fa9e47a-bundle\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:07:58.805633 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:58.805587 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j7hs7\" (UniqueName: \"kubernetes.io/projected/a211b55a-db07-40fb-9b78-16c37fa9e47a-kube-api-access-j7hs7\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:07:59.508321 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:59.508288 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnmrb" event={"ID":"a211b55a-db07-40fb-9b78-16c37fa9e47a","Type":"ContainerDied","Data":"8528c0a639104bf202ef054f42d72027a136a728af8380f7363a3cb9f46c601f"} Jul 15 21:07:59.508497 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:59.508334 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8528c0a639104bf202ef054f42d72027a136a728af8380f7363a3cb9f46c601f" Jul 15 21:07:59.508497 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:07:59.508332 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnmrb" Jul 15 21:08:03.711825 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:03.711782 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:08:03.812512 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:03.812464 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:08:05.976702 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:05.976647 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:08:06.076514 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:06.076479 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:08:06.621531 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:06.621483 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:08:06.914897 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:06.914794 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:08:07.060159 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:07.060098 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:08:07.085921 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:07.085888 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-65c67f598f-wb248"] Jul 15 21:08:07.086189 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:07.086177 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a211b55a-db07-40fb-9b78-16c37fa9e47a" containerName="util" Jul 15 21:08:07.086248 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:07.086190 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a211b55a-db07-40fb-9b78-16c37fa9e47a" containerName="util" Jul 15 21:08:07.086248 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:07.086205 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a211b55a-db07-40fb-9b78-16c37fa9e47a" containerName="pull" Jul 15 21:08:07.086248 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:07.086210 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a211b55a-db07-40fb-9b78-16c37fa9e47a" containerName="pull" Jul 15 21:08:07.086248 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:07.086226 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a211b55a-db07-40fb-9b78-16c37fa9e47a" containerName="extract" Jul 15 21:08:07.086248 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:07.086232 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a211b55a-db07-40fb-9b78-16c37fa9e47a" containerName="extract" Jul 15 21:08:07.086388 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:07.086278 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a211b55a-db07-40fb-9b78-16c37fa9e47a" containerName="extract" Jul 15 21:08:07.088544 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:07.088526 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-65c67f598f-wb248" Jul 15 21:08:07.093642 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:07.093617 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 15 21:08:07.093870 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:07.093653 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 15 21:08:07.094080 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:07.094047 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 15 21:08:07.094186 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:07.094102 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-cfr2c\"" Jul 15 21:08:07.095017 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:07.094845 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 15 21:08:07.106552 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:07.106523 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-65c67f598f-wb248"] Jul 15 21:08:07.166649 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:07.166547 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22xbf\" (UniqueName: \"kubernetes.io/projected/7193df13-36d0-4f6b-826b-4c876d9c2299-kube-api-access-22xbf\") pod \"opendatahub-operator-controller-manager-65c67f598f-wb248\" (UID: \"7193df13-36d0-4f6b-826b-4c876d9c2299\") " pod="opendatahub/opendatahub-operator-controller-manager-65c67f598f-wb248" Jul 15 21:08:07.166649 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:07.166592 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7193df13-36d0-4f6b-826b-4c876d9c2299-webhook-cert\") pod \"opendatahub-operator-controller-manager-65c67f598f-wb248\" (UID: \"7193df13-36d0-4f6b-826b-4c876d9c2299\") " pod="opendatahub/opendatahub-operator-controller-manager-65c67f598f-wb248" Jul 15 21:08:07.166649 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:07.166615 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7193df13-36d0-4f6b-826b-4c876d9c2299-apiservice-cert\") pod \"opendatahub-operator-controller-manager-65c67f598f-wb248\" (UID: \"7193df13-36d0-4f6b-826b-4c876d9c2299\") " pod="opendatahub/opendatahub-operator-controller-manager-65c67f598f-wb248" Jul 15 21:08:07.166899 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:07.166692 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7193df13-36d0-4f6b-826b-4c876d9c2299-manifests\") pod \"opendatahub-operator-controller-manager-65c67f598f-wb248\" (UID: \"7193df13-36d0-4f6b-826b-4c876d9c2299\") " pod="opendatahub/opendatahub-operator-controller-manager-65c67f598f-wb248" Jul 15 21:08:07.166899 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:07.166748 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7193df13-36d0-4f6b-826b-4c876d9c2299-tmp\") pod \"opendatahub-operator-controller-manager-65c67f598f-wb248\" (UID: \"7193df13-36d0-4f6b-826b-4c876d9c2299\") " pod="opendatahub/opendatahub-operator-controller-manager-65c67f598f-wb248" Jul 15 21:08:07.267986 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:07.267943 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7193df13-36d0-4f6b-826b-4c876d9c2299-tmp\") pod \"opendatahub-operator-controller-manager-65c67f598f-wb248\" (UID: \"7193df13-36d0-4f6b-826b-4c876d9c2299\") " pod="opendatahub/opendatahub-operator-controller-manager-65c67f598f-wb248" Jul 15 21:08:07.268205 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:07.267997 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22xbf\" (UniqueName: \"kubernetes.io/projected/7193df13-36d0-4f6b-826b-4c876d9c2299-kube-api-access-22xbf\") pod \"opendatahub-operator-controller-manager-65c67f598f-wb248\" (UID: \"7193df13-36d0-4f6b-826b-4c876d9c2299\") " pod="opendatahub/opendatahub-operator-controller-manager-65c67f598f-wb248" Jul 15 21:08:07.268205 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:07.268030 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7193df13-36d0-4f6b-826b-4c876d9c2299-webhook-cert\") pod \"opendatahub-operator-controller-manager-65c67f598f-wb248\" (UID: \"7193df13-36d0-4f6b-826b-4c876d9c2299\") " pod="opendatahub/opendatahub-operator-controller-manager-65c67f598f-wb248" Jul 15 21:08:07.268205 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:07.268051 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7193df13-36d0-4f6b-826b-4c876d9c2299-apiservice-cert\") pod \"opendatahub-operator-controller-manager-65c67f598f-wb248\" (UID: \"7193df13-36d0-4f6b-826b-4c876d9c2299\") " pod="opendatahub/opendatahub-operator-controller-manager-65c67f598f-wb248" Jul 15 21:08:07.268205 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:07.268105 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7193df13-36d0-4f6b-826b-4c876d9c2299-manifests\") pod \"opendatahub-operator-controller-manager-65c67f598f-wb248\" (UID: \"7193df13-36d0-4f6b-826b-4c876d9c2299\") " pod="opendatahub/opendatahub-operator-controller-manager-65c67f598f-wb248" Jul 15 21:08:07.268482 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:07.268459 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7193df13-36d0-4f6b-826b-4c876d9c2299-manifests\") pod \"opendatahub-operator-controller-manager-65c67f598f-wb248\" (UID: \"7193df13-36d0-4f6b-826b-4c876d9c2299\") " pod="opendatahub/opendatahub-operator-controller-manager-65c67f598f-wb248" Jul 15 21:08:07.270234 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:07.270200 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7193df13-36d0-4f6b-826b-4c876d9c2299-tmp\") pod \"opendatahub-operator-controller-manager-65c67f598f-wb248\" (UID: \"7193df13-36d0-4f6b-826b-4c876d9c2299\") " pod="opendatahub/opendatahub-operator-controller-manager-65c67f598f-wb248" Jul 15 21:08:07.270473 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:07.270454 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7193df13-36d0-4f6b-826b-4c876d9c2299-webhook-cert\") pod \"opendatahub-operator-controller-manager-65c67f598f-wb248\" (UID: \"7193df13-36d0-4f6b-826b-4c876d9c2299\") " pod="opendatahub/opendatahub-operator-controller-manager-65c67f598f-wb248" Jul 15 21:08:07.270636 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:07.270618 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7193df13-36d0-4f6b-826b-4c876d9c2299-apiservice-cert\") pod \"opendatahub-operator-controller-manager-65c67f598f-wb248\" (UID: \"7193df13-36d0-4f6b-826b-4c876d9c2299\") " pod="opendatahub/opendatahub-operator-controller-manager-65c67f598f-wb248" Jul 15 21:08:07.282491 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:07.282462 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22xbf\" (UniqueName: \"kubernetes.io/projected/7193df13-36d0-4f6b-826b-4c876d9c2299-kube-api-access-22xbf\") pod \"opendatahub-operator-controller-manager-65c67f598f-wb248\" (UID: \"7193df13-36d0-4f6b-826b-4c876d9c2299\") " pod="opendatahub/opendatahub-operator-controller-manager-65c67f598f-wb248" Jul 15 21:08:07.310581 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:07.310545 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:08:07.398797 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:07.398760 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-65c67f598f-wb248" Jul 15 21:08:07.412620 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:07.412580 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:08:07.537193 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:07.537170 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-65c67f598f-wb248"] Jul 15 21:08:07.538971 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:08:07.538939 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7193df13_36d0_4f6b_826b_4c876d9c2299.slice/crio-6e5b05f2df8264392af6ef326870d6bd6480ad106c37ebc59a0d3825d903effc WatchSource:0}: Error finding container 6e5b05f2df8264392af6ef326870d6bd6480ad106c37ebc59a0d3825d903effc: Status 404 returned error can't find the container with id 6e5b05f2df8264392af6ef326870d6bd6480ad106c37ebc59a0d3825d903effc Jul 15 21:08:07.541572 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:07.541548 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:08:07.642546 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:07.642509 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:08:07.740256 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:07.740166 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:08:07.845919 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:07.845879 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:08:08.544059 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:08.544009 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65c67f598f-wb248" event={"ID":"7193df13-36d0-4f6b-826b-4c876d9c2299","Type":"ContainerStarted","Data":"6e5b05f2df8264392af6ef326870d6bd6480ad106c37ebc59a0d3825d903effc"} Jul 15 21:08:10.555630 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:10.555593 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65c67f598f-wb248" event={"ID":"7193df13-36d0-4f6b-826b-4c876d9c2299","Type":"ContainerStarted","Data":"c5f2b1f026887314e9a7a87a8c5993b5dab8148ee49e33ba2e89b311ff03479e"} Jul 15 21:08:11.560842 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:11.560805 2579 generic.go:356] "Generic (PLEG): container finished" podID="7193df13-36d0-4f6b-826b-4c876d9c2299" containerID="c5f2b1f026887314e9a7a87a8c5993b5dab8148ee49e33ba2e89b311ff03479e" exitCode=0 Jul 15 21:08:11.561308 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:11.560886 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65c67f598f-wb248" event={"ID":"7193df13-36d0-4f6b-826b-4c876d9c2299","Type":"ContainerDied","Data":"c5f2b1f026887314e9a7a87a8c5993b5dab8148ee49e33ba2e89b311ff03479e"} Jul 15 21:08:12.568931 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:12.568894 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65c67f598f-wb248" event={"ID":"7193df13-36d0-4f6b-826b-4c876d9c2299","Type":"ContainerStarted","Data":"a4f8b35df3b1028bb04d3f827a82449bddfdb71279fba88ac318287bcac4b930"} Jul 15 21:08:12.569352 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:12.569049 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-65c67f598f-wb248" Jul 15 21:08:12.597008 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:12.596949 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-65c67f598f-wb248" podStartSLOduration=1.27742365 podStartE2EDuration="5.596933648s" totalImagesPullingTime="3.311202832s" totalInitContainerRuntime="1.008307166s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:08:07 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:08:12.594516099 +0000 UTC m=+397.135828648" watchObservedRunningTime="2026-07-15 21:08:12.596933648 +0000 UTC m=+397.138246197" Jul 15 21:08:14.544056 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:14.544023 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:08:14.652255 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:14.652209 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:08:23.575232 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:23.575195 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-65c67f598f-wb248" Jul 15 21:08:25.366771 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:25.366732 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:08:25.542860 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:25.542825 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:08:26.201246 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:26.201206 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-2mq85"] Jul 15 21:08:26.206923 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:26.206897 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-2mq85" Jul 15 21:08:26.209735 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:26.209707 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 15 21:08:26.209852 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:26.209785 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-tf26c\"" Jul 15 21:08:26.215699 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:26.215673 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-2mq85"] Jul 15 21:08:26.242696 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:26.242647 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:08:26.319224 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:26.319183 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5g4q\" (UniqueName: \"kubernetes.io/projected/c75adccd-fdbc-47a6-a744-c7020160ebb5-kube-api-access-w5g4q\") pod \"odh-model-controller-7998c67dc4-2mq85\" (UID: \"c75adccd-fdbc-47a6-a744-c7020160ebb5\") " pod="opendatahub/odh-model-controller-7998c67dc4-2mq85" Jul 15 21:08:26.319408 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:26.319265 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c75adccd-fdbc-47a6-a744-c7020160ebb5-cert\") pod \"odh-model-controller-7998c67dc4-2mq85\" (UID: \"c75adccd-fdbc-47a6-a744-c7020160ebb5\") " pod="opendatahub/odh-model-controller-7998c67dc4-2mq85" Jul 15 21:08:26.367673 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:26.367639 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:08:26.419650 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:26.419618 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c75adccd-fdbc-47a6-a744-c7020160ebb5-cert\") pod \"odh-model-controller-7998c67dc4-2mq85\" (UID: \"c75adccd-fdbc-47a6-a744-c7020160ebb5\") " pod="opendatahub/odh-model-controller-7998c67dc4-2mq85" Jul 15 21:08:26.419853 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:26.419684 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5g4q\" (UniqueName: \"kubernetes.io/projected/c75adccd-fdbc-47a6-a744-c7020160ebb5-kube-api-access-w5g4q\") pod \"odh-model-controller-7998c67dc4-2mq85\" (UID: \"c75adccd-fdbc-47a6-a744-c7020160ebb5\") " pod="opendatahub/odh-model-controller-7998c67dc4-2mq85" Jul 15 21:08:26.419853 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:08:26.419773 2579 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 15 21:08:26.419853 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:08:26.419851 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c75adccd-fdbc-47a6-a744-c7020160ebb5-cert podName:c75adccd-fdbc-47a6-a744-c7020160ebb5 nodeName:}" failed. No retries permitted until 2026-07-15 21:08:26.919830943 +0000 UTC m=+411.461143474 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c75adccd-fdbc-47a6-a744-c7020160ebb5-cert") pod "odh-model-controller-7998c67dc4-2mq85" (UID: "c75adccd-fdbc-47a6-a744-c7020160ebb5") : secret "odh-model-controller-webhook-cert" not found Jul 15 21:08:26.429754 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:26.429718 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5g4q\" (UniqueName: \"kubernetes.io/projected/c75adccd-fdbc-47a6-a744-c7020160ebb5-kube-api-access-w5g4q\") pod \"odh-model-controller-7998c67dc4-2mq85\" (UID: \"c75adccd-fdbc-47a6-a744-c7020160ebb5\") " pod="opendatahub/odh-model-controller-7998c67dc4-2mq85" Jul 15 21:08:26.467149 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:26.467028 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:08:26.566742 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:26.566690 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:08:26.923743 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:26.923709 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c75adccd-fdbc-47a6-a744-c7020160ebb5-cert\") pod \"odh-model-controller-7998c67dc4-2mq85\" (UID: \"c75adccd-fdbc-47a6-a744-c7020160ebb5\") " pod="opendatahub/odh-model-controller-7998c67dc4-2mq85" Jul 15 21:08:26.926554 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:26.926521 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c75adccd-fdbc-47a6-a744-c7020160ebb5-cert\") pod \"odh-model-controller-7998c67dc4-2mq85\" (UID: \"c75adccd-fdbc-47a6-a744-c7020160ebb5\") " pod="opendatahub/odh-model-controller-7998c67dc4-2mq85" Jul 15 21:08:26.939812 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:26.939775 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:08:27.117507 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:27.117477 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-2mq85" Jul 15 21:08:27.267568 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:27.267536 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-2mq85"] Jul 15 21:08:27.269545 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:08:27.269512 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc75adccd_fdbc_47a6_a744_c7020160ebb5.slice/crio-9d2bba6af52bc52894b8100b42b5bba933d1339fe3c2b7d66df39a6f705c42b0 WatchSource:0}: Error finding container 9d2bba6af52bc52894b8100b42b5bba933d1339fe3c2b7d66df39a6f705c42b0: Status 404 returned error can't find the container with id 9d2bba6af52bc52894b8100b42b5bba933d1339fe3c2b7d66df39a6f705c42b0 Jul 15 21:08:27.632939 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:27.632892 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-2mq85" event={"ID":"c75adccd-fdbc-47a6-a744-c7020160ebb5","Type":"ContainerStarted","Data":"9d2bba6af52bc52894b8100b42b5bba933d1339fe3c2b7d66df39a6f705c42b0"} Jul 15 21:08:27.660188 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:27.660153 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:08:31.431809 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:31.431766 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:08:31.497247 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:31.497211 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6f6cc69c87-gvww8"] Jul 15 21:08:31.500475 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:31.500456 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6f6cc69c87-gvww8" Jul 15 21:08:31.503507 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:31.503484 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 15 21:08:31.503639 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:31.503492 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 15 21:08:31.504999 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:31.504975 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-p2qjl\"" Jul 15 21:08:31.505107 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:31.505023 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 15 21:08:31.505107 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:31.505023 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 15 21:08:31.516183 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:31.516158 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6f6cc69c87-gvww8"] Jul 15 21:08:31.656212 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:31.656177 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-2mq85" event={"ID":"c75adccd-fdbc-47a6-a744-c7020160ebb5","Type":"ContainerStarted","Data":"0d1bc41716da076f771d8b46d81c0cd7f579c2ce52d527173f6cbde423967cee"} Jul 15 21:08:31.656381 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:31.656335 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-2mq85" Jul 15 21:08:31.663428 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:31.663398 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1273fbd3-a61d-44b9-91cf-b796085ddffa-tmp\") pod \"kube-auth-proxy-6f6cc69c87-gvww8\" (UID: \"1273fbd3-a61d-44b9-91cf-b796085ddffa\") " pod="openshift-ingress/kube-auth-proxy-6f6cc69c87-gvww8" Jul 15 21:08:31.663583 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:31.663446 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1273fbd3-a61d-44b9-91cf-b796085ddffa-tls-certs\") pod \"kube-auth-proxy-6f6cc69c87-gvww8\" (UID: \"1273fbd3-a61d-44b9-91cf-b796085ddffa\") " pod="openshift-ingress/kube-auth-proxy-6f6cc69c87-gvww8" Jul 15 21:08:31.663583 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:31.663475 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7865x\" (UniqueName: \"kubernetes.io/projected/1273fbd3-a61d-44b9-91cf-b796085ddffa-kube-api-access-7865x\") pod \"kube-auth-proxy-6f6cc69c87-gvww8\" (UID: \"1273fbd3-a61d-44b9-91cf-b796085ddffa\") " pod="openshift-ingress/kube-auth-proxy-6f6cc69c87-gvww8" Jul 15 21:08:31.676034 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:31.675974 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-2mq85" podStartSLOduration=1.955448557 podStartE2EDuration="5.675953346s" totalImagesPullingTime="3.720504789s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:08:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:08:31.67458152 +0000 UTC m=+416.215894073" watchObservedRunningTime="2026-07-15 21:08:31.675953346 +0000 UTC m=+416.217265901" Jul 15 21:08:31.764081 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:31.763972 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1273fbd3-a61d-44b9-91cf-b796085ddffa-tmp\") pod \"kube-auth-proxy-6f6cc69c87-gvww8\" (UID: \"1273fbd3-a61d-44b9-91cf-b796085ddffa\") " pod="openshift-ingress/kube-auth-proxy-6f6cc69c87-gvww8" Jul 15 21:08:31.764239 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:31.764137 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1273fbd3-a61d-44b9-91cf-b796085ddffa-tls-certs\") pod \"kube-auth-proxy-6f6cc69c87-gvww8\" (UID: \"1273fbd3-a61d-44b9-91cf-b796085ddffa\") " pod="openshift-ingress/kube-auth-proxy-6f6cc69c87-gvww8" Jul 15 21:08:31.764239 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:31.764170 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7865x\" (UniqueName: \"kubernetes.io/projected/1273fbd3-a61d-44b9-91cf-b796085ddffa-kube-api-access-7865x\") pod \"kube-auth-proxy-6f6cc69c87-gvww8\" (UID: \"1273fbd3-a61d-44b9-91cf-b796085ddffa\") " pod="openshift-ingress/kube-auth-proxy-6f6cc69c87-gvww8" Jul 15 21:08:31.766442 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:31.766411 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1273fbd3-a61d-44b9-91cf-b796085ddffa-tmp\") pod \"kube-auth-proxy-6f6cc69c87-gvww8\" (UID: \"1273fbd3-a61d-44b9-91cf-b796085ddffa\") " pod="openshift-ingress/kube-auth-proxy-6f6cc69c87-gvww8" Jul 15 21:08:31.766633 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:31.766612 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1273fbd3-a61d-44b9-91cf-b796085ddffa-tls-certs\") pod \"kube-auth-proxy-6f6cc69c87-gvww8\" (UID: \"1273fbd3-a61d-44b9-91cf-b796085ddffa\") " pod="openshift-ingress/kube-auth-proxy-6f6cc69c87-gvww8" Jul 15 21:08:31.774971 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:31.774936 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7865x\" (UniqueName: \"kubernetes.io/projected/1273fbd3-a61d-44b9-91cf-b796085ddffa-kube-api-access-7865x\") pod \"kube-auth-proxy-6f6cc69c87-gvww8\" (UID: \"1273fbd3-a61d-44b9-91cf-b796085ddffa\") " pod="openshift-ingress/kube-auth-proxy-6f6cc69c87-gvww8" Jul 15 21:08:31.803559 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:31.803522 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:08:31.811327 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:31.811304 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6f6cc69c87-gvww8" Jul 15 21:08:31.946988 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:31.946935 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6f6cc69c87-gvww8"] Jul 15 21:08:31.949392 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:08:31.949365 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1273fbd3_a61d_44b9_91cf_b796085ddffa.slice/crio-f461d45906ba14472ab2de2d468ebb848ff0e6e20fb8256ce74e0609b873578a WatchSource:0}: Error finding container f461d45906ba14472ab2de2d468ebb848ff0e6e20fb8256ce74e0609b873578a: Status 404 returned error can't find the container with id f461d45906ba14472ab2de2d468ebb848ff0e6e20fb8256ce74e0609b873578a Jul 15 21:08:32.663565 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:32.663521 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6f6cc69c87-gvww8" event={"ID":"1273fbd3-a61d-44b9-91cf-b796085ddffa","Type":"ContainerStarted","Data":"f461d45906ba14472ab2de2d468ebb848ff0e6e20fb8256ce74e0609b873578a"} Jul 15 21:08:32.740870 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:32.740837 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:08:32.749872 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:32.749839 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-l2f68"] Jul 15 21:08:32.752755 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:32.752739 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-l2f68" Jul 15 21:08:32.755923 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:32.755900 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-mj8wt\"" Jul 15 21:08:32.756174 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:32.756158 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 15 21:08:32.756251 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:32.756159 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 15 21:08:32.756420 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:32.756407 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 15 21:08:32.756556 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:32.756541 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 15 21:08:32.774633 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:32.774588 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-l2f68"] Jul 15 21:08:32.874902 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:32.874863 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a77cf029-d8a2-4595-b61e-05ef69681de0-local-certs\") pod \"istiod-openshift-gateway-7574459656-l2f68\" (UID: \"a77cf029-d8a2-4595-b61e-05ef69681de0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-l2f68" Jul 15 21:08:32.875120 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:32.874932 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a77cf029-d8a2-4595-b61e-05ef69681de0-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-l2f68\" (UID: \"a77cf029-d8a2-4595-b61e-05ef69681de0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-l2f68" Jul 15 21:08:32.875120 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:32.874969 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a77cf029-d8a2-4595-b61e-05ef69681de0-istio-token\") pod \"istiod-openshift-gateway-7574459656-l2f68\" (UID: \"a77cf029-d8a2-4595-b61e-05ef69681de0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-l2f68" Jul 15 21:08:32.875120 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:32.874995 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnxgl\" (UniqueName: \"kubernetes.io/projected/a77cf029-d8a2-4595-b61e-05ef69681de0-kube-api-access-pnxgl\") pod \"istiod-openshift-gateway-7574459656-l2f68\" (UID: \"a77cf029-d8a2-4595-b61e-05ef69681de0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-l2f68" Jul 15 21:08:32.875280 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:32.875110 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a77cf029-d8a2-4595-b61e-05ef69681de0-cacerts\") pod \"istiod-openshift-gateway-7574459656-l2f68\" (UID: \"a77cf029-d8a2-4595-b61e-05ef69681de0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-l2f68" Jul 15 21:08:32.875280 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:32.875150 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a77cf029-d8a2-4595-b61e-05ef69681de0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-l2f68\" (UID: \"a77cf029-d8a2-4595-b61e-05ef69681de0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-l2f68" Jul 15 21:08:32.875280 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:32.875187 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a77cf029-d8a2-4595-b61e-05ef69681de0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-l2f68\" (UID: \"a77cf029-d8a2-4595-b61e-05ef69681de0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-l2f68" Jul 15 21:08:32.881646 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:32.881609 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:08:32.976544 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:32.976448 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a77cf029-d8a2-4595-b61e-05ef69681de0-local-certs\") pod \"istiod-openshift-gateway-7574459656-l2f68\" (UID: \"a77cf029-d8a2-4595-b61e-05ef69681de0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-l2f68" Jul 15 21:08:32.976544 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:32.976521 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a77cf029-d8a2-4595-b61e-05ef69681de0-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-l2f68\" (UID: \"a77cf029-d8a2-4595-b61e-05ef69681de0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-l2f68" Jul 15 21:08:32.976759 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:32.976558 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a77cf029-d8a2-4595-b61e-05ef69681de0-istio-token\") pod \"istiod-openshift-gateway-7574459656-l2f68\" (UID: \"a77cf029-d8a2-4595-b61e-05ef69681de0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-l2f68" Jul 15 21:08:32.976759 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:32.976585 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnxgl\" (UniqueName: \"kubernetes.io/projected/a77cf029-d8a2-4595-b61e-05ef69681de0-kube-api-access-pnxgl\") pod \"istiod-openshift-gateway-7574459656-l2f68\" (UID: \"a77cf029-d8a2-4595-b61e-05ef69681de0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-l2f68" Jul 15 21:08:32.976759 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:32.976719 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a77cf029-d8a2-4595-b61e-05ef69681de0-cacerts\") pod \"istiod-openshift-gateway-7574459656-l2f68\" (UID: \"a77cf029-d8a2-4595-b61e-05ef69681de0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-l2f68" Jul 15 21:08:32.976759 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:32.976750 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a77cf029-d8a2-4595-b61e-05ef69681de0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-l2f68\" (UID: \"a77cf029-d8a2-4595-b61e-05ef69681de0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-l2f68" Jul 15 21:08:32.976971 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:32.976783 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a77cf029-d8a2-4595-b61e-05ef69681de0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-l2f68\" (UID: \"a77cf029-d8a2-4595-b61e-05ef69681de0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-l2f68" Jul 15 21:08:32.977711 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:32.977434 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a77cf029-d8a2-4595-b61e-05ef69681de0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-l2f68\" (UID: \"a77cf029-d8a2-4595-b61e-05ef69681de0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-l2f68" Jul 15 21:08:32.979479 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:32.979455 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a77cf029-d8a2-4595-b61e-05ef69681de0-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-l2f68\" (UID: \"a77cf029-d8a2-4595-b61e-05ef69681de0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-l2f68" Jul 15 21:08:32.979788 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:32.979762 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a77cf029-d8a2-4595-b61e-05ef69681de0-local-certs\") pod \"istiod-openshift-gateway-7574459656-l2f68\" (UID: \"a77cf029-d8a2-4595-b61e-05ef69681de0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-l2f68" Jul 15 21:08:32.979891 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:32.979784 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a77cf029-d8a2-4595-b61e-05ef69681de0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-l2f68\" (UID: \"a77cf029-d8a2-4595-b61e-05ef69681de0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-l2f68" Jul 15 21:08:32.979891 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:32.979849 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a77cf029-d8a2-4595-b61e-05ef69681de0-cacerts\") pod \"istiod-openshift-gateway-7574459656-l2f68\" (UID: \"a77cf029-d8a2-4595-b61e-05ef69681de0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-l2f68" Jul 15 21:08:32.985656 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:32.985625 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a77cf029-d8a2-4595-b61e-05ef69681de0-istio-token\") pod \"istiod-openshift-gateway-7574459656-l2f68\" (UID: \"a77cf029-d8a2-4595-b61e-05ef69681de0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-l2f68" Jul 15 21:08:32.985903 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:32.985885 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnxgl\" (UniqueName: \"kubernetes.io/projected/a77cf029-d8a2-4595-b61e-05ef69681de0-kube-api-access-pnxgl\") pod \"istiod-openshift-gateway-7574459656-l2f68\" (UID: \"a77cf029-d8a2-4595-b61e-05ef69681de0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-l2f68" Jul 15 21:08:33.062354 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:33.062320 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-l2f68" Jul 15 21:08:33.239935 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:33.239852 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-l2f68"] Jul 15 21:08:33.242971 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:08:33.242942 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda77cf029_d8a2_4595_b61e_05ef69681de0.slice/crio-1861daf6347390f59361023ea68b3b12031afa4dcfec8d4f878cbe3f9a88801f WatchSource:0}: Error finding container 1861daf6347390f59361023ea68b3b12031afa4dcfec8d4f878cbe3f9a88801f: Status 404 returned error can't find the container with id 1861daf6347390f59361023ea68b3b12031afa4dcfec8d4f878cbe3f9a88801f Jul 15 21:08:33.669668 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:33.669627 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-l2f68" event={"ID":"a77cf029-d8a2-4595-b61e-05ef69681de0","Type":"ContainerStarted","Data":"1861daf6347390f59361023ea68b3b12031afa4dcfec8d4f878cbe3f9a88801f"} Jul 15 21:08:34.240339 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:34.240266 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:08:35.682327 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:35.682270 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6f6cc69c87-gvww8" event={"ID":"1273fbd3-a61d-44b9-91cf-b796085ddffa","Type":"ContainerStarted","Data":"5494cadbe08a23ea3cd52576f4d6dd643f74dd7ccd3e9698bc2ebe938d19f8da"} Jul 15 21:08:35.732793 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:35.732728 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6f6cc69c87-gvww8" podStartSLOduration=1.6401388639999999 podStartE2EDuration="4.732710768s" totalImagesPullingTime="3.092571904s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:08:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:08:35.730962553 +0000 UTC m=+420.272275106" watchObservedRunningTime="2026-07-15 21:08:35.732710768 +0000 UTC m=+420.274023318" Jul 15 21:08:36.459922 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:36.459877 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:08:36.460037 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:36.459949 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:08:36.687961 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:36.687917 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-l2f68" event={"ID":"a77cf029-d8a2-4595-b61e-05ef69681de0","Type":"ContainerStarted","Data":"1cbb521d69fb86fd4099075bc3524d6e86c4c2f6e1e1185dd7fcb0850bf09ddc"} Jul 15 21:08:36.688392 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:36.688183 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-l2f68" Jul 15 21:08:36.744544 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:36.744437 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-l2f68" podStartSLOduration=1.529968904 podStartE2EDuration="4.744421697s" totalImagesPullingTime="3.214452793s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:08:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:08:36.743686128 +0000 UTC m=+421.284998679" watchObservedRunningTime="2026-07-15 21:08:36.744421697 +0000 UTC m=+421.285734247" Jul 15 21:08:37.007462 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:37.007346 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:08:37.067400 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:37.067364 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-gxtc2"] Jul 15 21:08:37.070628 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:37.070603 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gxtc2" Jul 15 21:08:37.074422 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:37.074404 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 15 21:08:37.074870 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:37.074842 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-qzp5p\"" Jul 15 21:08:37.083768 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:37.083732 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-mcldd"] Jul 15 21:08:37.087386 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:37.087359 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mcldd" Jul 15 21:08:37.089498 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:37.089477 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-gxtc2"] Jul 15 21:08:37.091037 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:37.091018 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 15 21:08:37.091198 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:37.091182 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-crv65\"" Jul 15 21:08:37.104537 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:37.104427 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-mcldd"] Jul 15 21:08:37.130691 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:37.128238 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:08:37.219039 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:37.218973 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pp2qg\" (UniqueName: \"kubernetes.io/projected/ccbdd9b9-e1f6-4075-b80d-9ea7827580cf-kube-api-access-pp2qg\") pod \"llmisvc-controller-manager-777c6bd7d5-mcldd\" (UID: \"ccbdd9b9-e1f6-4075-b80d-9ea7827580cf\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mcldd" Jul 15 21:08:37.219216 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:37.219046 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dfvs\" (UniqueName: \"kubernetes.io/projected/46fc8345-bb02-4935-bfa9-227aba6b8077-kube-api-access-2dfvs\") pod \"kserve-controller-manager-d6f4b65b8-gxtc2\" (UID: \"46fc8345-bb02-4935-bfa9-227aba6b8077\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gxtc2" Jul 15 21:08:37.219216 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:37.219102 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ccbdd9b9-e1f6-4075-b80d-9ea7827580cf-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-mcldd\" (UID: \"ccbdd9b9-e1f6-4075-b80d-9ea7827580cf\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mcldd" Jul 15 21:08:37.219216 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:37.219181 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/46fc8345-bb02-4935-bfa9-227aba6b8077-cert\") pod \"kserve-controller-manager-d6f4b65b8-gxtc2\" (UID: \"46fc8345-bb02-4935-bfa9-227aba6b8077\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gxtc2" Jul 15 21:08:37.295039 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:37.294939 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:08:37.320175 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:37.320135 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pp2qg\" (UniqueName: \"kubernetes.io/projected/ccbdd9b9-e1f6-4075-b80d-9ea7827580cf-kube-api-access-pp2qg\") pod \"llmisvc-controller-manager-777c6bd7d5-mcldd\" (UID: \"ccbdd9b9-e1f6-4075-b80d-9ea7827580cf\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mcldd" Jul 15 21:08:37.320341 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:37.320191 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dfvs\" (UniqueName: \"kubernetes.io/projected/46fc8345-bb02-4935-bfa9-227aba6b8077-kube-api-access-2dfvs\") pod \"kserve-controller-manager-d6f4b65b8-gxtc2\" (UID: \"46fc8345-bb02-4935-bfa9-227aba6b8077\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gxtc2" Jul 15 21:08:37.320341 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:37.320307 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ccbdd9b9-e1f6-4075-b80d-9ea7827580cf-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-mcldd\" (UID: \"ccbdd9b9-e1f6-4075-b80d-9ea7827580cf\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mcldd" Jul 15 21:08:37.320414 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:37.320362 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/46fc8345-bb02-4935-bfa9-227aba6b8077-cert\") pod \"kserve-controller-manager-d6f4b65b8-gxtc2\" (UID: \"46fc8345-bb02-4935-bfa9-227aba6b8077\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gxtc2" Jul 15 21:08:37.322932 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:37.322905 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ccbdd9b9-e1f6-4075-b80d-9ea7827580cf-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-mcldd\" (UID: \"ccbdd9b9-e1f6-4075-b80d-9ea7827580cf\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mcldd" Jul 15 21:08:37.322932 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:37.322921 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/46fc8345-bb02-4935-bfa9-227aba6b8077-cert\") pod \"kserve-controller-manager-d6f4b65b8-gxtc2\" (UID: \"46fc8345-bb02-4935-bfa9-227aba6b8077\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gxtc2" Jul 15 21:08:37.332188 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:37.332161 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pp2qg\" (UniqueName: \"kubernetes.io/projected/ccbdd9b9-e1f6-4075-b80d-9ea7827580cf-kube-api-access-pp2qg\") pod \"llmisvc-controller-manager-777c6bd7d5-mcldd\" (UID: \"ccbdd9b9-e1f6-4075-b80d-9ea7827580cf\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mcldd" Jul 15 21:08:37.339243 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:37.339209 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dfvs\" (UniqueName: \"kubernetes.io/projected/46fc8345-bb02-4935-bfa9-227aba6b8077-kube-api-access-2dfvs\") pod \"kserve-controller-manager-d6f4b65b8-gxtc2\" (UID: \"46fc8345-bb02-4935-bfa9-227aba6b8077\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gxtc2" Jul 15 21:08:37.386124 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:37.386084 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gxtc2" Jul 15 21:08:37.398425 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:37.398396 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mcldd" Jul 15 21:08:37.496422 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:37.496337 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:08:37.592509 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:37.592478 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-gxtc2"] Jul 15 21:08:37.594190 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:08:37.594153 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46fc8345_bb02_4935_bfa9_227aba6b8077.slice/crio-184887f4126bebe174e40c751577583fb385dc38050b5afffc9120c5cf266477 WatchSource:0}: Error finding container 184887f4126bebe174e40c751577583fb385dc38050b5afffc9120c5cf266477: Status 404 returned error can't find the container with id 184887f4126bebe174e40c751577583fb385dc38050b5afffc9120c5cf266477 Jul 15 21:08:37.598632 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:37.598608 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-mcldd"] Jul 15 21:08:37.601239 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:08:37.601208 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podccbdd9b9_e1f6_4075_b80d_9ea7827580cf.slice/crio-bc9ce1956a489c20c7a47330867589cec141cd619d4a1067d73e8dbcf11e692b WatchSource:0}: Error finding container bc9ce1956a489c20c7a47330867589cec141cd619d4a1067d73e8dbcf11e692b: Status 404 returned error can't find the container with id bc9ce1956a489c20c7a47330867589cec141cd619d4a1067d73e8dbcf11e692b Jul 15 21:08:37.693503 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:37.693466 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gxtc2" event={"ID":"46fc8345-bb02-4935-bfa9-227aba6b8077","Type":"ContainerStarted","Data":"184887f4126bebe174e40c751577583fb385dc38050b5afffc9120c5cf266477"} Jul 15 21:08:37.694640 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:37.694620 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mcldd" event={"ID":"ccbdd9b9-e1f6-4075-b80d-9ea7827580cf","Type":"ContainerStarted","Data":"bc9ce1956a489c20c7a47330867589cec141cd619d4a1067d73e8dbcf11e692b"} Jul 15 21:08:37.696310 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:37.696282 2579 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-l2f68 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 15 21:08:37.696408 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:37.696343 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-l2f68" podUID="a77cf029-d8a2-4595-b61e-05ef69681de0" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 21:08:38.308333 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:38.308293 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:08:38.531666 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:38.531629 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:08:38.550719 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:38.550689 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zskbp"] Jul 15 21:08:38.553750 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:38.553726 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zskbp" Jul 15 21:08:38.557216 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:38.557189 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-88s9g\"" Jul 15 21:08:38.636758 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:38.636722 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zskbp"] Jul 15 21:08:38.663505 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:38.663454 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:08:38.732766 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:38.732727 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0aa66e88-9447-4b63-8141-7f3bedff0ae5-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zskbp\" (UID: \"0aa66e88-9447-4b63-8141-7f3bedff0ae5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zskbp" Jul 15 21:08:38.733188 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:38.732778 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0aa66e88-9447-4b63-8141-7f3bedff0ae5-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zskbp\" (UID: \"0aa66e88-9447-4b63-8141-7f3bedff0ae5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zskbp" Jul 15 21:08:38.733188 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:38.732811 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0aa66e88-9447-4b63-8141-7f3bedff0ae5-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zskbp\" (UID: \"0aa66e88-9447-4b63-8141-7f3bedff0ae5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zskbp" Jul 15 21:08:38.733188 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:38.732883 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0aa66e88-9447-4b63-8141-7f3bedff0ae5-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zskbp\" (UID: \"0aa66e88-9447-4b63-8141-7f3bedff0ae5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zskbp" Jul 15 21:08:38.733188 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:38.732923 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0aa66e88-9447-4b63-8141-7f3bedff0ae5-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zskbp\" (UID: \"0aa66e88-9447-4b63-8141-7f3bedff0ae5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zskbp" Jul 15 21:08:38.733188 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:38.732953 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0aa66e88-9447-4b63-8141-7f3bedff0ae5-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zskbp\" (UID: \"0aa66e88-9447-4b63-8141-7f3bedff0ae5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zskbp" Jul 15 21:08:38.733188 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:38.732994 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0aa66e88-9447-4b63-8141-7f3bedff0ae5-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zskbp\" (UID: \"0aa66e88-9447-4b63-8141-7f3bedff0ae5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zskbp" Jul 15 21:08:38.733188 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:38.733026 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqqtz\" (UniqueName: \"kubernetes.io/projected/0aa66e88-9447-4b63-8141-7f3bedff0ae5-kube-api-access-kqqtz\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zskbp\" (UID: \"0aa66e88-9447-4b63-8141-7f3bedff0ae5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zskbp" Jul 15 21:08:38.733188 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:38.733056 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0aa66e88-9447-4b63-8141-7f3bedff0ae5-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zskbp\" (UID: \"0aa66e88-9447-4b63-8141-7f3bedff0ae5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zskbp" Jul 15 21:08:38.764295 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:38.764229 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:08:38.833887 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:38.833810 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0aa66e88-9447-4b63-8141-7f3bedff0ae5-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zskbp\" (UID: \"0aa66e88-9447-4b63-8141-7f3bedff0ae5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zskbp" Jul 15 21:08:38.833887 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:38.833854 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqqtz\" (UniqueName: \"kubernetes.io/projected/0aa66e88-9447-4b63-8141-7f3bedff0ae5-kube-api-access-kqqtz\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zskbp\" (UID: \"0aa66e88-9447-4b63-8141-7f3bedff0ae5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zskbp" Jul 15 21:08:38.834119 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:38.833930 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0aa66e88-9447-4b63-8141-7f3bedff0ae5-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zskbp\" (UID: \"0aa66e88-9447-4b63-8141-7f3bedff0ae5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zskbp" Jul 15 21:08:38.834119 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:38.834015 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0aa66e88-9447-4b63-8141-7f3bedff0ae5-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zskbp\" (UID: \"0aa66e88-9447-4b63-8141-7f3bedff0ae5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zskbp" Jul 15 21:08:38.834119 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:38.834044 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0aa66e88-9447-4b63-8141-7f3bedff0ae5-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zskbp\" (UID: \"0aa66e88-9447-4b63-8141-7f3bedff0ae5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zskbp" Jul 15 21:08:38.834293 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:38.834123 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0aa66e88-9447-4b63-8141-7f3bedff0ae5-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zskbp\" (UID: \"0aa66e88-9447-4b63-8141-7f3bedff0ae5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zskbp" Jul 15 21:08:38.834293 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:38.834193 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0aa66e88-9447-4b63-8141-7f3bedff0ae5-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zskbp\" (UID: \"0aa66e88-9447-4b63-8141-7f3bedff0ae5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zskbp" Jul 15 21:08:38.834293 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:38.834251 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0aa66e88-9447-4b63-8141-7f3bedff0ae5-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zskbp\" (UID: \"0aa66e88-9447-4b63-8141-7f3bedff0ae5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zskbp" Jul 15 21:08:38.834293 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:38.834278 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0aa66e88-9447-4b63-8141-7f3bedff0ae5-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zskbp\" (UID: \"0aa66e88-9447-4b63-8141-7f3bedff0ae5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zskbp" Jul 15 21:08:38.834481 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:38.834396 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0aa66e88-9447-4b63-8141-7f3bedff0ae5-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zskbp\" (UID: \"0aa66e88-9447-4b63-8141-7f3bedff0ae5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zskbp" Jul 15 21:08:38.834481 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:38.834419 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0aa66e88-9447-4b63-8141-7f3bedff0ae5-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zskbp\" (UID: \"0aa66e88-9447-4b63-8141-7f3bedff0ae5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zskbp" Jul 15 21:08:38.834728 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:38.834704 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0aa66e88-9447-4b63-8141-7f3bedff0ae5-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zskbp\" (UID: \"0aa66e88-9447-4b63-8141-7f3bedff0ae5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zskbp" Jul 15 21:08:38.834823 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:38.834730 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0aa66e88-9447-4b63-8141-7f3bedff0ae5-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zskbp\" (UID: \"0aa66e88-9447-4b63-8141-7f3bedff0ae5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zskbp" Jul 15 21:08:38.835003 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:38.834982 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0aa66e88-9447-4b63-8141-7f3bedff0ae5-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zskbp\" (UID: \"0aa66e88-9447-4b63-8141-7f3bedff0ae5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zskbp" Jul 15 21:08:38.836751 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:38.836726 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0aa66e88-9447-4b63-8141-7f3bedff0ae5-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zskbp\" (UID: \"0aa66e88-9447-4b63-8141-7f3bedff0ae5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zskbp" Jul 15 21:08:38.836946 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:38.836925 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0aa66e88-9447-4b63-8141-7f3bedff0ae5-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zskbp\" (UID: \"0aa66e88-9447-4b63-8141-7f3bedff0ae5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zskbp" Jul 15 21:08:38.845981 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:38.845955 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0aa66e88-9447-4b63-8141-7f3bedff0ae5-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zskbp\" (UID: \"0aa66e88-9447-4b63-8141-7f3bedff0ae5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zskbp" Jul 15 21:08:38.846991 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:38.846967 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqqtz\" (UniqueName: \"kubernetes.io/projected/0aa66e88-9447-4b63-8141-7f3bedff0ae5-kube-api-access-kqqtz\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zskbp\" (UID: \"0aa66e88-9447-4b63-8141-7f3bedff0ae5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zskbp" Jul 15 21:08:38.864403 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:38.864340 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zskbp" Jul 15 21:08:38.864746 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:38.864712 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:08:39.156193 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:39.156160 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zskbp"] Jul 15 21:08:39.157102 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:08:39.157052 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0aa66e88_9447_4b63_8141_7f3bedff0ae5.slice/crio-8d2d2311b051afd25ce7b7a1a08bd0b121fc3d0b58f537bf69e7caf7d602d9c8 WatchSource:0}: Error finding container 8d2d2311b051afd25ce7b7a1a08bd0b121fc3d0b58f537bf69e7caf7d602d9c8: Status 404 returned error can't find the container with id 8d2d2311b051afd25ce7b7a1a08bd0b121fc3d0b58f537bf69e7caf7d602d9c8 Jul 15 21:08:39.705950 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:39.705914 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zskbp" event={"ID":"0aa66e88-9447-4b63-8141-7f3bedff0ae5","Type":"ContainerStarted","Data":"8d2d2311b051afd25ce7b7a1a08bd0b121fc3d0b58f537bf69e7caf7d602d9c8"} Jul 15 21:08:40.696362 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:40.696233 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-l2f68" Jul 15 21:08:42.464044 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:42.464013 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:08:42.464501 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:42.464095 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:08:42.464501 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:42.464126 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:08:42.666394 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:42.666362 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-2mq85" Jul 15 21:08:42.723591 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:42.723487 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mcldd" event={"ID":"ccbdd9b9-e1f6-4075-b80d-9ea7827580cf","Type":"ContainerStarted","Data":"0d45cfc026d5db823492e31b8cd7f8fa89e5b1abbcfed0f598b4f0604475a7fd"} Jul 15 21:08:42.723815 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:42.723650 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mcldd" Jul 15 21:08:42.725329 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:42.725287 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gxtc2" event={"ID":"46fc8345-bb02-4935-bfa9-227aba6b8077","Type":"ContainerStarted","Data":"ec2657694d8391fad7704d004bd1b4f7aa8fff8f1a175a9ff222d3b3065d72d4"} Jul 15 21:08:42.725570 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:42.725504 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gxtc2" Jul 15 21:08:42.726957 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:42.726934 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zskbp" event={"ID":"0aa66e88-9447-4b63-8141-7f3bedff0ae5","Type":"ContainerStarted","Data":"4978cb2e114f1804b7fb71496db7c8a6b0f8b1587109cb8b456fd610287ab1f1"} Jul 15 21:08:42.746390 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:42.746322 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mcldd" podStartSLOduration=0.892875449 podStartE2EDuration="5.746298533s" totalImagesPullingTime="4.853423084s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:08:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:08:42.743671363 +0000 UTC m=+427.284983914" watchObservedRunningTime="2026-07-15 21:08:42.746298533 +0000 UTC m=+427.287611084" Jul 15 21:08:42.769657 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:42.769594 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zskbp" podStartSLOduration=1.465479376 podStartE2EDuration="4.769574251s" totalImagesPullingTime="3.304094875s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:08:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:08:42.767284237 +0000 UTC m=+427.308596790" watchObservedRunningTime="2026-07-15 21:08:42.769574251 +0000 UTC m=+427.310886802" Jul 15 21:08:42.785883 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:42.785827 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gxtc2" podStartSLOduration=0.924715441 podStartE2EDuration="5.785811126s" totalImagesPullingTime="4.861095685s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:08:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:08:42.785052199 +0000 UTC m=+427.326364751" watchObservedRunningTime="2026-07-15 21:08:42.785811126 +0000 UTC m=+427.327123677" Jul 15 21:08:42.865384 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:42.865348 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zskbp" Jul 15 21:08:43.870144 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:43.870113 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zskbp" Jul 15 21:08:44.737017 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:44.736979 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zskbp" Jul 15 21:08:44.737935 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:44.737917 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zskbp" Jul 15 21:08:52.773044 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:52.773007 2579 generic.go:356] "Generic (PLEG): container finished" podID="46fc8345-bb02-4935-bfa9-227aba6b8077" containerID="ec2657694d8391fad7704d004bd1b4f7aa8fff8f1a175a9ff222d3b3065d72d4" exitCode=1 Jul 15 21:08:52.773391 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:52.773097 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gxtc2" event={"ID":"46fc8345-bb02-4935-bfa9-227aba6b8077","Type":"ContainerDied","Data":"ec2657694d8391fad7704d004bd1b4f7aa8fff8f1a175a9ff222d3b3065d72d4"} Jul 15 21:08:52.773445 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:52.773432 2579 scope.go:117] "RemoveContainer" containerID="ec2657694d8391fad7704d004bd1b4f7aa8fff8f1a175a9ff222d3b3065d72d4" Jul 15 21:08:53.000907 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:53.000821 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-l2f68"] Jul 15 21:08:53.001049 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:53.001023 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-l2f68" podUID="a77cf029-d8a2-4595-b61e-05ef69681de0" containerName="discovery" containerID="cri-o://1cbb521d69fb86fd4099075bc3524d6e86c4c2f6e1e1185dd7fcb0850bf09ddc" gracePeriod=30 Jul 15 21:08:53.256361 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:53.256302 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-l2f68" Jul 15 21:08:53.361320 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:53.361289 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a77cf029-d8a2-4595-b61e-05ef69681de0-istio-token\") pod \"a77cf029-d8a2-4595-b61e-05ef69681de0\" (UID: \"a77cf029-d8a2-4595-b61e-05ef69681de0\") " Jul 15 21:08:53.361525 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:53.361350 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a77cf029-d8a2-4595-b61e-05ef69681de0-istio-csr-ca-configmap\") pod \"a77cf029-d8a2-4595-b61e-05ef69681de0\" (UID: \"a77cf029-d8a2-4595-b61e-05ef69681de0\") " Jul 15 21:08:53.361525 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:53.361404 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a77cf029-d8a2-4595-b61e-05ef69681de0-local-certs\") pod \"a77cf029-d8a2-4595-b61e-05ef69681de0\" (UID: \"a77cf029-d8a2-4595-b61e-05ef69681de0\") " Jul 15 21:08:53.361648 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:53.361563 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a77cf029-d8a2-4595-b61e-05ef69681de0-istio-kubeconfig\") pod \"a77cf029-d8a2-4595-b61e-05ef69681de0\" (UID: \"a77cf029-d8a2-4595-b61e-05ef69681de0\") " Jul 15 21:08:53.361648 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:53.361625 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a77cf029-d8a2-4595-b61e-05ef69681de0-istio-csr-dns-cert\") pod \"a77cf029-d8a2-4595-b61e-05ef69681de0\" (UID: \"a77cf029-d8a2-4595-b61e-05ef69681de0\") " Jul 15 21:08:53.361747 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:53.361656 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a77cf029-d8a2-4595-b61e-05ef69681de0-cacerts\") pod \"a77cf029-d8a2-4595-b61e-05ef69681de0\" (UID: \"a77cf029-d8a2-4595-b61e-05ef69681de0\") " Jul 15 21:08:53.361747 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:53.361703 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pnxgl\" (UniqueName: \"kubernetes.io/projected/a77cf029-d8a2-4595-b61e-05ef69681de0-kube-api-access-pnxgl\") pod \"a77cf029-d8a2-4595-b61e-05ef69681de0\" (UID: \"a77cf029-d8a2-4595-b61e-05ef69681de0\") " Jul 15 21:08:53.361855 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:53.361743 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a77cf029-d8a2-4595-b61e-05ef69681de0-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "a77cf029-d8a2-4595-b61e-05ef69681de0" (UID: "a77cf029-d8a2-4595-b61e-05ef69681de0"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 21:08:53.361981 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:53.361960 2579 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a77cf029-d8a2-4595-b61e-05ef69681de0-istio-csr-ca-configmap\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:08:53.363846 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:53.363816 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a77cf029-d8a2-4595-b61e-05ef69681de0-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "a77cf029-d8a2-4595-b61e-05ef69681de0" (UID: "a77cf029-d8a2-4595-b61e-05ef69681de0"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 21:08:53.363846 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:53.363829 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a77cf029-d8a2-4595-b61e-05ef69681de0-local-certs" (OuterVolumeSpecName: "local-certs") pod "a77cf029-d8a2-4595-b61e-05ef69681de0" (UID: "a77cf029-d8a2-4595-b61e-05ef69681de0"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:08:53.364016 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:53.363962 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a77cf029-d8a2-4595-b61e-05ef69681de0-istio-token" (OuterVolumeSpecName: "istio-token") pod "a77cf029-d8a2-4595-b61e-05ef69681de0" (UID: "a77cf029-d8a2-4595-b61e-05ef69681de0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:08:53.364141 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:53.364120 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a77cf029-d8a2-4595-b61e-05ef69681de0-cacerts" (OuterVolumeSpecName: "cacerts") pod "a77cf029-d8a2-4595-b61e-05ef69681de0" (UID: "a77cf029-d8a2-4595-b61e-05ef69681de0"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 21:08:53.364194 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:53.364140 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a77cf029-d8a2-4595-b61e-05ef69681de0-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "a77cf029-d8a2-4595-b61e-05ef69681de0" (UID: "a77cf029-d8a2-4595-b61e-05ef69681de0"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 21:08:53.364232 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:53.364221 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a77cf029-d8a2-4595-b61e-05ef69681de0-kube-api-access-pnxgl" (OuterVolumeSpecName: "kube-api-access-pnxgl") pod "a77cf029-d8a2-4595-b61e-05ef69681de0" (UID: "a77cf029-d8a2-4595-b61e-05ef69681de0"). InnerVolumeSpecName "kube-api-access-pnxgl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:08:53.462916 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:53.462876 2579 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a77cf029-d8a2-4595-b61e-05ef69681de0-local-certs\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:08:53.462916 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:53.462915 2579 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a77cf029-d8a2-4595-b61e-05ef69681de0-istio-kubeconfig\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:08:53.463153 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:53.462930 2579 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a77cf029-d8a2-4595-b61e-05ef69681de0-istio-csr-dns-cert\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:08:53.463153 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:53.462946 2579 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a77cf029-d8a2-4595-b61e-05ef69681de0-cacerts\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:08:53.463153 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:53.462962 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pnxgl\" (UniqueName: \"kubernetes.io/projected/a77cf029-d8a2-4595-b61e-05ef69681de0-kube-api-access-pnxgl\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:08:53.463153 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:53.462975 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a77cf029-d8a2-4595-b61e-05ef69681de0-istio-token\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:08:53.779556 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:53.779516 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gxtc2" event={"ID":"46fc8345-bb02-4935-bfa9-227aba6b8077","Type":"ContainerStarted","Data":"ee64779c84f1c3235c4b249811eab98222058861d319c3d0d5b2afab2e129dc7"} Jul 15 21:08:53.779945 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:53.779926 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gxtc2" Jul 15 21:08:53.780867 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:53.780843 2579 generic.go:356] "Generic (PLEG): container finished" podID="a77cf029-d8a2-4595-b61e-05ef69681de0" containerID="1cbb521d69fb86fd4099075bc3524d6e86c4c2f6e1e1185dd7fcb0850bf09ddc" exitCode=0 Jul 15 21:08:53.780968 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:53.780880 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-l2f68" Jul 15 21:08:53.780968 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:53.780927 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-l2f68" event={"ID":"a77cf029-d8a2-4595-b61e-05ef69681de0","Type":"ContainerDied","Data":"1cbb521d69fb86fd4099075bc3524d6e86c4c2f6e1e1185dd7fcb0850bf09ddc"} Jul 15 21:08:53.781034 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:53.780973 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-l2f68" event={"ID":"a77cf029-d8a2-4595-b61e-05ef69681de0","Type":"ContainerDied","Data":"1861daf6347390f59361023ea68b3b12031afa4dcfec8d4f878cbe3f9a88801f"} Jul 15 21:08:53.781034 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:53.781003 2579 scope.go:117] "RemoveContainer" containerID="1cbb521d69fb86fd4099075bc3524d6e86c4c2f6e1e1185dd7fcb0850bf09ddc" Jul 15 21:08:53.789367 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:53.789347 2579 scope.go:117] "RemoveContainer" containerID="1cbb521d69fb86fd4099075bc3524d6e86c4c2f6e1e1185dd7fcb0850bf09ddc" Jul 15 21:08:53.789645 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:08:53.789620 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1cbb521d69fb86fd4099075bc3524d6e86c4c2f6e1e1185dd7fcb0850bf09ddc\": container with ID starting with 1cbb521d69fb86fd4099075bc3524d6e86c4c2f6e1e1185dd7fcb0850bf09ddc not found: ID does not exist" containerID="1cbb521d69fb86fd4099075bc3524d6e86c4c2f6e1e1185dd7fcb0850bf09ddc" Jul 15 21:08:53.789702 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:53.789655 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1cbb521d69fb86fd4099075bc3524d6e86c4c2f6e1e1185dd7fcb0850bf09ddc"} err="failed to get container status \"1cbb521d69fb86fd4099075bc3524d6e86c4c2f6e1e1185dd7fcb0850bf09ddc\": rpc error: code = NotFound desc = could not find container \"1cbb521d69fb86fd4099075bc3524d6e86c4c2f6e1e1185dd7fcb0850bf09ddc\": container with ID starting with 1cbb521d69fb86fd4099075bc3524d6e86c4c2f6e1e1185dd7fcb0850bf09ddc not found: ID does not exist" Jul 15 21:08:53.822890 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:53.822847 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-l2f68"] Jul 15 21:08:53.829825 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:53.829798 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-l2f68"] Jul 15 21:08:53.996581 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:08:53.996501 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a77cf029-d8a2-4595-b61e-05ef69681de0" path="/var/lib/kubelet/pods/a77cf029-d8a2-4595-b61e-05ef69681de0/volumes" Jul 15 21:09:13.734639 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:13.734565 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mcldd" Jul 15 21:09:24.791648 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:24.791613 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gxtc2" Jul 15 21:09:26.929150 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:26.929104 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:09:27.019084 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:27.019017 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:09:27.042834 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:27.042801 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jgjll"] Jul 15 21:09:27.043122 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:27.043109 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a77cf029-d8a2-4595-b61e-05ef69681de0" containerName="discovery" Jul 15 21:09:27.043172 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:27.043123 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a77cf029-d8a2-4595-b61e-05ef69681de0" containerName="discovery" Jul 15 21:09:27.043211 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:27.043184 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a77cf029-d8a2-4595-b61e-05ef69681de0" containerName="discovery" Jul 15 21:09:27.055148 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:27.055119 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jgjll" Jul 15 21:09:27.058291 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:27.058266 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 15 21:09:27.059090 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:27.059035 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jgjll"] Jul 15 21:09:27.059967 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:27.059826 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-tbmdp\"" Jul 15 21:09:27.059967 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:27.059834 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 15 21:09:27.118377 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:27.118334 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:09:27.242009 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:27.241911 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxvn9\" (UniqueName: \"kubernetes.io/projected/a80c0837-3b93-47da-b0f7-5b05cdb8c631-kube-api-access-sxvn9\") pod \"kuadrant-operator-catalog-jgjll\" (UID: \"a80c0837-3b93-47da-b0f7-5b05cdb8c631\") " pod="kuadrant-system/kuadrant-operator-catalog-jgjll" Jul 15 21:09:27.342615 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:27.342583 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxvn9\" (UniqueName: \"kubernetes.io/projected/a80c0837-3b93-47da-b0f7-5b05cdb8c631-kube-api-access-sxvn9\") pod \"kuadrant-operator-catalog-jgjll\" (UID: \"a80c0837-3b93-47da-b0f7-5b05cdb8c631\") " pod="kuadrant-system/kuadrant-operator-catalog-jgjll" Jul 15 21:09:27.351399 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:27.351361 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxvn9\" (UniqueName: \"kubernetes.io/projected/a80c0837-3b93-47da-b0f7-5b05cdb8c631-kube-api-access-sxvn9\") pod \"kuadrant-operator-catalog-jgjll\" (UID: \"a80c0837-3b93-47da-b0f7-5b05cdb8c631\") " pod="kuadrant-system/kuadrant-operator-catalog-jgjll" Jul 15 21:09:27.367030 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:27.367002 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jgjll" Jul 15 21:09:27.402469 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:27.402438 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jgjll"] Jul 15 21:09:27.498378 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:27.498351 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jgjll"] Jul 15 21:09:27.500311 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:09:27.500279 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda80c0837_3b93_47da_b0f7_5b05cdb8c631.slice/crio-c020eba25549a9769d44b07320421abc306de2fd3c26108bb6dcde026c9be3f1 WatchSource:0}: Error finding container c020eba25549a9769d44b07320421abc306de2fd3c26108bb6dcde026c9be3f1: Status 404 returned error can't find the container with id c020eba25549a9769d44b07320421abc306de2fd3c26108bb6dcde026c9be3f1 Jul 15 21:09:27.606563 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:27.606524 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kbgvd"] Jul 15 21:09:27.609233 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:27.609217 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-kbgvd" Jul 15 21:09:27.618470 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:27.618442 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kbgvd"] Jul 15 21:09:27.745378 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:27.745340 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8khv\" (UniqueName: \"kubernetes.io/projected/2152674f-5bf6-4df0-a6d7-5c3d4c6f01b9-kube-api-access-b8khv\") pod \"kuadrant-operator-catalog-kbgvd\" (UID: \"2152674f-5bf6-4df0-a6d7-5c3d4c6f01b9\") " pod="kuadrant-system/kuadrant-operator-catalog-kbgvd" Jul 15 21:09:27.831546 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:27.831457 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:09:27.846494 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:27.846460 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8khv\" (UniqueName: \"kubernetes.io/projected/2152674f-5bf6-4df0-a6d7-5c3d4c6f01b9-kube-api-access-b8khv\") pod \"kuadrant-operator-catalog-kbgvd\" (UID: \"2152674f-5bf6-4df0-a6d7-5c3d4c6f01b9\") " pod="kuadrant-system/kuadrant-operator-catalog-kbgvd" Jul 15 21:09:27.858372 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:27.858340 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8khv\" (UniqueName: \"kubernetes.io/projected/2152674f-5bf6-4df0-a6d7-5c3d4c6f01b9-kube-api-access-b8khv\") pod \"kuadrant-operator-catalog-kbgvd\" (UID: \"2152674f-5bf6-4df0-a6d7-5c3d4c6f01b9\") " pod="kuadrant-system/kuadrant-operator-catalog-kbgvd" Jul 15 21:09:27.919734 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:27.919698 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-kbgvd" Jul 15 21:09:27.931238 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:27.931210 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jgjll" event={"ID":"a80c0837-3b93-47da-b0f7-5b05cdb8c631","Type":"ContainerStarted","Data":"c020eba25549a9769d44b07320421abc306de2fd3c26108bb6dcde026c9be3f1"} Jul 15 21:09:28.050642 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:28.050609 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kbgvd"] Jul 15 21:09:28.052507 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:09:28.052477 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2152674f_5bf6_4df0_a6d7_5c3d4c6f01b9.slice/crio-7b5b453c06c4ca81ca5e4fd39eef381d6b6dcbce5520e95e695dac7f01715a89 WatchSource:0}: Error finding container 7b5b453c06c4ca81ca5e4fd39eef381d6b6dcbce5520e95e695dac7f01715a89: Status 404 returned error can't find the container with id 7b5b453c06c4ca81ca5e4fd39eef381d6b6dcbce5520e95e695dac7f01715a89 Jul 15 21:09:28.052781 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:28.052757 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:09:28.938232 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:28.938184 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-kbgvd" event={"ID":"2152674f-5bf6-4df0-a6d7-5c3d4c6f01b9","Type":"ContainerStarted","Data":"7b5b453c06c4ca81ca5e4fd39eef381d6b6dcbce5520e95e695dac7f01715a89"} Jul 15 21:09:29.949161 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:29.949035 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jgjll" event={"ID":"a80c0837-3b93-47da-b0f7-5b05cdb8c631","Type":"ContainerStarted","Data":"d0076a75e86f127131732291adbc89a50b3803dbdfb40e8a950f9082656c3d73"} Jul 15 21:09:29.949161 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:29.949076 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-jgjll" podUID="a80c0837-3b93-47da-b0f7-5b05cdb8c631" containerName="registry-server" containerID="cri-o://d0076a75e86f127131732291adbc89a50b3803dbdfb40e8a950f9082656c3d73" gracePeriod=2 Jul 15 21:09:29.951261 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:29.951233 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-kbgvd" event={"ID":"2152674f-5bf6-4df0-a6d7-5c3d4c6f01b9","Type":"ContainerStarted","Data":"b3db19b39e91f9eb0028ba3443af36e9a2d85d59d13e7b5f1a329815553948cd"} Jul 15 21:09:29.982970 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:29.982924 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-jgjll" podStartSLOduration=0.720089282 podStartE2EDuration="2.982906152s" totalImagesPullingTime="2.26281687s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:09:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:09:29.982031157 +0000 UTC m=+474.523343706" watchObservedRunningTime="2026-07-15 21:09:29.982906152 +0000 UTC m=+474.524218701" Jul 15 21:09:30.006830 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:30.006736 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-kbgvd" podStartSLOduration=1.293081428 podStartE2EDuration="3.006718304s" totalImagesPullingTime="1.713636876s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:09:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:09:30.006132871 +0000 UTC m=+474.547445420" watchObservedRunningTime="2026-07-15 21:09:30.006718304 +0000 UTC m=+474.548030910" Jul 15 21:09:30.194643 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:30.194620 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-jgjll_a80c0837-3b93-47da-b0f7-5b05cdb8c631/registry-server/0.log" Jul 15 21:09:30.196788 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:30.196773 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jgjll" Jul 15 21:09:30.371863 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:30.371825 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sxvn9\" (UniqueName: \"kubernetes.io/projected/a80c0837-3b93-47da-b0f7-5b05cdb8c631-kube-api-access-sxvn9\") pod \"a80c0837-3b93-47da-b0f7-5b05cdb8c631\" (UID: \"a80c0837-3b93-47da-b0f7-5b05cdb8c631\") " Jul 15 21:09:30.374056 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:30.374028 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a80c0837-3b93-47da-b0f7-5b05cdb8c631-kube-api-access-sxvn9" (OuterVolumeSpecName: "kube-api-access-sxvn9") pod "a80c0837-3b93-47da-b0f7-5b05cdb8c631" (UID: "a80c0837-3b93-47da-b0f7-5b05cdb8c631"). InnerVolumeSpecName "kube-api-access-sxvn9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:09:30.472605 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:30.472561 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sxvn9\" (UniqueName: \"kubernetes.io/projected/a80c0837-3b93-47da-b0f7-5b05cdb8c631-kube-api-access-sxvn9\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:09:30.956894 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:30.956868 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-jgjll_a80c0837-3b93-47da-b0f7-5b05cdb8c631/registry-server/0.log" Jul 15 21:09:30.957302 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:30.956906 2579 generic.go:356] "Generic (PLEG): container finished" podID="a80c0837-3b93-47da-b0f7-5b05cdb8c631" containerID="d0076a75e86f127131732291adbc89a50b3803dbdfb40e8a950f9082656c3d73" exitCode=1 Jul 15 21:09:30.957302 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:30.956962 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jgjll" event={"ID":"a80c0837-3b93-47da-b0f7-5b05cdb8c631","Type":"ContainerDied","Data":"d0076a75e86f127131732291adbc89a50b3803dbdfb40e8a950f9082656c3d73"} Jul 15 21:09:30.957302 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:30.956991 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jgjll" Jul 15 21:09:30.957302 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:30.957008 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jgjll" event={"ID":"a80c0837-3b93-47da-b0f7-5b05cdb8c631","Type":"ContainerDied","Data":"c020eba25549a9769d44b07320421abc306de2fd3c26108bb6dcde026c9be3f1"} Jul 15 21:09:30.957302 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:30.957027 2579 scope.go:117] "RemoveContainer" containerID="d0076a75e86f127131732291adbc89a50b3803dbdfb40e8a950f9082656c3d73" Jul 15 21:09:30.965915 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:30.965896 2579 scope.go:117] "RemoveContainer" containerID="d0076a75e86f127131732291adbc89a50b3803dbdfb40e8a950f9082656c3d73" Jul 15 21:09:30.966200 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:09:30.966181 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0076a75e86f127131732291adbc89a50b3803dbdfb40e8a950f9082656c3d73\": container with ID starting with d0076a75e86f127131732291adbc89a50b3803dbdfb40e8a950f9082656c3d73 not found: ID does not exist" containerID="d0076a75e86f127131732291adbc89a50b3803dbdfb40e8a950f9082656c3d73" Jul 15 21:09:30.966289 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:30.966213 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0076a75e86f127131732291adbc89a50b3803dbdfb40e8a950f9082656c3d73"} err="failed to get container status \"d0076a75e86f127131732291adbc89a50b3803dbdfb40e8a950f9082656c3d73\": rpc error: code = NotFound desc = could not find container \"d0076a75e86f127131732291adbc89a50b3803dbdfb40e8a950f9082656c3d73\": container with ID starting with d0076a75e86f127131732291adbc89a50b3803dbdfb40e8a950f9082656c3d73 not found: ID does not exist" Jul 15 21:09:30.983813 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:30.983784 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jgjll"] Jul 15 21:09:30.989234 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:30.989210 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jgjll"] Jul 15 21:09:31.997350 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:31.997317 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a80c0837-3b93-47da-b0f7-5b05cdb8c631" path="/var/lib/kubelet/pods/a80c0837-3b93-47da-b0f7-5b05cdb8c631/volumes" Jul 15 21:09:37.920177 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:37.920140 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-kbgvd" Jul 15 21:09:37.920581 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:37.920205 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-kbgvd" Jul 15 21:09:37.942558 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:37.942531 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-kbgvd" Jul 15 21:09:37.995741 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:37.995701 2579 generic.go:356] "Generic (PLEG): container finished" podID="b7682475-5146-4dc5-badc-1cf1def70ebe" containerID="784285cb819cdb1171d59cea2dafd5c7afbce06e3d37342d2d16d55ee399837e" exitCode=1 Jul 15 21:09:37.996109 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:37.996078 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-jkp58" event={"ID":"b7682475-5146-4dc5-badc-1cf1def70ebe","Type":"ContainerDied","Data":"784285cb819cdb1171d59cea2dafd5c7afbce06e3d37342d2d16d55ee399837e"} Jul 15 21:09:37.996401 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:37.996383 2579 scope.go:117] "RemoveContainer" containerID="784285cb819cdb1171d59cea2dafd5c7afbce06e3d37342d2d16d55ee399837e" Jul 15 21:09:38.019301 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:38.019278 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-kbgvd" Jul 15 21:09:39.002280 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:39.002244 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-jkp58" event={"ID":"b7682475-5146-4dc5-badc-1cf1def70ebe","Type":"ContainerStarted","Data":"3e226f00ee2899ea176e07c33e7859061df0d3cc042b028372c0b2dbf3215db0"} Jul 15 21:09:39.265339 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:39.265253 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zlc2"] Jul 15 21:09:39.265617 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:39.265603 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a80c0837-3b93-47da-b0f7-5b05cdb8c631" containerName="registry-server" Jul 15 21:09:39.265654 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:39.265621 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a80c0837-3b93-47da-b0f7-5b05cdb8c631" containerName="registry-server" Jul 15 21:09:39.265696 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:39.265686 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a80c0837-3b93-47da-b0f7-5b05cdb8c631" containerName="registry-server" Jul 15 21:09:39.268914 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:39.268898 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zlc2" Jul 15 21:09:39.272215 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:39.272195 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-22259\"" Jul 15 21:09:39.281266 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:39.281244 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zlc2"] Jul 15 21:09:39.338724 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:39.338687 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlvct\" (UniqueName: \"kubernetes.io/projected/563f133b-0a69-4af8-b620-c7e5dcb1f38e-kube-api-access-tlvct\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zlc2\" (UID: \"563f133b-0a69-4af8-b620-c7e5dcb1f38e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zlc2" Jul 15 21:09:39.338904 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:39.338755 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/563f133b-0a69-4af8-b620-c7e5dcb1f38e-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zlc2\" (UID: \"563f133b-0a69-4af8-b620-c7e5dcb1f38e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zlc2" Jul 15 21:09:39.338904 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:39.338838 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/563f133b-0a69-4af8-b620-c7e5dcb1f38e-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zlc2\" (UID: \"563f133b-0a69-4af8-b620-c7e5dcb1f38e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zlc2" Jul 15 21:09:39.439382 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:39.439337 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/563f133b-0a69-4af8-b620-c7e5dcb1f38e-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zlc2\" (UID: \"563f133b-0a69-4af8-b620-c7e5dcb1f38e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zlc2" Jul 15 21:09:39.439382 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:39.439388 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/563f133b-0a69-4af8-b620-c7e5dcb1f38e-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zlc2\" (UID: \"563f133b-0a69-4af8-b620-c7e5dcb1f38e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zlc2" Jul 15 21:09:39.439635 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:39.439520 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlvct\" (UniqueName: \"kubernetes.io/projected/563f133b-0a69-4af8-b620-c7e5dcb1f38e-kube-api-access-tlvct\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zlc2\" (UID: \"563f133b-0a69-4af8-b620-c7e5dcb1f38e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zlc2" Jul 15 21:09:39.439747 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:39.439725 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/563f133b-0a69-4af8-b620-c7e5dcb1f38e-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zlc2\" (UID: \"563f133b-0a69-4af8-b620-c7e5dcb1f38e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zlc2" Jul 15 21:09:39.439803 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:39.439773 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/563f133b-0a69-4af8-b620-c7e5dcb1f38e-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zlc2\" (UID: \"563f133b-0a69-4af8-b620-c7e5dcb1f38e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zlc2" Jul 15 21:09:39.450340 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:39.450311 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlvct\" (UniqueName: \"kubernetes.io/projected/563f133b-0a69-4af8-b620-c7e5dcb1f38e-kube-api-access-tlvct\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zlc2\" (UID: \"563f133b-0a69-4af8-b620-c7e5dcb1f38e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zlc2" Jul 15 21:09:39.577681 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:39.577591 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zlc2" Jul 15 21:09:39.709441 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:39.709412 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zlc2"] Jul 15 21:09:39.710992 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:09:39.710962 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod563f133b_0a69_4af8_b620_c7e5dcb1f38e.slice/crio-a7f2677088bcc9d7c19ef0a855867e47cc965e80bf23f71f6e284a99ec6d7edf WatchSource:0}: Error finding container a7f2677088bcc9d7c19ef0a855867e47cc965e80bf23f71f6e284a99ec6d7edf: Status 404 returned error can't find the container with id a7f2677088bcc9d7c19ef0a855867e47cc965e80bf23f71f6e284a99ec6d7edf Jul 15 21:09:39.866868 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:39.866834 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tmgpz"] Jul 15 21:09:39.870513 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:39.870488 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tmgpz" Jul 15 21:09:39.883451 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:39.883429 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tmgpz"] Jul 15 21:09:39.942655 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:39.942617 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnvhr\" (UniqueName: \"kubernetes.io/projected/a5f59b43-6fed-4fa8-b782-ed5457723590-kube-api-access-fnvhr\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tmgpz\" (UID: \"a5f59b43-6fed-4fa8-b782-ed5457723590\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tmgpz" Jul 15 21:09:39.942839 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:39.942660 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a5f59b43-6fed-4fa8-b782-ed5457723590-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tmgpz\" (UID: \"a5f59b43-6fed-4fa8-b782-ed5457723590\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tmgpz" Jul 15 21:09:39.942839 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:39.942695 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a5f59b43-6fed-4fa8-b782-ed5457723590-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tmgpz\" (UID: \"a5f59b43-6fed-4fa8-b782-ed5457723590\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tmgpz" Jul 15 21:09:40.007564 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:40.007528 2579 generic.go:356] "Generic (PLEG): container finished" podID="563f133b-0a69-4af8-b620-c7e5dcb1f38e" containerID="a4e994fd837c7df819b0bfea107590b39656e0990d98d347c09194d5ea5acbd8" exitCode=0 Jul 15 21:09:40.007932 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:40.007606 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zlc2" event={"ID":"563f133b-0a69-4af8-b620-c7e5dcb1f38e","Type":"ContainerDied","Data":"a4e994fd837c7df819b0bfea107590b39656e0990d98d347c09194d5ea5acbd8"} Jul 15 21:09:40.007932 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:40.007643 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zlc2" event={"ID":"563f133b-0a69-4af8-b620-c7e5dcb1f38e","Type":"ContainerStarted","Data":"a7f2677088bcc9d7c19ef0a855867e47cc965e80bf23f71f6e284a99ec6d7edf"} Jul 15 21:09:40.044006 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:40.043966 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a5f59b43-6fed-4fa8-b782-ed5457723590-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tmgpz\" (UID: \"a5f59b43-6fed-4fa8-b782-ed5457723590\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tmgpz" Jul 15 21:09:40.044213 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:40.044041 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnvhr\" (UniqueName: \"kubernetes.io/projected/a5f59b43-6fed-4fa8-b782-ed5457723590-kube-api-access-fnvhr\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tmgpz\" (UID: \"a5f59b43-6fed-4fa8-b782-ed5457723590\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tmgpz" Jul 15 21:09:40.044213 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:40.044119 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a5f59b43-6fed-4fa8-b782-ed5457723590-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tmgpz\" (UID: \"a5f59b43-6fed-4fa8-b782-ed5457723590\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tmgpz" Jul 15 21:09:40.044408 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:40.044391 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a5f59b43-6fed-4fa8-b782-ed5457723590-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tmgpz\" (UID: \"a5f59b43-6fed-4fa8-b782-ed5457723590\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tmgpz" Jul 15 21:09:40.044567 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:40.044545 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a5f59b43-6fed-4fa8-b782-ed5457723590-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tmgpz\" (UID: \"a5f59b43-6fed-4fa8-b782-ed5457723590\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tmgpz" Jul 15 21:09:40.053531 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:40.053504 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnvhr\" (UniqueName: \"kubernetes.io/projected/a5f59b43-6fed-4fa8-b782-ed5457723590-kube-api-access-fnvhr\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tmgpz\" (UID: \"a5f59b43-6fed-4fa8-b782-ed5457723590\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tmgpz" Jul 15 21:09:40.185858 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:40.185766 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tmgpz" Jul 15 21:09:40.315418 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:40.315393 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tmgpz"] Jul 15 21:09:40.317196 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:09:40.317166 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5f59b43_6fed_4fa8_b782_ed5457723590.slice/crio-6401083f6c39bb3ccd600f265098d5268407e16262819118ee46876075ef19ea WatchSource:0}: Error finding container 6401083f6c39bb3ccd600f265098d5268407e16262819118ee46876075ef19ea: Status 404 returned error can't find the container with id 6401083f6c39bb3ccd600f265098d5268407e16262819118ee46876075ef19ea Jul 15 21:09:40.479958 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:40.479929 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597plq7"] Jul 15 21:09:40.483213 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:40.483195 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597plq7" Jul 15 21:09:40.495266 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:40.495246 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597plq7"] Jul 15 21:09:40.548415 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:40.548381 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cf45cf6f-0a81-470a-b31d-26d7e6591639-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597plq7\" (UID: \"cf45cf6f-0a81-470a-b31d-26d7e6591639\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597plq7" Jul 15 21:09:40.548589 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:40.548427 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cf45cf6f-0a81-470a-b31d-26d7e6591639-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597plq7\" (UID: \"cf45cf6f-0a81-470a-b31d-26d7e6591639\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597plq7" Jul 15 21:09:40.548589 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:40.548512 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cd9qc\" (UniqueName: \"kubernetes.io/projected/cf45cf6f-0a81-470a-b31d-26d7e6591639-kube-api-access-cd9qc\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597plq7\" (UID: \"cf45cf6f-0a81-470a-b31d-26d7e6591639\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597plq7" Jul 15 21:09:40.649694 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:40.649646 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cf45cf6f-0a81-470a-b31d-26d7e6591639-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597plq7\" (UID: \"cf45cf6f-0a81-470a-b31d-26d7e6591639\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597plq7" Jul 15 21:09:40.649830 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:40.649750 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cd9qc\" (UniqueName: \"kubernetes.io/projected/cf45cf6f-0a81-470a-b31d-26d7e6591639-kube-api-access-cd9qc\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597plq7\" (UID: \"cf45cf6f-0a81-470a-b31d-26d7e6591639\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597plq7" Jul 15 21:09:40.649931 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:40.649885 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cf45cf6f-0a81-470a-b31d-26d7e6591639-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597plq7\" (UID: \"cf45cf6f-0a81-470a-b31d-26d7e6591639\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597plq7" Jul 15 21:09:40.650023 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:40.649996 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cf45cf6f-0a81-470a-b31d-26d7e6591639-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597plq7\" (UID: \"cf45cf6f-0a81-470a-b31d-26d7e6591639\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597plq7" Jul 15 21:09:40.650172 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:40.650157 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cf45cf6f-0a81-470a-b31d-26d7e6591639-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597plq7\" (UID: \"cf45cf6f-0a81-470a-b31d-26d7e6591639\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597plq7" Jul 15 21:09:40.660957 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:40.660921 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cd9qc\" (UniqueName: \"kubernetes.io/projected/cf45cf6f-0a81-470a-b31d-26d7e6591639-kube-api-access-cd9qc\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597plq7\" (UID: \"cf45cf6f-0a81-470a-b31d-26d7e6591639\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597plq7" Jul 15 21:09:40.791307 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:40.791285 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597plq7" Jul 15 21:09:40.947188 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:40.947161 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597plq7"] Jul 15 21:09:40.949552 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:09:40.949526 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf45cf6f_0a81_470a_b31d_26d7e6591639.slice/crio-6de359572ba3e42f9efb9adba98843f7462f715a865e61964e268f80fcb15055 WatchSource:0}: Error finding container 6de359572ba3e42f9efb9adba98843f7462f715a865e61964e268f80fcb15055: Status 404 returned error can't find the container with id 6de359572ba3e42f9efb9adba98843f7462f715a865e61964e268f80fcb15055 Jul 15 21:09:41.015228 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:41.015197 2579 generic.go:356] "Generic (PLEG): container finished" podID="a5f59b43-6fed-4fa8-b782-ed5457723590" containerID="7ba12d79c67eda2458b486c598078d0af340ab44d0244300d163baefc9f6dfb2" exitCode=0 Jul 15 21:09:41.015613 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:41.015292 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tmgpz" event={"ID":"a5f59b43-6fed-4fa8-b782-ed5457723590","Type":"ContainerDied","Data":"7ba12d79c67eda2458b486c598078d0af340ab44d0244300d163baefc9f6dfb2"} Jul 15 21:09:41.015613 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:41.015359 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tmgpz" event={"ID":"a5f59b43-6fed-4fa8-b782-ed5457723590","Type":"ContainerStarted","Data":"6401083f6c39bb3ccd600f265098d5268407e16262819118ee46876075ef19ea"} Jul 15 21:09:41.017187 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:41.017160 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597plq7" event={"ID":"cf45cf6f-0a81-470a-b31d-26d7e6591639","Type":"ContainerStarted","Data":"62f3bf964ee1e7f701e113a3a2e2d5aee42905eaba5971269916863b2b765b4c"} Jul 15 21:09:41.017302 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:41.017210 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597plq7" event={"ID":"cf45cf6f-0a81-470a-b31d-26d7e6591639","Type":"ContainerStarted","Data":"6de359572ba3e42f9efb9adba98843f7462f715a865e61964e268f80fcb15055"} Jul 15 21:09:41.019251 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:41.019227 2579 generic.go:356] "Generic (PLEG): container finished" podID="563f133b-0a69-4af8-b620-c7e5dcb1f38e" containerID="9049450ffe57458cb5799635b879ced0cdd32c5fad4a3da91adb622e28fc62eb" exitCode=0 Jul 15 21:09:41.019360 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:41.019286 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zlc2" event={"ID":"563f133b-0a69-4af8-b620-c7e5dcb1f38e","Type":"ContainerDied","Data":"9049450ffe57458cb5799635b879ced0cdd32c5fad4a3da91adb622e28fc62eb"} Jul 15 21:09:41.060428 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:41.060397 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dp99w"] Jul 15 21:09:41.064117 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:41.064093 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dp99w" Jul 15 21:09:41.076748 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:41.075273 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dp99w"] Jul 15 21:09:41.155124 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:41.155083 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a720aa3c-7268-4fbf-8fc9-ef8739108fdf-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dp99w\" (UID: \"a720aa3c-7268-4fbf-8fc9-ef8739108fdf\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dp99w" Jul 15 21:09:41.155124 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:41.155130 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a720aa3c-7268-4fbf-8fc9-ef8739108fdf-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dp99w\" (UID: \"a720aa3c-7268-4fbf-8fc9-ef8739108fdf\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dp99w" Jul 15 21:09:41.155363 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:41.155236 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncrr8\" (UniqueName: \"kubernetes.io/projected/a720aa3c-7268-4fbf-8fc9-ef8739108fdf-kube-api-access-ncrr8\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dp99w\" (UID: \"a720aa3c-7268-4fbf-8fc9-ef8739108fdf\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dp99w" Jul 15 21:09:41.256189 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:41.256088 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncrr8\" (UniqueName: \"kubernetes.io/projected/a720aa3c-7268-4fbf-8fc9-ef8739108fdf-kube-api-access-ncrr8\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dp99w\" (UID: \"a720aa3c-7268-4fbf-8fc9-ef8739108fdf\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dp99w" Jul 15 21:09:41.256189 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:41.256142 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a720aa3c-7268-4fbf-8fc9-ef8739108fdf-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dp99w\" (UID: \"a720aa3c-7268-4fbf-8fc9-ef8739108fdf\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dp99w" Jul 15 21:09:41.256386 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:41.256264 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a720aa3c-7268-4fbf-8fc9-ef8739108fdf-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dp99w\" (UID: \"a720aa3c-7268-4fbf-8fc9-ef8739108fdf\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dp99w" Jul 15 21:09:41.256514 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:41.256494 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a720aa3c-7268-4fbf-8fc9-ef8739108fdf-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dp99w\" (UID: \"a720aa3c-7268-4fbf-8fc9-ef8739108fdf\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dp99w" Jul 15 21:09:41.256644 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:41.256625 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a720aa3c-7268-4fbf-8fc9-ef8739108fdf-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dp99w\" (UID: \"a720aa3c-7268-4fbf-8fc9-ef8739108fdf\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dp99w" Jul 15 21:09:41.265762 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:41.265737 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncrr8\" (UniqueName: \"kubernetes.io/projected/a720aa3c-7268-4fbf-8fc9-ef8739108fdf-kube-api-access-ncrr8\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dp99w\" (UID: \"a720aa3c-7268-4fbf-8fc9-ef8739108fdf\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dp99w" Jul 15 21:09:41.422128 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:41.422058 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dp99w" Jul 15 21:09:41.558539 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:41.558506 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dp99w"] Jul 15 21:09:41.559510 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:09:41.559483 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda720aa3c_7268_4fbf_8fc9_ef8739108fdf.slice/crio-0ccffa1f71345a930ac942c96546e6d590220654ef3d485da63a8eee66687161 WatchSource:0}: Error finding container 0ccffa1f71345a930ac942c96546e6d590220654ef3d485da63a8eee66687161: Status 404 returned error can't find the container with id 0ccffa1f71345a930ac942c96546e6d590220654ef3d485da63a8eee66687161 Jul 15 21:09:42.025944 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:42.025913 2579 generic.go:356] "Generic (PLEG): container finished" podID="a5f59b43-6fed-4fa8-b782-ed5457723590" containerID="321c8119b1d5817cb3d39604c535d4a519c067d75a1fd7820438f530e031e743" exitCode=0 Jul 15 21:09:42.026424 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:42.026002 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tmgpz" event={"ID":"a5f59b43-6fed-4fa8-b782-ed5457723590","Type":"ContainerDied","Data":"321c8119b1d5817cb3d39604c535d4a519c067d75a1fd7820438f530e031e743"} Jul 15 21:09:42.027626 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:42.027612 2579 generic.go:356] "Generic (PLEG): container finished" podID="cf45cf6f-0a81-470a-b31d-26d7e6591639" containerID="62f3bf964ee1e7f701e113a3a2e2d5aee42905eaba5971269916863b2b765b4c" exitCode=0 Jul 15 21:09:42.027710 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:42.027690 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597plq7" event={"ID":"cf45cf6f-0a81-470a-b31d-26d7e6591639","Type":"ContainerDied","Data":"62f3bf964ee1e7f701e113a3a2e2d5aee42905eaba5971269916863b2b765b4c"} Jul 15 21:09:42.029430 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:42.029406 2579 generic.go:356] "Generic (PLEG): container finished" podID="a720aa3c-7268-4fbf-8fc9-ef8739108fdf" containerID="d40645a031e4a14e6f96b6e5fb2673edf09b54f1140cd5cd57afb7a1fb92b35b" exitCode=0 Jul 15 21:09:42.029520 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:42.029493 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dp99w" event={"ID":"a720aa3c-7268-4fbf-8fc9-ef8739108fdf","Type":"ContainerDied","Data":"d40645a031e4a14e6f96b6e5fb2673edf09b54f1140cd5cd57afb7a1fb92b35b"} Jul 15 21:09:42.029594 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:42.029559 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dp99w" event={"ID":"a720aa3c-7268-4fbf-8fc9-ef8739108fdf","Type":"ContainerStarted","Data":"0ccffa1f71345a930ac942c96546e6d590220654ef3d485da63a8eee66687161"} Jul 15 21:09:42.031770 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:42.031747 2579 generic.go:356] "Generic (PLEG): container finished" podID="563f133b-0a69-4af8-b620-c7e5dcb1f38e" containerID="f6741ba6fb8e5eb08e1f775bdc3ed1638478b0b625847c4f9d8e50107deb4c79" exitCode=0 Jul 15 21:09:42.031868 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:42.031806 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zlc2" event={"ID":"563f133b-0a69-4af8-b620-c7e5dcb1f38e","Type":"ContainerDied","Data":"f6741ba6fb8e5eb08e1f775bdc3ed1638478b0b625847c4f9d8e50107deb4c79"} Jul 15 21:09:43.038874 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:43.038836 2579 generic.go:356] "Generic (PLEG): container finished" podID="a5f59b43-6fed-4fa8-b782-ed5457723590" containerID="0ccb4e0966d5ba4f457b0e84c7bbfa19c015cac72cd046d7525b45ea692d8e24" exitCode=0 Jul 15 21:09:43.038874 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:43.038864 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tmgpz" event={"ID":"a5f59b43-6fed-4fa8-b782-ed5457723590","Type":"ContainerDied","Data":"0ccb4e0966d5ba4f457b0e84c7bbfa19c015cac72cd046d7525b45ea692d8e24"} Jul 15 21:09:43.040537 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:43.040516 2579 generic.go:356] "Generic (PLEG): container finished" podID="cf45cf6f-0a81-470a-b31d-26d7e6591639" containerID="f3340369ec0294a597f8a1d7fee06272afa6e5a31af9fd314dfe992b40559b1e" exitCode=0 Jul 15 21:09:43.040617 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:43.040591 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597plq7" event={"ID":"cf45cf6f-0a81-470a-b31d-26d7e6591639","Type":"ContainerDied","Data":"f3340369ec0294a597f8a1d7fee06272afa6e5a31af9fd314dfe992b40559b1e"} Jul 15 21:09:43.042452 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:43.042434 2579 generic.go:356] "Generic (PLEG): container finished" podID="a720aa3c-7268-4fbf-8fc9-ef8739108fdf" containerID="8afa58f3b9c84fe224c289a126b780133be93059d4e342201cd3dbba509370f5" exitCode=0 Jul 15 21:09:43.042530 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:43.042521 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dp99w" event={"ID":"a720aa3c-7268-4fbf-8fc9-ef8739108fdf","Type":"ContainerDied","Data":"8afa58f3b9c84fe224c289a126b780133be93059d4e342201cd3dbba509370f5"} Jul 15 21:09:43.182311 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:43.182280 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zlc2" Jul 15 21:09:43.272046 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:43.272010 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/563f133b-0a69-4af8-b620-c7e5dcb1f38e-util\") pod \"563f133b-0a69-4af8-b620-c7e5dcb1f38e\" (UID: \"563f133b-0a69-4af8-b620-c7e5dcb1f38e\") " Jul 15 21:09:43.272220 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:43.272087 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tlvct\" (UniqueName: \"kubernetes.io/projected/563f133b-0a69-4af8-b620-c7e5dcb1f38e-kube-api-access-tlvct\") pod \"563f133b-0a69-4af8-b620-c7e5dcb1f38e\" (UID: \"563f133b-0a69-4af8-b620-c7e5dcb1f38e\") " Jul 15 21:09:43.272220 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:43.272111 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/563f133b-0a69-4af8-b620-c7e5dcb1f38e-bundle\") pod \"563f133b-0a69-4af8-b620-c7e5dcb1f38e\" (UID: \"563f133b-0a69-4af8-b620-c7e5dcb1f38e\") " Jul 15 21:09:43.272721 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:43.272693 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/563f133b-0a69-4af8-b620-c7e5dcb1f38e-bundle" (OuterVolumeSpecName: "bundle") pod "563f133b-0a69-4af8-b620-c7e5dcb1f38e" (UID: "563f133b-0a69-4af8-b620-c7e5dcb1f38e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:09:43.274180 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:43.274151 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/563f133b-0a69-4af8-b620-c7e5dcb1f38e-kube-api-access-tlvct" (OuterVolumeSpecName: "kube-api-access-tlvct") pod "563f133b-0a69-4af8-b620-c7e5dcb1f38e" (UID: "563f133b-0a69-4af8-b620-c7e5dcb1f38e"). InnerVolumeSpecName "kube-api-access-tlvct". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:09:43.277586 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:43.277561 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/563f133b-0a69-4af8-b620-c7e5dcb1f38e-util" (OuterVolumeSpecName: "util") pod "563f133b-0a69-4af8-b620-c7e5dcb1f38e" (UID: "563f133b-0a69-4af8-b620-c7e5dcb1f38e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:09:43.373285 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:43.373196 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tlvct\" (UniqueName: \"kubernetes.io/projected/563f133b-0a69-4af8-b620-c7e5dcb1f38e-kube-api-access-tlvct\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:09:43.373285 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:43.373228 2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/563f133b-0a69-4af8-b620-c7e5dcb1f38e-bundle\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:09:43.373285 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:43.373237 2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/563f133b-0a69-4af8-b620-c7e5dcb1f38e-util\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:09:44.049350 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:44.049314 2579 generic.go:356] "Generic (PLEG): container finished" podID="a720aa3c-7268-4fbf-8fc9-ef8739108fdf" containerID="6bbdc2e78e9351e4c73a9857e1e5fa203786ac4c41425dbafcf59e659a1c3cf5" exitCode=0 Jul 15 21:09:44.049712 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:44.049398 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dp99w" event={"ID":"a720aa3c-7268-4fbf-8fc9-ef8739108fdf","Type":"ContainerDied","Data":"6bbdc2e78e9351e4c73a9857e1e5fa203786ac4c41425dbafcf59e659a1c3cf5"} Jul 15 21:09:44.051221 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:44.051201 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zlc2" Jul 15 21:09:44.051377 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:44.051217 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zlc2" event={"ID":"563f133b-0a69-4af8-b620-c7e5dcb1f38e","Type":"ContainerDied","Data":"a7f2677088bcc9d7c19ef0a855867e47cc965e80bf23f71f6e284a99ec6d7edf"} Jul 15 21:09:44.051377 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:44.051261 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a7f2677088bcc9d7c19ef0a855867e47cc965e80bf23f71f6e284a99ec6d7edf" Jul 15 21:09:44.053535 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:44.053514 2579 generic.go:356] "Generic (PLEG): container finished" podID="cf45cf6f-0a81-470a-b31d-26d7e6591639" containerID="b24d84526ce9fdef885b10f3ac33a416a38c68d5410c2573d5fe6b759d284328" exitCode=0 Jul 15 21:09:44.053651 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:44.053582 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597plq7" event={"ID":"cf45cf6f-0a81-470a-b31d-26d7e6591639","Type":"ContainerDied","Data":"b24d84526ce9fdef885b10f3ac33a416a38c68d5410c2573d5fe6b759d284328"} Jul 15 21:09:44.188874 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:44.188850 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tmgpz" Jul 15 21:09:44.281169 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:44.281085 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fnvhr\" (UniqueName: \"kubernetes.io/projected/a5f59b43-6fed-4fa8-b782-ed5457723590-kube-api-access-fnvhr\") pod \"a5f59b43-6fed-4fa8-b782-ed5457723590\" (UID: \"a5f59b43-6fed-4fa8-b782-ed5457723590\") " Jul 15 21:09:44.281169 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:44.281122 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a5f59b43-6fed-4fa8-b782-ed5457723590-bundle\") pod \"a5f59b43-6fed-4fa8-b782-ed5457723590\" (UID: \"a5f59b43-6fed-4fa8-b782-ed5457723590\") " Jul 15 21:09:44.281169 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:44.281143 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a5f59b43-6fed-4fa8-b782-ed5457723590-util\") pod \"a5f59b43-6fed-4fa8-b782-ed5457723590\" (UID: \"a5f59b43-6fed-4fa8-b782-ed5457723590\") " Jul 15 21:09:44.281597 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:44.281574 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5f59b43-6fed-4fa8-b782-ed5457723590-bundle" (OuterVolumeSpecName: "bundle") pod "a5f59b43-6fed-4fa8-b782-ed5457723590" (UID: "a5f59b43-6fed-4fa8-b782-ed5457723590"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:09:44.283311 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:44.283287 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5f59b43-6fed-4fa8-b782-ed5457723590-kube-api-access-fnvhr" (OuterVolumeSpecName: "kube-api-access-fnvhr") pod "a5f59b43-6fed-4fa8-b782-ed5457723590" (UID: "a5f59b43-6fed-4fa8-b782-ed5457723590"). InnerVolumeSpecName "kube-api-access-fnvhr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:09:44.286532 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:44.286513 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5f59b43-6fed-4fa8-b782-ed5457723590-util" (OuterVolumeSpecName: "util") pod "a5f59b43-6fed-4fa8-b782-ed5457723590" (UID: "a5f59b43-6fed-4fa8-b782-ed5457723590"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:09:44.382532 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:44.382497 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fnvhr\" (UniqueName: \"kubernetes.io/projected/a5f59b43-6fed-4fa8-b782-ed5457723590-kube-api-access-fnvhr\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:09:44.382532 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:44.382529 2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a5f59b43-6fed-4fa8-b782-ed5457723590-bundle\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:09:44.382532 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:44.382538 2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a5f59b43-6fed-4fa8-b782-ed5457723590-util\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:09:45.059729 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:45.059700 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tmgpz" Jul 15 21:09:45.060151 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:45.059734 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tmgpz" event={"ID":"a5f59b43-6fed-4fa8-b782-ed5457723590","Type":"ContainerDied","Data":"6401083f6c39bb3ccd600f265098d5268407e16262819118ee46876075ef19ea"} Jul 15 21:09:45.060151 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:45.059791 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6401083f6c39bb3ccd600f265098d5268407e16262819118ee46876075ef19ea" Jul 15 21:09:45.211120 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:45.211091 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597plq7" Jul 15 21:09:45.213520 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:45.213500 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dp99w" Jul 15 21:09:45.289193 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:45.289159 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ncrr8\" (UniqueName: \"kubernetes.io/projected/a720aa3c-7268-4fbf-8fc9-ef8739108fdf-kube-api-access-ncrr8\") pod \"a720aa3c-7268-4fbf-8fc9-ef8739108fdf\" (UID: \"a720aa3c-7268-4fbf-8fc9-ef8739108fdf\") " Jul 15 21:09:45.289361 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:45.289229 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cf45cf6f-0a81-470a-b31d-26d7e6591639-bundle\") pod \"cf45cf6f-0a81-470a-b31d-26d7e6591639\" (UID: \"cf45cf6f-0a81-470a-b31d-26d7e6591639\") " Jul 15 21:09:45.289361 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:45.289276 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a720aa3c-7268-4fbf-8fc9-ef8739108fdf-bundle\") pod \"a720aa3c-7268-4fbf-8fc9-ef8739108fdf\" (UID: \"a720aa3c-7268-4fbf-8fc9-ef8739108fdf\") " Jul 15 21:09:45.289361 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:45.289317 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cd9qc\" (UniqueName: \"kubernetes.io/projected/cf45cf6f-0a81-470a-b31d-26d7e6591639-kube-api-access-cd9qc\") pod \"cf45cf6f-0a81-470a-b31d-26d7e6591639\" (UID: \"cf45cf6f-0a81-470a-b31d-26d7e6591639\") " Jul 15 21:09:45.289530 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:45.289365 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cf45cf6f-0a81-470a-b31d-26d7e6591639-util\") pod \"cf45cf6f-0a81-470a-b31d-26d7e6591639\" (UID: \"cf45cf6f-0a81-470a-b31d-26d7e6591639\") " Jul 15 21:09:45.289530 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:45.289384 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a720aa3c-7268-4fbf-8fc9-ef8739108fdf-util\") pod \"a720aa3c-7268-4fbf-8fc9-ef8739108fdf\" (UID: \"a720aa3c-7268-4fbf-8fc9-ef8739108fdf\") " Jul 15 21:09:45.289876 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:45.289839 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf45cf6f-0a81-470a-b31d-26d7e6591639-bundle" (OuterVolumeSpecName: "bundle") pod "cf45cf6f-0a81-470a-b31d-26d7e6591639" (UID: "cf45cf6f-0a81-470a-b31d-26d7e6591639"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:09:45.290132 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:45.290104 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a720aa3c-7268-4fbf-8fc9-ef8739108fdf-bundle" (OuterVolumeSpecName: "bundle") pod "a720aa3c-7268-4fbf-8fc9-ef8739108fdf" (UID: "a720aa3c-7268-4fbf-8fc9-ef8739108fdf"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:09:45.291511 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:45.291483 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf45cf6f-0a81-470a-b31d-26d7e6591639-kube-api-access-cd9qc" (OuterVolumeSpecName: "kube-api-access-cd9qc") pod "cf45cf6f-0a81-470a-b31d-26d7e6591639" (UID: "cf45cf6f-0a81-470a-b31d-26d7e6591639"). InnerVolumeSpecName "kube-api-access-cd9qc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:09:45.291612 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:45.291507 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a720aa3c-7268-4fbf-8fc9-ef8739108fdf-kube-api-access-ncrr8" (OuterVolumeSpecName: "kube-api-access-ncrr8") pod "a720aa3c-7268-4fbf-8fc9-ef8739108fdf" (UID: "a720aa3c-7268-4fbf-8fc9-ef8739108fdf"). InnerVolumeSpecName "kube-api-access-ncrr8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:09:45.295137 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:45.295113 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf45cf6f-0a81-470a-b31d-26d7e6591639-util" (OuterVolumeSpecName: "util") pod "cf45cf6f-0a81-470a-b31d-26d7e6591639" (UID: "cf45cf6f-0a81-470a-b31d-26d7e6591639"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:09:45.298258 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:45.298235 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a720aa3c-7268-4fbf-8fc9-ef8739108fdf-util" (OuterVolumeSpecName: "util") pod "a720aa3c-7268-4fbf-8fc9-ef8739108fdf" (UID: "a720aa3c-7268-4fbf-8fc9-ef8739108fdf"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:09:45.390070 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:45.390039 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cd9qc\" (UniqueName: \"kubernetes.io/projected/cf45cf6f-0a81-470a-b31d-26d7e6591639-kube-api-access-cd9qc\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:09:45.390201 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:45.390096 2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cf45cf6f-0a81-470a-b31d-26d7e6591639-util\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:09:45.390201 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:45.390109 2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a720aa3c-7268-4fbf-8fc9-ef8739108fdf-util\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:09:45.390201 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:45.390118 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ncrr8\" (UniqueName: \"kubernetes.io/projected/a720aa3c-7268-4fbf-8fc9-ef8739108fdf-kube-api-access-ncrr8\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:09:45.390201 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:45.390127 2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cf45cf6f-0a81-470a-b31d-26d7e6591639-bundle\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:09:45.390201 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:45.390135 2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a720aa3c-7268-4fbf-8fc9-ef8739108fdf-bundle\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:09:46.068517 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:46.068482 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dp99w" Jul 15 21:09:46.068973 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:46.068542 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dp99w" event={"ID":"a720aa3c-7268-4fbf-8fc9-ef8739108fdf","Type":"ContainerDied","Data":"0ccffa1f71345a930ac942c96546e6d590220654ef3d485da63a8eee66687161"} Jul 15 21:09:46.068973 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:46.068602 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0ccffa1f71345a930ac942c96546e6d590220654ef3d485da63a8eee66687161" Jul 15 21:09:46.070903 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:46.070873 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597plq7" event={"ID":"cf45cf6f-0a81-470a-b31d-26d7e6591639","Type":"ContainerDied","Data":"6de359572ba3e42f9efb9adba98843f7462f715a865e61964e268f80fcb15055"} Jul 15 21:09:46.071029 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:46.070927 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597plq7" Jul 15 21:09:46.071029 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:46.070943 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6de359572ba3e42f9efb9adba98843f7462f715a865e61964e268f80fcb15055" Jul 15 21:09:48.070430 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:48.070398 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:09:48.901512 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:48.901472 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:09:52.251952 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:52.251914 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-7bzzs"] Jul 15 21:09:52.252334 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:52.252229 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5f59b43-6fed-4fa8-b782-ed5457723590" containerName="pull" Jul 15 21:09:52.252334 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:52.252240 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5f59b43-6fed-4fa8-b782-ed5457723590" containerName="pull" Jul 15 21:09:52.252334 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:52.252248 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a720aa3c-7268-4fbf-8fc9-ef8739108fdf" containerName="pull" Jul 15 21:09:52.252334 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:52.252254 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a720aa3c-7268-4fbf-8fc9-ef8739108fdf" containerName="pull" Jul 15 21:09:52.252334 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:52.252263 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5f59b43-6fed-4fa8-b782-ed5457723590" containerName="util" Jul 15 21:09:52.252334 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:52.252268 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5f59b43-6fed-4fa8-b782-ed5457723590" containerName="util" Jul 15 21:09:52.252334 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:52.252274 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5f59b43-6fed-4fa8-b782-ed5457723590" containerName="extract" Jul 15 21:09:52.252334 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:52.252279 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5f59b43-6fed-4fa8-b782-ed5457723590" containerName="extract" Jul 15 21:09:52.252334 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:52.252285 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf45cf6f-0a81-470a-b31d-26d7e6591639" containerName="pull" Jul 15 21:09:52.252334 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:52.252290 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf45cf6f-0a81-470a-b31d-26d7e6591639" containerName="pull" Jul 15 21:09:52.252334 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:52.252301 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf45cf6f-0a81-470a-b31d-26d7e6591639" containerName="util" Jul 15 21:09:52.252334 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:52.252306 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf45cf6f-0a81-470a-b31d-26d7e6591639" containerName="util" Jul 15 21:09:52.252334 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:52.252313 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a720aa3c-7268-4fbf-8fc9-ef8739108fdf" containerName="util" Jul 15 21:09:52.252334 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:52.252318 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a720aa3c-7268-4fbf-8fc9-ef8739108fdf" containerName="util" Jul 15 21:09:52.252334 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:52.252325 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="563f133b-0a69-4af8-b620-c7e5dcb1f38e" containerName="pull" Jul 15 21:09:52.252334 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:52.252330 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="563f133b-0a69-4af8-b620-c7e5dcb1f38e" containerName="pull" Jul 15 21:09:52.252334 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:52.252336 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="563f133b-0a69-4af8-b620-c7e5dcb1f38e" containerName="extract" Jul 15 21:09:52.252334 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:52.252342 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="563f133b-0a69-4af8-b620-c7e5dcb1f38e" containerName="extract" Jul 15 21:09:52.252831 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:52.252349 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf45cf6f-0a81-470a-b31d-26d7e6591639" containerName="extract" Jul 15 21:09:52.252831 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:52.252354 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf45cf6f-0a81-470a-b31d-26d7e6591639" containerName="extract" Jul 15 21:09:52.252831 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:52.252361 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a720aa3c-7268-4fbf-8fc9-ef8739108fdf" containerName="extract" Jul 15 21:09:52.252831 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:52.252366 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a720aa3c-7268-4fbf-8fc9-ef8739108fdf" containerName="extract" Jul 15 21:09:52.252831 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:52.252373 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="563f133b-0a69-4af8-b620-c7e5dcb1f38e" containerName="util" Jul 15 21:09:52.252831 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:52.252378 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="563f133b-0a69-4af8-b620-c7e5dcb1f38e" containerName="util" Jul 15 21:09:52.252831 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:52.252440 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf45cf6f-0a81-470a-b31d-26d7e6591639" containerName="extract" Jul 15 21:09:52.252831 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:52.252447 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a720aa3c-7268-4fbf-8fc9-ef8739108fdf" containerName="extract" Jul 15 21:09:52.252831 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:52.252454 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5f59b43-6fed-4fa8-b782-ed5457723590" containerName="extract" Jul 15 21:09:52.252831 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:52.252460 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="563f133b-0a69-4af8-b620-c7e5dcb1f38e" containerName="extract" Jul 15 21:09:52.256394 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:52.256376 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7bzzs" Jul 15 21:09:52.259875 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:52.259855 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-zx6mx\"" Jul 15 21:09:52.268577 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:52.268552 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-7bzzs"] Jul 15 21:09:52.350915 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:52.350863 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5s4zs\" (UniqueName: \"kubernetes.io/projected/cd1829fe-534c-44fc-b6cf-5761acfa46a0-kube-api-access-5s4zs\") pod \"authorino-operator-75cc8d5cb7-7bzzs\" (UID: \"cd1829fe-534c-44fc-b6cf-5761acfa46a0\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-7bzzs" Jul 15 21:09:52.451858 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:52.451808 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5s4zs\" (UniqueName: \"kubernetes.io/projected/cd1829fe-534c-44fc-b6cf-5761acfa46a0-kube-api-access-5s4zs\") pod \"authorino-operator-75cc8d5cb7-7bzzs\" (UID: \"cd1829fe-534c-44fc-b6cf-5761acfa46a0\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-7bzzs" Jul 15 21:09:52.481256 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:52.481224 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5s4zs\" (UniqueName: \"kubernetes.io/projected/cd1829fe-534c-44fc-b6cf-5761acfa46a0-kube-api-access-5s4zs\") pod \"authorino-operator-75cc8d5cb7-7bzzs\" (UID: \"cd1829fe-534c-44fc-b6cf-5761acfa46a0\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-7bzzs" Jul 15 21:09:52.566031 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:52.565946 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7bzzs" Jul 15 21:09:52.710708 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:52.710677 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-7bzzs"] Jul 15 21:09:52.711989 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:09:52.711958 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd1829fe_534c_44fc_b6cf_5761acfa46a0.slice/crio-c3a353425fd11768e66100a4524765198d4ba1a91d8448e127271e1cdb3a9f75 WatchSource:0}: Error finding container c3a353425fd11768e66100a4524765198d4ba1a91d8448e127271e1cdb3a9f75: Status 404 returned error can't find the container with id c3a353425fd11768e66100a4524765198d4ba1a91d8448e127271e1cdb3a9f75 Jul 15 21:09:53.103859 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:53.103823 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7bzzs" event={"ID":"cd1829fe-534c-44fc-b6cf-5761acfa46a0","Type":"ContainerStarted","Data":"c3a353425fd11768e66100a4524765198d4ba1a91d8448e127271e1cdb3a9f75"} Jul 15 21:09:53.356940 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:53.356910 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:09:54.989515 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:54.989473 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:09:55.185409 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:55.185370 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:09:56.120291 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:56.120253 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7bzzs" event={"ID":"cd1829fe-534c-44fc-b6cf-5761acfa46a0","Type":"ContainerStarted","Data":"37e519e0e4cd260d08da16d5acb9a7a27d3b7cea2535ede53350385e3d03ccb2"} Jul 15 21:09:56.120752 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:56.120345 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7bzzs" Jul 15 21:09:56.154971 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:56.154918 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7bzzs" podStartSLOduration=1.72144074 podStartE2EDuration="4.154901275s" totalImagesPullingTime="2.433460535s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:09:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:09:56.154141271 +0000 UTC m=+500.695453820" watchObservedRunningTime="2026-07-15 21:09:56.154901275 +0000 UTC m=+500.696213876" Jul 15 21:09:57.206039 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:57.206007 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:09:58.045969 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:58.045928 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zpp5t"] Jul 15 21:09:58.048794 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:58.048776 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zpp5t" Jul 15 21:09:58.057313 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:58.057293 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-jn62q\"" Jul 15 21:09:58.100922 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:58.100892 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zpp5t"] Jul 15 21:09:58.103859 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:58.103824 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sls7g\" (UniqueName: \"kubernetes.io/projected/f9e09443-653f-4079-a863-6fe601ab6842-kube-api-access-sls7g\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zpp5t\" (UID: \"f9e09443-653f-4079-a863-6fe601ab6842\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zpp5t" Jul 15 21:09:58.103972 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:58.103861 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9e09443-653f-4079-a863-6fe601ab6842-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zpp5t\" (UID: \"f9e09443-653f-4079-a863-6fe601ab6842\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zpp5t" Jul 15 21:09:58.204599 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:58.204561 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sls7g\" (UniqueName: \"kubernetes.io/projected/f9e09443-653f-4079-a863-6fe601ab6842-kube-api-access-sls7g\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zpp5t\" (UID: \"f9e09443-653f-4079-a863-6fe601ab6842\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zpp5t" Jul 15 21:09:58.204599 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:58.204603 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9e09443-653f-4079-a863-6fe601ab6842-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zpp5t\" (UID: \"f9e09443-653f-4079-a863-6fe601ab6842\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zpp5t" Jul 15 21:09:58.205096 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:58.205047 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9e09443-653f-4079-a863-6fe601ab6842-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zpp5t\" (UID: \"f9e09443-653f-4079-a863-6fe601ab6842\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zpp5t" Jul 15 21:09:58.215945 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:58.215919 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sls7g\" (UniqueName: \"kubernetes.io/projected/f9e09443-653f-4079-a863-6fe601ab6842-kube-api-access-sls7g\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zpp5t\" (UID: \"f9e09443-653f-4079-a863-6fe601ab6842\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zpp5t" Jul 15 21:09:58.358688 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:58.358650 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zpp5t" Jul 15 21:09:58.516438 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:58.510362 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zpp5t"] Jul 15 21:09:58.520537 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:58.520510 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:09:59.136215 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:09:59.136170 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zpp5t" event={"ID":"f9e09443-653f-4079-a863-6fe601ab6842","Type":"ContainerStarted","Data":"1e9be492eba346a1abfa7b8e9cef07efbacbc4eb5ff48819658434cc04fff2ed"} Jul 15 21:10:00.088051 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:00.088007 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-4cj4n"] Jul 15 21:10:00.091329 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:00.091304 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4cj4n" Jul 15 21:10:00.097928 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:00.097621 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 15 21:10:00.098175 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:00.098158 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-ppd92\"" Jul 15 21:10:00.126303 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:00.126268 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-4cj4n"] Jul 15 21:10:00.220730 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:00.220691 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxhql\" (UniqueName: \"kubernetes.io/projected/799804cd-f071-4c3f-8217-7a20956aeac7-kube-api-access-zxhql\") pod \"dns-operator-controller-manager-864dd78c4c-4cj4n\" (UID: \"799804cd-f071-4c3f-8217-7a20956aeac7\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4cj4n" Jul 15 21:10:00.321454 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:00.321411 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxhql\" (UniqueName: \"kubernetes.io/projected/799804cd-f071-4c3f-8217-7a20956aeac7-kube-api-access-zxhql\") pod \"dns-operator-controller-manager-864dd78c4c-4cj4n\" (UID: \"799804cd-f071-4c3f-8217-7a20956aeac7\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4cj4n" Jul 15 21:10:00.336344 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:00.336309 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxhql\" (UniqueName: \"kubernetes.io/projected/799804cd-f071-4c3f-8217-7a20956aeac7-kube-api-access-zxhql\") pod \"dns-operator-controller-manager-864dd78c4c-4cj4n\" (UID: \"799804cd-f071-4c3f-8217-7a20956aeac7\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4cj4n" Jul 15 21:10:00.402189 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:00.402150 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4cj4n" Jul 15 21:10:00.510517 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:00.510409 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:10:00.567596 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:00.567562 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-4cj4n"] Jul 15 21:10:00.570137 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:10:00.570102 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod799804cd_f071_4c3f_8217_7a20956aeac7.slice/crio-e3cea809014a53429e072908dccc4cae4c2591d955836b59d809c2766281e270 WatchSource:0}: Error finding container e3cea809014a53429e072908dccc4cae4c2591d955836b59d809c2766281e270: Status 404 returned error can't find the container with id e3cea809014a53429e072908dccc4cae4c2591d955836b59d809c2766281e270 Jul 15 21:10:00.662602 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:00.662516 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:10:01.146834 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:01.146801 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4cj4n" event={"ID":"799804cd-f071-4c3f-8217-7a20956aeac7","Type":"ContainerStarted","Data":"e3cea809014a53429e072908dccc4cae4c2591d955836b59d809c2766281e270"} Jul 15 21:10:02.915090 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:02.915023 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:10:03.022580 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:03.022493 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j77m4"] Jul 15 21:10:03.025986 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:03.025956 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j77m4" Jul 15 21:10:03.031838 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:03.031815 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-xvq6c\"" Jul 15 21:10:03.051432 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:03.051392 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j77m4"] Jul 15 21:10:03.145418 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:03.145379 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tbmw\" (UniqueName: \"kubernetes.io/projected/42665a9e-7946-4c53-9cbe-33c37d0b1852-kube-api-access-7tbmw\") pod \"limitador-operator-controller-manager-859d7b78bd-j77m4\" (UID: \"42665a9e-7946-4c53-9cbe-33c37d0b1852\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j77m4" Jul 15 21:10:03.161468 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:03.161426 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4cj4n" event={"ID":"799804cd-f071-4c3f-8217-7a20956aeac7","Type":"ContainerStarted","Data":"7795763290e0b3d9ecdc5528a1e596e8e9c70b86209a1f195627e6f11bdf00e9"} Jul 15 21:10:03.161661 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:03.161526 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4cj4n" Jul 15 21:10:03.206502 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:03.206434 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4cj4n" podStartSLOduration=1.053971038 podStartE2EDuration="3.206413184s" totalImagesPullingTime="2.152442146s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:10:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:10:03.204747548 +0000 UTC m=+507.746060098" watchObservedRunningTime="2026-07-15 21:10:03.206413184 +0000 UTC m=+507.747725735" Jul 15 21:10:03.245947 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:03.245890 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tbmw\" (UniqueName: \"kubernetes.io/projected/42665a9e-7946-4c53-9cbe-33c37d0b1852-kube-api-access-7tbmw\") pod \"limitador-operator-controller-manager-859d7b78bd-j77m4\" (UID: \"42665a9e-7946-4c53-9cbe-33c37d0b1852\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j77m4" Jul 15 21:10:03.284159 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:03.284084 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tbmw\" (UniqueName: \"kubernetes.io/projected/42665a9e-7946-4c53-9cbe-33c37d0b1852-kube-api-access-7tbmw\") pod \"limitador-operator-controller-manager-859d7b78bd-j77m4\" (UID: \"42665a9e-7946-4c53-9cbe-33c37d0b1852\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j77m4" Jul 15 21:10:03.338107 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:03.338041 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j77m4" Jul 15 21:10:04.375742 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:04.375719 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j77m4"] Jul 15 21:10:04.637171 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:04.637129 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:10:04.787667 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:04.787618 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:10:05.013436 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:05.013349 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:10:05.174104 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:05.174043 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j77m4" event={"ID":"42665a9e-7946-4c53-9cbe-33c37d0b1852","Type":"ContainerStarted","Data":"ad7055ea996d0ea4a7fb37770f3d81bc9a16ea37dbfcd2b4a29f3b51b83fe2fa"} Jul 15 21:10:05.175599 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:05.175577 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zpp5t" event={"ID":"f9e09443-653f-4079-a863-6fe601ab6842","Type":"ContainerStarted","Data":"49139c5caad1ec8958e8b683f1ba8918bb314e8f1673cde27d1f5a22ae463374"} Jul 15 21:10:05.175732 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:05.175716 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zpp5t" Jul 15 21:10:05.358045 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:05.358005 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:10:07.127843 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:07.127810 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7bzzs" Jul 15 21:10:07.177399 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:07.177334 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zpp5t" podStartSLOduration=4.443528309 podStartE2EDuration="10.177312381s" totalImagesPullingTime="5.733784072s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:09:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:10:05.235981719 +0000 UTC m=+509.777294280" watchObservedRunningTime="2026-07-15 21:10:07.177312381 +0000 UTC m=+511.718624932" Jul 15 21:10:07.642343 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:07.642246 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:10:08.153997 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:08.153946 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:10:08.193728 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:08.193694 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j77m4" event={"ID":"42665a9e-7946-4c53-9cbe-33c37d0b1852","Type":"ContainerStarted","Data":"577033e92d2901ae2459bea547b423c49ca609b2c2c53d835f0aa14885723ac9"} Jul 15 21:10:08.193897 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:08.193769 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j77m4" Jul 15 21:10:08.263959 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:08.263908 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j77m4" podStartSLOduration=3.335730872 podStartE2EDuration="6.263892055s" totalImagesPullingTime="2.928161183s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:10:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:10:08.262803307 +0000 UTC m=+512.804115857" watchObservedRunningTime="2026-07-15 21:10:08.263892055 +0000 UTC m=+512.805204604" Jul 15 21:10:08.576693 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:08.576584 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:10:10.528962 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:10.528929 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:10:10.688893 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:10.688849 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:10:14.170416 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:14.170388 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4cj4n" Jul 15 21:10:16.183583 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:16.183548 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zpp5t" Jul 15 21:10:16.510285 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:16.510182 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:10:16.609997 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:16.609948 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:10:18.319481 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:18.319447 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zpp5t"] Jul 15 21:10:18.319866 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:18.319635 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zpp5t" podUID="f9e09443-653f-4079-a863-6fe601ab6842" containerName="manager" containerID="cri-o://49139c5caad1ec8958e8b683f1ba8918bb314e8f1673cde27d1f5a22ae463374" gracePeriod=2 Jul 15 21:10:18.327164 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:18.327135 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zpp5t"] Jul 15 21:10:18.355009 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:18.354978 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j77m4"] Jul 15 21:10:18.355240 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:18.355209 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j77m4" podUID="42665a9e-7946-4c53-9cbe-33c37d0b1852" containerName="manager" containerID="cri-o://577033e92d2901ae2459bea547b423c49ca609b2c2c53d835f0aa14885723ac9" gracePeriod=2 Jul 15 21:10:18.357206 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:18.357181 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j77m4" Jul 15 21:10:18.360110 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:10:18.360051 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-zpp5t\" is forbidden: User \"system:node:ip-10-0-137-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-185.ec2.internal' and this object" podUID="f9e09443-653f-4079-a863-6fe601ab6842" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zpp5t" Jul 15 21:10:18.372127 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:18.372098 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j77m4"] Jul 15 21:10:18.393624 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:18.393599 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n6qwj"] Jul 15 21:10:18.393926 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:18.393912 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42665a9e-7946-4c53-9cbe-33c37d0b1852" containerName="manager" Jul 15 21:10:18.393926 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:18.393927 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="42665a9e-7946-4c53-9cbe-33c37d0b1852" containerName="manager" Jul 15 21:10:18.394054 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:18.393945 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9e09443-653f-4079-a863-6fe601ab6842" containerName="manager" Jul 15 21:10:18.394054 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:18.393951 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9e09443-653f-4079-a863-6fe601ab6842" containerName="manager" Jul 15 21:10:18.394054 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:18.394003 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9e09443-653f-4079-a863-6fe601ab6842" containerName="manager" Jul 15 21:10:18.394054 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:18.394014 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="42665a9e-7946-4c53-9cbe-33c37d0b1852" containerName="manager" Jul 15 21:10:18.397193 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:18.397169 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n6qwj" Jul 15 21:10:18.402193 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:10:18.402164 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-zpp5t\" is forbidden: User \"system:node:ip-10-0-137-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-185.ec2.internal' and this object" podUID="f9e09443-653f-4079-a863-6fe601ab6842" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zpp5t" Jul 15 21:10:18.404650 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:10:18.404628 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-j77m4\" is forbidden: User \"system:node:ip-10-0-137-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-185.ec2.internal' and this object" podUID="42665a9e-7946-4c53-9cbe-33c37d0b1852" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j77m4" Jul 15 21:10:18.407818 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:10:18.407798 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-zpp5t\" is forbidden: User \"system:node:ip-10-0-137-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-185.ec2.internal' and this object" podUID="f9e09443-653f-4079-a863-6fe601ab6842" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zpp5t" Jul 15 21:10:18.433454 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:18.433423 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fpp7b"] Jul 15 21:10:18.436576 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:18.436550 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fpp7b" Jul 15 21:10:18.449665 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:18.449636 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n6qwj"] Jul 15 21:10:18.478659 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:18.478623 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrcwt\" (UniqueName: \"kubernetes.io/projected/d3d1b60e-5107-499a-a26b-b095aefe3d92-kube-api-access-vrcwt\") pod \"kuadrant-operator-controller-manager-8684c9ff49-n6qwj\" (UID: \"d3d1b60e-5107-499a-a26b-b095aefe3d92\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n6qwj" Jul 15 21:10:18.478765 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:18.478744 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d3d1b60e-5107-499a-a26b-b095aefe3d92-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-n6qwj\" (UID: \"d3d1b60e-5107-499a-a26b-b095aefe3d92\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n6qwj" Jul 15 21:10:18.479677 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:18.479649 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fpp7b"] Jul 15 21:10:18.499596 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:10:18.499557 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-j77m4\" is forbidden: User \"system:node:ip-10-0-137-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-185.ec2.internal' and this object" podUID="42665a9e-7946-4c53-9cbe-33c37d0b1852" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j77m4" Jul 15 21:10:18.540702 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:10:18.540662 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-zpp5t\" is forbidden: User \"system:node:ip-10-0-137-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-185.ec2.internal' and this object" podUID="f9e09443-653f-4079-a863-6fe601ab6842" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zpp5t" Jul 15 21:10:18.543838 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:10:18.543794 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-j77m4\" is forbidden: User \"system:node:ip-10-0-137-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-185.ec2.internal' and this object" podUID="42665a9e-7946-4c53-9cbe-33c37d0b1852" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j77m4" Jul 15 21:10:18.579268 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:18.579187 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nszs\" (UniqueName: \"kubernetes.io/projected/86c51d13-9e24-46d2-af54-3002ca89dec3-kube-api-access-6nszs\") pod \"limitador-operator-controller-manager-859d7b78bd-fpp7b\" (UID: \"86c51d13-9e24-46d2-af54-3002ca89dec3\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fpp7b" Jul 15 21:10:18.579268 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:18.579229 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrcwt\" (UniqueName: \"kubernetes.io/projected/d3d1b60e-5107-499a-a26b-b095aefe3d92-kube-api-access-vrcwt\") pod \"kuadrant-operator-controller-manager-8684c9ff49-n6qwj\" (UID: \"d3d1b60e-5107-499a-a26b-b095aefe3d92\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n6qwj" Jul 15 21:10:18.579268 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:18.579266 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d3d1b60e-5107-499a-a26b-b095aefe3d92-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-n6qwj\" (UID: \"d3d1b60e-5107-499a-a26b-b095aefe3d92\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n6qwj" Jul 15 21:10:18.579588 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:18.579569 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d3d1b60e-5107-499a-a26b-b095aefe3d92-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-n6qwj\" (UID: \"d3d1b60e-5107-499a-a26b-b095aefe3d92\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n6qwj" Jul 15 21:10:18.597208 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:18.597187 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrcwt\" (UniqueName: \"kubernetes.io/projected/d3d1b60e-5107-499a-a26b-b095aefe3d92-kube-api-access-vrcwt\") pod \"kuadrant-operator-controller-manager-8684c9ff49-n6qwj\" (UID: \"d3d1b60e-5107-499a-a26b-b095aefe3d92\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n6qwj" Jul 15 21:10:18.598935 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:18.598912 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zpp5t" Jul 15 21:10:18.601988 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:10:18.601956 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-j77m4\" is forbidden: User \"system:node:ip-10-0-137-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-185.ec2.internal' and this object" podUID="42665a9e-7946-4c53-9cbe-33c37d0b1852" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j77m4" Jul 15 21:10:18.606043 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:10:18.605999 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-zpp5t\" is forbidden: User \"system:node:ip-10-0-137-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-185.ec2.internal' and this object" podUID="f9e09443-653f-4079-a863-6fe601ab6842" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zpp5t" Jul 15 21:10:18.613241 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:18.613222 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j77m4" Jul 15 21:10:18.615632 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:10:18.615609 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-j77m4\" is forbidden: User \"system:node:ip-10-0-137-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-185.ec2.internal' and this object" podUID="42665a9e-7946-4c53-9cbe-33c37d0b1852" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j77m4" Jul 15 21:10:18.620090 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:10:18.620034 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-zpp5t\" is forbidden: User \"system:node:ip-10-0-137-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-185.ec2.internal' and this object" podUID="f9e09443-653f-4079-a863-6fe601ab6842" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zpp5t" Jul 15 21:10:18.668956 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:18.668921 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:10:18.679626 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:18.679600 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7tbmw\" (UniqueName: \"kubernetes.io/projected/42665a9e-7946-4c53-9cbe-33c37d0b1852-kube-api-access-7tbmw\") pod \"42665a9e-7946-4c53-9cbe-33c37d0b1852\" (UID: \"42665a9e-7946-4c53-9cbe-33c37d0b1852\") " Jul 15 21:10:18.679785 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:18.679664 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sls7g\" (UniqueName: \"kubernetes.io/projected/f9e09443-653f-4079-a863-6fe601ab6842-kube-api-access-sls7g\") pod \"f9e09443-653f-4079-a863-6fe601ab6842\" (UID: \"f9e09443-653f-4079-a863-6fe601ab6842\") " Jul 15 21:10:18.679785 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:18.679717 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9e09443-653f-4079-a863-6fe601ab6842-extensions-socket-volume\") pod \"f9e09443-653f-4079-a863-6fe601ab6842\" (UID: \"f9e09443-653f-4079-a863-6fe601ab6842\") " Jul 15 21:10:18.680037 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:18.679969 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nszs\" (UniqueName: \"kubernetes.io/projected/86c51d13-9e24-46d2-af54-3002ca89dec3-kube-api-access-6nszs\") pod \"limitador-operator-controller-manager-859d7b78bd-fpp7b\" (UID: \"86c51d13-9e24-46d2-af54-3002ca89dec3\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fpp7b" Jul 15 21:10:18.680362 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:18.680334 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9e09443-653f-4079-a863-6fe601ab6842-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f9e09443-653f-4079-a863-6fe601ab6842" (UID: "f9e09443-653f-4079-a863-6fe601ab6842"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:10:18.682341 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:18.682318 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9e09443-653f-4079-a863-6fe601ab6842-kube-api-access-sls7g" (OuterVolumeSpecName: "kube-api-access-sls7g") pod "f9e09443-653f-4079-a863-6fe601ab6842" (UID: "f9e09443-653f-4079-a863-6fe601ab6842"). InnerVolumeSpecName "kube-api-access-sls7g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:10:18.682425 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:18.682321 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42665a9e-7946-4c53-9cbe-33c37d0b1852-kube-api-access-7tbmw" (OuterVolumeSpecName: "kube-api-access-7tbmw") pod "42665a9e-7946-4c53-9cbe-33c37d0b1852" (UID: "42665a9e-7946-4c53-9cbe-33c37d0b1852"). InnerVolumeSpecName "kube-api-access-7tbmw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:10:18.693575 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:18.693536 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nszs\" (UniqueName: \"kubernetes.io/projected/86c51d13-9e24-46d2-af54-3002ca89dec3-kube-api-access-6nszs\") pod \"limitador-operator-controller-manager-859d7b78bd-fpp7b\" (UID: \"86c51d13-9e24-46d2-af54-3002ca89dec3\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fpp7b" Jul 15 21:10:18.746982 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:18.746952 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n6qwj" Jul 15 21:10:18.754159 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:18.754138 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fpp7b" Jul 15 21:10:18.781349 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:18.781319 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sls7g\" (UniqueName: \"kubernetes.io/projected/f9e09443-653f-4079-a863-6fe601ab6842-kube-api-access-sls7g\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:10:18.781349 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:18.781346 2579 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9e09443-653f-4079-a863-6fe601ab6842-extensions-socket-volume\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:10:18.781349 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:18.781359 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7tbmw\" (UniqueName: \"kubernetes.io/projected/42665a9e-7946-4c53-9cbe-33c37d0b1852-kube-api-access-7tbmw\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:10:18.835539 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:18.835453 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:10:18.912479 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:18.912450 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n6qwj"] Jul 15 21:10:18.914413 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:10:18.914387 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3d1b60e_5107_499a_a26b_b095aefe3d92.slice/crio-45d67c0633c2508ebacfe53d925ae6a501d954d2336967c9f675ea291df50c3d WatchSource:0}: Error finding container 45d67c0633c2508ebacfe53d925ae6a501d954d2336967c9f675ea291df50c3d: Status 404 returned error can't find the container with id 45d67c0633c2508ebacfe53d925ae6a501d954d2336967c9f675ea291df50c3d Jul 15 21:10:18.929290 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:18.929259 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fpp7b"] Jul 15 21:10:18.929634 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:10:18.929608 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86c51d13_9e24_46d2_af54_3002ca89dec3.slice/crio-47b681ba2643bf8a61e143d4c77e2da51667c1e860e866e99d96b7b2c7b97feb WatchSource:0}: Error finding container 47b681ba2643bf8a61e143d4c77e2da51667c1e860e866e99d96b7b2c7b97feb: Status 404 returned error can't find the container with id 47b681ba2643bf8a61e143d4c77e2da51667c1e860e866e99d96b7b2c7b97feb Jul 15 21:10:18.980803 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:18.980774 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:10:19.247641 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:19.247609 2579 generic.go:356] "Generic (PLEG): container finished" podID="42665a9e-7946-4c53-9cbe-33c37d0b1852" containerID="577033e92d2901ae2459bea547b423c49ca609b2c2c53d835f0aa14885723ac9" exitCode=0 Jul 15 21:10:19.247823 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:19.247652 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j77m4" Jul 15 21:10:19.247823 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:19.247699 2579 scope.go:117] "RemoveContainer" containerID="577033e92d2901ae2459bea547b423c49ca609b2c2c53d835f0aa14885723ac9" Jul 15 21:10:19.249370 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:19.249336 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fpp7b" event={"ID":"86c51d13-9e24-46d2-af54-3002ca89dec3","Type":"ContainerStarted","Data":"359ab3592d84f1d0bb73acc135bd3113de6165448712cd1324b870b792a9262a"} Jul 15 21:10:19.249509 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:19.249408 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fpp7b" Jul 15 21:10:19.249509 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:19.249436 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fpp7b" event={"ID":"86c51d13-9e24-46d2-af54-3002ca89dec3","Type":"ContainerStarted","Data":"47b681ba2643bf8a61e143d4c77e2da51667c1e860e866e99d96b7b2c7b97feb"} Jul 15 21:10:19.250755 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:19.250733 2579 generic.go:356] "Generic (PLEG): container finished" podID="f9e09443-653f-4079-a863-6fe601ab6842" containerID="49139c5caad1ec8958e8b683f1ba8918bb314e8f1673cde27d1f5a22ae463374" exitCode=0 Jul 15 21:10:19.250876 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:19.250769 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zpp5t" Jul 15 21:10:19.251319 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:10:19.251287 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-j77m4\" is forbidden: User \"system:node:ip-10-0-137-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-185.ec2.internal' and this object" podUID="42665a9e-7946-4c53-9cbe-33c37d0b1852" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j77m4" Jul 15 21:10:19.254773 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:10:19.254741 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-zpp5t\" is forbidden: User \"system:node:ip-10-0-137-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-185.ec2.internal' and this object" podUID="f9e09443-653f-4079-a863-6fe601ab6842" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zpp5t" Jul 15 21:10:19.257416 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:10:19.257384 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-j77m4\" is forbidden: User \"system:node:ip-10-0-137-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-185.ec2.internal' and this object" podUID="42665a9e-7946-4c53-9cbe-33c37d0b1852" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j77m4" Jul 15 21:10:19.259995 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:19.259966 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n6qwj" event={"ID":"d3d1b60e-5107-499a-a26b-b095aefe3d92","Type":"ContainerStarted","Data":"18dee81007fb7eb94e26bd064d81d3875303af078c3172464044e2156dc9ae88"} Jul 15 21:10:19.260146 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:19.260052 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n6qwj" event={"ID":"d3d1b60e-5107-499a-a26b-b095aefe3d92","Type":"ContainerStarted","Data":"45d67c0633c2508ebacfe53d925ae6a501d954d2336967c9f675ea291df50c3d"} Jul 15 21:10:19.260209 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:19.260173 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n6qwj" Jul 15 21:10:19.263949 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:19.263933 2579 scope.go:117] "RemoveContainer" containerID="577033e92d2901ae2459bea547b423c49ca609b2c2c53d835f0aa14885723ac9" Jul 15 21:10:19.264209 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:10:19.264191 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"577033e92d2901ae2459bea547b423c49ca609b2c2c53d835f0aa14885723ac9\": container with ID starting with 577033e92d2901ae2459bea547b423c49ca609b2c2c53d835f0aa14885723ac9 not found: ID does not exist" containerID="577033e92d2901ae2459bea547b423c49ca609b2c2c53d835f0aa14885723ac9" Jul 15 21:10:19.264269 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:19.264218 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"577033e92d2901ae2459bea547b423c49ca609b2c2c53d835f0aa14885723ac9"} err="failed to get container status \"577033e92d2901ae2459bea547b423c49ca609b2c2c53d835f0aa14885723ac9\": rpc error: code = NotFound desc = could not find container \"577033e92d2901ae2459bea547b423c49ca609b2c2c53d835f0aa14885723ac9\": container with ID starting with 577033e92d2901ae2459bea547b423c49ca609b2c2c53d835f0aa14885723ac9 not found: ID does not exist" Jul 15 21:10:19.264269 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:19.264234 2579 scope.go:117] "RemoveContainer" containerID="49139c5caad1ec8958e8b683f1ba8918bb314e8f1673cde27d1f5a22ae463374" Jul 15 21:10:19.272322 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:19.272304 2579 scope.go:117] "RemoveContainer" containerID="49139c5caad1ec8958e8b683f1ba8918bb314e8f1673cde27d1f5a22ae463374" Jul 15 21:10:19.272577 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:10:19.272557 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49139c5caad1ec8958e8b683f1ba8918bb314e8f1673cde27d1f5a22ae463374\": container with ID starting with 49139c5caad1ec8958e8b683f1ba8918bb314e8f1673cde27d1f5a22ae463374 not found: ID does not exist" containerID="49139c5caad1ec8958e8b683f1ba8918bb314e8f1673cde27d1f5a22ae463374" Jul 15 21:10:19.272616 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:19.272586 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49139c5caad1ec8958e8b683f1ba8918bb314e8f1673cde27d1f5a22ae463374"} err="failed to get container status \"49139c5caad1ec8958e8b683f1ba8918bb314e8f1673cde27d1f5a22ae463374\": rpc error: code = NotFound desc = could not find container \"49139c5caad1ec8958e8b683f1ba8918bb314e8f1673cde27d1f5a22ae463374\": container with ID starting with 49139c5caad1ec8958e8b683f1ba8918bb314e8f1673cde27d1f5a22ae463374 not found: ID does not exist" Jul 15 21:10:19.282046 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:19.282010 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fpp7b" podStartSLOduration=1.2819965930000001 podStartE2EDuration="1.281996593s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:10:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:10:19.280637217 +0000 UTC m=+523.821949785" watchObservedRunningTime="2026-07-15 21:10:19.281996593 +0000 UTC m=+523.823309147" Jul 15 21:10:19.284214 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:10:19.284183 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-zpp5t\" is forbidden: User \"system:node:ip-10-0-137-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-185.ec2.internal' and this object" podUID="f9e09443-653f-4079-a863-6fe601ab6842" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zpp5t" Jul 15 21:10:19.287351 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:10:19.287324 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-j77m4\" is forbidden: User \"system:node:ip-10-0-137-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-185.ec2.internal' and this object" podUID="42665a9e-7946-4c53-9cbe-33c37d0b1852" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j77m4" Jul 15 21:10:19.291617 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:10:19.291591 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-zpp5t\" is forbidden: User \"system:node:ip-10-0-137-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-185.ec2.internal' and this object" podUID="f9e09443-653f-4079-a863-6fe601ab6842" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zpp5t" Jul 15 21:10:19.327279 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:19.327218 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n6qwj" podStartSLOduration=1.327196624 podStartE2EDuration="1.327196624s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:10:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:10:19.325086609 +0000 UTC m=+523.866399151" watchObservedRunningTime="2026-07-15 21:10:19.327196624 +0000 UTC m=+523.868509175" Jul 15 21:10:19.843141 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:19.843103 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vbm4v"] Jul 15 21:10:19.848012 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:19.847986 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vbm4v" Jul 15 21:10:19.851357 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:10:19.851327 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-zpp5t\" is forbidden: User \"system:node:ip-10-0-137-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-185.ec2.internal' and this object" podUID="f9e09443-653f-4079-a863-6fe601ab6842" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zpp5t" Jul 15 21:10:19.864175 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:19.864152 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vbm4v"] Jul 15 21:10:19.889878 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:10:19.889828 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-j77m4\" is forbidden: User \"system:node:ip-10-0-137-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-185.ec2.internal' and this object" podUID="42665a9e-7946-4c53-9cbe-33c37d0b1852" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j77m4" Jul 15 21:10:19.990730 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:19.990686 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a7538513-8059-4dc3-b3c5-41fcbab87240-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-vbm4v\" (UID: \"a7538513-8059-4dc3-b3c5-41fcbab87240\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vbm4v" Jul 15 21:10:19.990925 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:19.990748 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7vzh\" (UniqueName: \"kubernetes.io/projected/a7538513-8059-4dc3-b3c5-41fcbab87240-kube-api-access-d7vzh\") pod \"kuadrant-operator-controller-manager-86878c769f-vbm4v\" (UID: \"a7538513-8059-4dc3-b3c5-41fcbab87240\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vbm4v" Jul 15 21:10:19.996574 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:19.996547 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42665a9e-7946-4c53-9cbe-33c37d0b1852" path="/var/lib/kubelet/pods/42665a9e-7946-4c53-9cbe-33c37d0b1852/volumes" Jul 15 21:10:19.996888 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:19.996874 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9e09443-653f-4079-a863-6fe601ab6842" path="/var/lib/kubelet/pods/f9e09443-653f-4079-a863-6fe601ab6842/volumes" Jul 15 21:10:20.091923 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:20.091886 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a7538513-8059-4dc3-b3c5-41fcbab87240-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-vbm4v\" (UID: \"a7538513-8059-4dc3-b3c5-41fcbab87240\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vbm4v" Jul 15 21:10:20.092203 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:20.092170 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7vzh\" (UniqueName: \"kubernetes.io/projected/a7538513-8059-4dc3-b3c5-41fcbab87240-kube-api-access-d7vzh\") pod \"kuadrant-operator-controller-manager-86878c769f-vbm4v\" (UID: \"a7538513-8059-4dc3-b3c5-41fcbab87240\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vbm4v" Jul 15 21:10:20.092301 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:20.092281 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a7538513-8059-4dc3-b3c5-41fcbab87240-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-vbm4v\" (UID: \"a7538513-8059-4dc3-b3c5-41fcbab87240\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vbm4v" Jul 15 21:10:20.107684 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:20.107658 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7vzh\" (UniqueName: \"kubernetes.io/projected/a7538513-8059-4dc3-b3c5-41fcbab87240-kube-api-access-d7vzh\") pod \"kuadrant-operator-controller-manager-86878c769f-vbm4v\" (UID: \"a7538513-8059-4dc3-b3c5-41fcbab87240\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vbm4v" Jul 15 21:10:20.158156 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:20.158119 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vbm4v" Jul 15 21:10:20.295524 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:20.295494 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vbm4v"] Jul 15 21:10:20.297735 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:10:20.297708 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7538513_8059_4dc3_b3c5_41fcbab87240.slice/crio-5fbc6e997cf5e20e7c078b535a08bd469fd933e2302dbb4f4757e84df3039623 WatchSource:0}: Error finding container 5fbc6e997cf5e20e7c078b535a08bd469fd933e2302dbb4f4757e84df3039623: Status 404 returned error can't find the container with id 5fbc6e997cf5e20e7c078b535a08bd469fd933e2302dbb4f4757e84df3039623 Jul 15 21:10:21.275138 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:21.275098 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vbm4v" event={"ID":"a7538513-8059-4dc3-b3c5-41fcbab87240","Type":"ContainerStarted","Data":"21e1f5d751ae65106f2a37b3075c8bd6bba4714b96d240f3c7bca91c6382484c"} Jul 15 21:10:21.275562 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:21.275160 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vbm4v" event={"ID":"a7538513-8059-4dc3-b3c5-41fcbab87240","Type":"ContainerStarted","Data":"5fbc6e997cf5e20e7c078b535a08bd469fd933e2302dbb4f4757e84df3039623"} Jul 15 21:10:21.275562 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:21.275293 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vbm4v" Jul 15 21:10:21.315164 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:21.315110 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vbm4v" podStartSLOduration=2.315092436 podStartE2EDuration="2.315092436s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:10:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:10:21.311984055 +0000 UTC m=+525.853296605" watchObservedRunningTime="2026-07-15 21:10:21.315092436 +0000 UTC m=+525.856404986" Jul 15 21:10:21.527147 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:21.527026 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:10:21.630748 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:21.630705 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:10:24.827982 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:24.827924 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:10:25.082814 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:25.082722 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:10:29.305460 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:29.305377 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:10:29.406221 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:29.406179 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:10:30.270690 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:30.270659 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n6qwj" Jul 15 21:10:30.270885 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:30.270862 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fpp7b" Jul 15 21:10:32.282621 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:32.282592 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vbm4v" Jul 15 21:10:32.409928 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:32.409894 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n6qwj"] Jul 15 21:10:32.410143 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:32.410098 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n6qwj" podUID="d3d1b60e-5107-499a-a26b-b095aefe3d92" containerName="manager" containerID="cri-o://18dee81007fb7eb94e26bd064d81d3875303af078c3172464044e2156dc9ae88" gracePeriod=10 Jul 15 21:10:32.653829 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:32.653801 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n6qwj" Jul 15 21:10:32.803779 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:32.803750 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vrcwt\" (UniqueName: \"kubernetes.io/projected/d3d1b60e-5107-499a-a26b-b095aefe3d92-kube-api-access-vrcwt\") pod \"d3d1b60e-5107-499a-a26b-b095aefe3d92\" (UID: \"d3d1b60e-5107-499a-a26b-b095aefe3d92\") " Jul 15 21:10:32.803927 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:32.803783 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d3d1b60e-5107-499a-a26b-b095aefe3d92-extensions-socket-volume\") pod \"d3d1b60e-5107-499a-a26b-b095aefe3d92\" (UID: \"d3d1b60e-5107-499a-a26b-b095aefe3d92\") " Jul 15 21:10:32.804219 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:32.804182 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3d1b60e-5107-499a-a26b-b095aefe3d92-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d3d1b60e-5107-499a-a26b-b095aefe3d92" (UID: "d3d1b60e-5107-499a-a26b-b095aefe3d92"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:10:32.805824 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:32.805800 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3d1b60e-5107-499a-a26b-b095aefe3d92-kube-api-access-vrcwt" (OuterVolumeSpecName: "kube-api-access-vrcwt") pod "d3d1b60e-5107-499a-a26b-b095aefe3d92" (UID: "d3d1b60e-5107-499a-a26b-b095aefe3d92"). InnerVolumeSpecName "kube-api-access-vrcwt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:10:32.904476 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:32.904443 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vrcwt\" (UniqueName: \"kubernetes.io/projected/d3d1b60e-5107-499a-a26b-b095aefe3d92-kube-api-access-vrcwt\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:10:32.904476 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:32.904470 2579 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d3d1b60e-5107-499a-a26b-b095aefe3d92-extensions-socket-volume\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:10:33.340638 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:33.340543 2579 generic.go:356] "Generic (PLEG): container finished" podID="d3d1b60e-5107-499a-a26b-b095aefe3d92" containerID="18dee81007fb7eb94e26bd064d81d3875303af078c3172464044e2156dc9ae88" exitCode=0 Jul 15 21:10:33.340638 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:33.340583 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n6qwj" Jul 15 21:10:33.341105 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:33.340631 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n6qwj" event={"ID":"d3d1b60e-5107-499a-a26b-b095aefe3d92","Type":"ContainerDied","Data":"18dee81007fb7eb94e26bd064d81d3875303af078c3172464044e2156dc9ae88"} Jul 15 21:10:33.341105 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:33.340684 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n6qwj" event={"ID":"d3d1b60e-5107-499a-a26b-b095aefe3d92","Type":"ContainerDied","Data":"45d67c0633c2508ebacfe53d925ae6a501d954d2336967c9f675ea291df50c3d"} Jul 15 21:10:33.341105 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:33.340706 2579 scope.go:117] "RemoveContainer" containerID="18dee81007fb7eb94e26bd064d81d3875303af078c3172464044e2156dc9ae88" Jul 15 21:10:33.350208 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:33.350183 2579 scope.go:117] "RemoveContainer" containerID="18dee81007fb7eb94e26bd064d81d3875303af078c3172464044e2156dc9ae88" Jul 15 21:10:33.350464 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:10:33.350447 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18dee81007fb7eb94e26bd064d81d3875303af078c3172464044e2156dc9ae88\": container with ID starting with 18dee81007fb7eb94e26bd064d81d3875303af078c3172464044e2156dc9ae88 not found: ID does not exist" containerID="18dee81007fb7eb94e26bd064d81d3875303af078c3172464044e2156dc9ae88" Jul 15 21:10:33.350507 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:33.350473 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18dee81007fb7eb94e26bd064d81d3875303af078c3172464044e2156dc9ae88"} err="failed to get container status \"18dee81007fb7eb94e26bd064d81d3875303af078c3172464044e2156dc9ae88\": rpc error: code = NotFound desc = could not find container \"18dee81007fb7eb94e26bd064d81d3875303af078c3172464044e2156dc9ae88\": container with ID starting with 18dee81007fb7eb94e26bd064d81d3875303af078c3172464044e2156dc9ae88 not found: ID does not exist" Jul 15 21:10:33.391930 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:33.391893 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n6qwj"] Jul 15 21:10:33.393748 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:33.393729 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n6qwj"] Jul 15 21:10:33.996606 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:33.996566 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3d1b60e-5107-499a-a26b-b095aefe3d92" path="/var/lib/kubelet/pods/d3d1b60e-5107-499a-a26b-b095aefe3d92/volumes" Jul 15 21:10:35.616994 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:35.616952 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:10:35.846279 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:35.846245 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:10:40.139639 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:40.139601 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:10:40.306133 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:40.306093 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:10:47.226938 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:47.226898 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:10:49.006823 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:49.006789 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-622mb"] Jul 15 21:10:49.007322 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:49.007306 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3d1b60e-5107-499a-a26b-b095aefe3d92" containerName="manager" Jul 15 21:10:49.007390 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:49.007326 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3d1b60e-5107-499a-a26b-b095aefe3d92" containerName="manager" Jul 15 21:10:49.007442 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:49.007418 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3d1b60e-5107-499a-a26b-b095aefe3d92" containerName="manager" Jul 15 21:10:49.014719 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:49.014693 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-622mb" Jul 15 21:10:49.017820 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:49.017798 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-qbd2f\"" Jul 15 21:10:49.025572 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:49.025545 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-622mb"] Jul 15 21:10:49.028220 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:49.028192 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:10:49.031185 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:49.031158 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9de6c58-b9b9-479a-b22a-57e8b98286be-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-622mb\" (UID: \"c9de6c58-b9b9-479a-b22a-57e8b98286be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-622mb" Jul 15 21:10:49.031295 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:49.031186 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9de6c58-b9b9-479a-b22a-57e8b98286be-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-622mb\" (UID: \"c9de6c58-b9b9-479a-b22a-57e8b98286be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-622mb" Jul 15 21:10:49.031295 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:49.031215 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9de6c58-b9b9-479a-b22a-57e8b98286be-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-622mb\" (UID: \"c9de6c58-b9b9-479a-b22a-57e8b98286be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-622mb" Jul 15 21:10:49.031372 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:49.031324 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mv926\" (UniqueName: \"kubernetes.io/projected/c9de6c58-b9b9-479a-b22a-57e8b98286be-kube-api-access-mv926\") pod \"maas-default-gateway-openshift-default-56654dcf7-622mb\" (UID: \"c9de6c58-b9b9-479a-b22a-57e8b98286be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-622mb" Jul 15 21:10:49.031410 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:49.031373 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9de6c58-b9b9-479a-b22a-57e8b98286be-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-622mb\" (UID: \"c9de6c58-b9b9-479a-b22a-57e8b98286be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-622mb" Jul 15 21:10:49.031410 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:49.031399 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9de6c58-b9b9-479a-b22a-57e8b98286be-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-622mb\" (UID: \"c9de6c58-b9b9-479a-b22a-57e8b98286be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-622mb" Jul 15 21:10:49.031492 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:49.031419 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9de6c58-b9b9-479a-b22a-57e8b98286be-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-622mb\" (UID: \"c9de6c58-b9b9-479a-b22a-57e8b98286be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-622mb" Jul 15 21:10:49.031615 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:49.031586 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9de6c58-b9b9-479a-b22a-57e8b98286be-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-622mb\" (UID: \"c9de6c58-b9b9-479a-b22a-57e8b98286be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-622mb" Jul 15 21:10:49.031689 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:49.031673 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9de6c58-b9b9-479a-b22a-57e8b98286be-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-622mb\" (UID: \"c9de6c58-b9b9-479a-b22a-57e8b98286be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-622mb" Jul 15 21:10:49.132720 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:49.132683 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mv926\" (UniqueName: \"kubernetes.io/projected/c9de6c58-b9b9-479a-b22a-57e8b98286be-kube-api-access-mv926\") pod \"maas-default-gateway-openshift-default-56654dcf7-622mb\" (UID: \"c9de6c58-b9b9-479a-b22a-57e8b98286be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-622mb" Jul 15 21:10:49.132896 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:49.132735 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9de6c58-b9b9-479a-b22a-57e8b98286be-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-622mb\" (UID: \"c9de6c58-b9b9-479a-b22a-57e8b98286be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-622mb" Jul 15 21:10:49.132896 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:49.132768 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9de6c58-b9b9-479a-b22a-57e8b98286be-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-622mb\" (UID: \"c9de6c58-b9b9-479a-b22a-57e8b98286be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-622mb" Jul 15 21:10:49.132896 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:49.132791 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9de6c58-b9b9-479a-b22a-57e8b98286be-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-622mb\" (UID: \"c9de6c58-b9b9-479a-b22a-57e8b98286be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-622mb" Jul 15 21:10:49.132896 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:49.132831 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9de6c58-b9b9-479a-b22a-57e8b98286be-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-622mb\" (UID: \"c9de6c58-b9b9-479a-b22a-57e8b98286be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-622mb" Jul 15 21:10:49.132896 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:49.132869 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9de6c58-b9b9-479a-b22a-57e8b98286be-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-622mb\" (UID: \"c9de6c58-b9b9-479a-b22a-57e8b98286be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-622mb" Jul 15 21:10:49.133211 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:49.132913 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9de6c58-b9b9-479a-b22a-57e8b98286be-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-622mb\" (UID: \"c9de6c58-b9b9-479a-b22a-57e8b98286be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-622mb" Jul 15 21:10:49.133211 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:49.132942 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9de6c58-b9b9-479a-b22a-57e8b98286be-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-622mb\" (UID: \"c9de6c58-b9b9-479a-b22a-57e8b98286be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-622mb" Jul 15 21:10:49.133211 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:49.132979 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9de6c58-b9b9-479a-b22a-57e8b98286be-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-622mb\" (UID: \"c9de6c58-b9b9-479a-b22a-57e8b98286be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-622mb" Jul 15 21:10:49.133369 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:49.133309 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9de6c58-b9b9-479a-b22a-57e8b98286be-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-622mb\" (UID: \"c9de6c58-b9b9-479a-b22a-57e8b98286be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-622mb" Jul 15 21:10:49.133423 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:49.133390 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9de6c58-b9b9-479a-b22a-57e8b98286be-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-622mb\" (UID: \"c9de6c58-b9b9-479a-b22a-57e8b98286be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-622mb" Jul 15 21:10:49.133556 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:49.133528 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9de6c58-b9b9-479a-b22a-57e8b98286be-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-622mb\" (UID: \"c9de6c58-b9b9-479a-b22a-57e8b98286be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-622mb" Jul 15 21:10:49.133646 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:49.133554 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9de6c58-b9b9-479a-b22a-57e8b98286be-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-622mb\" (UID: \"c9de6c58-b9b9-479a-b22a-57e8b98286be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-622mb" Jul 15 21:10:49.133797 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:49.133775 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9de6c58-b9b9-479a-b22a-57e8b98286be-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-622mb\" (UID: \"c9de6c58-b9b9-479a-b22a-57e8b98286be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-622mb" Jul 15 21:10:49.135354 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:49.135335 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9de6c58-b9b9-479a-b22a-57e8b98286be-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-622mb\" (UID: \"c9de6c58-b9b9-479a-b22a-57e8b98286be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-622mb" Jul 15 21:10:49.135354 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:49.135345 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9de6c58-b9b9-479a-b22a-57e8b98286be-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-622mb\" (UID: \"c9de6c58-b9b9-479a-b22a-57e8b98286be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-622mb" Jul 15 21:10:49.144179 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:49.144149 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mv926\" (UniqueName: \"kubernetes.io/projected/c9de6c58-b9b9-479a-b22a-57e8b98286be-kube-api-access-mv926\") pod \"maas-default-gateway-openshift-default-56654dcf7-622mb\" (UID: \"c9de6c58-b9b9-479a-b22a-57e8b98286be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-622mb" Jul 15 21:10:49.145618 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:49.145596 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9de6c58-b9b9-479a-b22a-57e8b98286be-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-622mb\" (UID: \"c9de6c58-b9b9-479a-b22a-57e8b98286be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-622mb" Jul 15 21:10:49.314292 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:49.314205 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:10:49.328275 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:49.328250 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-622mb" Jul 15 21:10:49.469512 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:49.469486 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-622mb"] Jul 15 21:10:49.472148 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:10:49.472120 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9de6c58_b9b9_479a_b22a_57e8b98286be.slice/crio-8e4fa063ccfaabb7e1b68daf0989037a64c9deee16d518d796630b86c8a83698 WatchSource:0}: Error finding container 8e4fa063ccfaabb7e1b68daf0989037a64c9deee16d518d796630b86c8a83698: Status 404 returned error can't find the container with id 8e4fa063ccfaabb7e1b68daf0989037a64c9deee16d518d796630b86c8a83698 Jul 15 21:10:49.474161 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:49.474128 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:10:49.474285 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:49.474190 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:10:49.474285 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:49.474232 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:10:50.430007 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:50.429970 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-622mb" event={"ID":"c9de6c58-b9b9-479a-b22a-57e8b98286be","Type":"ContainerStarted","Data":"8af80057af526697843d5eaaf27c5f8af9986578c265e5e74f5bc6216949fbec"} Jul 15 21:10:50.430426 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:50.430025 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-622mb" event={"ID":"c9de6c58-b9b9-479a-b22a-57e8b98286be","Type":"ContainerStarted","Data":"8e4fa063ccfaabb7e1b68daf0989037a64c9deee16d518d796630b86c8a83698"} Jul 15 21:10:51.329482 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:51.329437 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-622mb" Jul 15 21:10:51.334216 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:51.334182 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-622mb" Jul 15 21:10:51.369077 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:51.369008 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-622mb" podStartSLOduration=3.368991662 podStartE2EDuration="3.368991662s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:10:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:10:50.469653122 +0000 UTC m=+555.010965675" watchObservedRunningTime="2026-07-15 21:10:51.368991662 +0000 UTC m=+555.910304211" Jul 15 21:10:51.435555 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:51.435521 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-622mb" Jul 15 21:10:51.436624 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:51.436605 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-622mb" Jul 15 21:10:51.706340 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:51.706302 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:10:51.878725 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:51.878685 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:10:52.154817 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:52.154784 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:10:52.158453 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:52.158427 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-nqhnc" Jul 15 21:10:52.163207 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:52.163184 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-22259\"" Jul 15 21:10:52.164933 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:52.164900 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:10:52.165586 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:52.165567 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 15 21:10:52.172678 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:52.172658 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:10:52.232694 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:52.232661 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:10:52.260567 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:52.260537 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1a841697-9292-45f5-8fbb-3dfb012e0eff-config-file\") pod \"limitador-limitador-54d6d988d5-nqhnc\" (UID: \"1a841697-9292-45f5-8fbb-3dfb012e0eff\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-nqhnc" Jul 15 21:10:52.260736 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:52.260612 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-md86v\" (UniqueName: \"kubernetes.io/projected/1a841697-9292-45f5-8fbb-3dfb012e0eff-kube-api-access-md86v\") pod \"limitador-limitador-54d6d988d5-nqhnc\" (UID: \"1a841697-9292-45f5-8fbb-3dfb012e0eff\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-nqhnc" Jul 15 21:10:52.361624 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:52.361580 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1a841697-9292-45f5-8fbb-3dfb012e0eff-config-file\") pod \"limitador-limitador-54d6d988d5-nqhnc\" (UID: \"1a841697-9292-45f5-8fbb-3dfb012e0eff\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-nqhnc" Jul 15 21:10:52.361799 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:52.361678 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-md86v\" (UniqueName: \"kubernetes.io/projected/1a841697-9292-45f5-8fbb-3dfb012e0eff-kube-api-access-md86v\") pod \"limitador-limitador-54d6d988d5-nqhnc\" (UID: \"1a841697-9292-45f5-8fbb-3dfb012e0eff\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-nqhnc" Jul 15 21:10:52.362261 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:52.362241 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1a841697-9292-45f5-8fbb-3dfb012e0eff-config-file\") pod \"limitador-limitador-54d6d988d5-nqhnc\" (UID: \"1a841697-9292-45f5-8fbb-3dfb012e0eff\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-nqhnc" Jul 15 21:10:52.375010 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:52.374984 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-md86v\" (UniqueName: \"kubernetes.io/projected/1a841697-9292-45f5-8fbb-3dfb012e0eff-kube-api-access-md86v\") pod \"limitador-limitador-54d6d988d5-nqhnc\" (UID: \"1a841697-9292-45f5-8fbb-3dfb012e0eff\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-nqhnc" Jul 15 21:10:52.469984 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:52.469903 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-nqhnc" Jul 15 21:10:52.633300 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:10:52.633258 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a841697_9292_45f5_8fbb_3dfb012e0eff.slice/crio-4ad4686084f7776ae318c86968230a0d3fbcacf7b154c87a0896bec0289731d6 WatchSource:0}: Error finding container 4ad4686084f7776ae318c86968230a0d3fbcacf7b154c87a0896bec0289731d6: Status 404 returned error can't find the container with id 4ad4686084f7776ae318c86968230a0d3fbcacf7b154c87a0896bec0289731d6 Jul 15 21:10:52.633460 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:52.633380 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:10:52.854582 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:52.854492 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:10:53.024197 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:53.024163 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-mmgmb"] Jul 15 21:10:53.027842 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:53.027814 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-mmgmb" Jul 15 21:10:53.035913 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:53.035880 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-jjhd7\"" Jul 15 21:10:53.046683 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:53.046655 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-mmgmb"] Jul 15 21:10:53.068227 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:53.068192 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlbpn\" (UniqueName: \"kubernetes.io/projected/699485ab-4d79-497a-ae7b-27e3bd042307-kube-api-access-mlbpn\") pod \"authorino-75d556bd77-mmgmb\" (UID: \"699485ab-4d79-497a-ae7b-27e3bd042307\") " pod="kuadrant-system/authorino-75d556bd77-mmgmb" Jul 15 21:10:53.115754 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:53.115717 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:10:53.170518 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:53.170480 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlbpn\" (UniqueName: \"kubernetes.io/projected/699485ab-4d79-497a-ae7b-27e3bd042307-kube-api-access-mlbpn\") pod \"authorino-75d556bd77-mmgmb\" (UID: \"699485ab-4d79-497a-ae7b-27e3bd042307\") " pod="kuadrant-system/authorino-75d556bd77-mmgmb" Jul 15 21:10:53.185851 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:53.185493 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlbpn\" (UniqueName: \"kubernetes.io/projected/699485ab-4d79-497a-ae7b-27e3bd042307-kube-api-access-mlbpn\") pod \"authorino-75d556bd77-mmgmb\" (UID: \"699485ab-4d79-497a-ae7b-27e3bd042307\") " pod="kuadrant-system/authorino-75d556bd77-mmgmb" Jul 15 21:10:53.338539 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:53.338495 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-mmgmb" Jul 15 21:10:53.449867 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:53.449729 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-nqhnc" event={"ID":"1a841697-9292-45f5-8fbb-3dfb012e0eff","Type":"ContainerStarted","Data":"4ad4686084f7776ae318c86968230a0d3fbcacf7b154c87a0896bec0289731d6"} Jul 15 21:10:53.514843 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:53.514721 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-mmgmb"] Jul 15 21:10:53.515314 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:10:53.515003 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod699485ab_4d79_497a_ae7b_27e3bd042307.slice/crio-cdf2876a94aba93f3dff80de98f4cb5abc64dde2602443a40a92b64206445558 WatchSource:0}: Error finding container cdf2876a94aba93f3dff80de98f4cb5abc64dde2602443a40a92b64206445558: Status 404 returned error can't find the container with id cdf2876a94aba93f3dff80de98f4cb5abc64dde2602443a40a92b64206445558 Jul 15 21:10:54.460567 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:54.460528 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-mmgmb" event={"ID":"699485ab-4d79-497a-ae7b-27e3bd042307","Type":"ContainerStarted","Data":"cdf2876a94aba93f3dff80de98f4cb5abc64dde2602443a40a92b64206445558"} Jul 15 21:10:55.470010 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:55.469968 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-nqhnc" event={"ID":"1a841697-9292-45f5-8fbb-3dfb012e0eff","Type":"ContainerStarted","Data":"e4eddade8322642dfdb9d05cf67f99f2a3feb8f69e78e1b6bdc2e175731bfa95"} Jul 15 21:10:55.470491 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:55.470124 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-nqhnc" Jul 15 21:10:55.496403 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:55.496238 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-nqhnc" podStartSLOduration=0.859440058 podStartE2EDuration="3.496216793s" totalImagesPullingTime="2.636776735s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:10:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:10:55.492583847 +0000 UTC m=+560.033896404" watchObservedRunningTime="2026-07-15 21:10:55.496216793 +0000 UTC m=+560.037529345" Jul 15 21:10:58.487031 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:58.486997 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-mmgmb" event={"ID":"699485ab-4d79-497a-ae7b-27e3bd042307","Type":"ContainerStarted","Data":"1b88f6ac3fd609f65b0822711756fea8bd8b0a5f859a3affc4eb9c0b2c813697"} Jul 15 21:10:58.506024 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:10:58.505970 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-mmgmb" podStartSLOduration=2.571582306 podStartE2EDuration="6.505954828s" totalImagesPullingTime="3.934372522s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:10:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:10:58.503942939 +0000 UTC m=+563.045255490" watchObservedRunningTime="2026-07-15 21:10:58.505954828 +0000 UTC m=+563.047267425" Jul 15 21:11:06.476812 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:11:06.476778 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-nqhnc" Jul 15 21:11:07.803857 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:11:07.803815 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:11:07.992286 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:11:07.992253 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:11:08.394078 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:11:08.394027 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:11:13.831343 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:11:13.831308 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:11:13.889470 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:11:13.889433 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:11:20.974854 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:11:20.974813 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:11:21.489516 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:11:21.489479 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:11:21.673652 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:11:21.673614 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5fcd48568d-wlklq"] Jul 15 21:11:21.679498 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:11:21.679474 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5fcd48568d-wlklq" Jul 15 21:11:21.682384 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:11:21.682355 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 15 21:11:21.684249 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:11:21.684220 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5fcd48568d-wlklq"] Jul 15 21:11:21.831910 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:11:21.831823 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dshdx\" (UniqueName: \"kubernetes.io/projected/cd06ec24-990c-434d-95b9-935802859152-kube-api-access-dshdx\") pod \"authorino-5fcd48568d-wlklq\" (UID: \"cd06ec24-990c-434d-95b9-935802859152\") " pod="kuadrant-system/authorino-5fcd48568d-wlklq" Jul 15 21:11:21.831910 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:11:21.831868 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/cd06ec24-990c-434d-95b9-935802859152-tls-cert\") pod \"authorino-5fcd48568d-wlklq\" (UID: \"cd06ec24-990c-434d-95b9-935802859152\") " pod="kuadrant-system/authorino-5fcd48568d-wlklq" Jul 15 21:11:21.932979 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:11:21.932944 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dshdx\" (UniqueName: \"kubernetes.io/projected/cd06ec24-990c-434d-95b9-935802859152-kube-api-access-dshdx\") pod \"authorino-5fcd48568d-wlklq\" (UID: \"cd06ec24-990c-434d-95b9-935802859152\") " pod="kuadrant-system/authorino-5fcd48568d-wlklq" Jul 15 21:11:21.933189 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:11:21.932994 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/cd06ec24-990c-434d-95b9-935802859152-tls-cert\") pod \"authorino-5fcd48568d-wlklq\" (UID: \"cd06ec24-990c-434d-95b9-935802859152\") " pod="kuadrant-system/authorino-5fcd48568d-wlklq" Jul 15 21:11:21.935424 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:11:21.935403 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/cd06ec24-990c-434d-95b9-935802859152-tls-cert\") pod \"authorino-5fcd48568d-wlklq\" (UID: \"cd06ec24-990c-434d-95b9-935802859152\") " pod="kuadrant-system/authorino-5fcd48568d-wlklq" Jul 15 21:11:21.942245 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:11:21.942216 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dshdx\" (UniqueName: \"kubernetes.io/projected/cd06ec24-990c-434d-95b9-935802859152-kube-api-access-dshdx\") pod \"authorino-5fcd48568d-wlklq\" (UID: \"cd06ec24-990c-434d-95b9-935802859152\") " pod="kuadrant-system/authorino-5fcd48568d-wlklq" Jul 15 21:11:21.990135 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:11:21.990088 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5fcd48568d-wlklq" Jul 15 21:11:22.128821 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:11:22.128787 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5fcd48568d-wlklq"] Jul 15 21:11:22.130546 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:11:22.130519 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcd06ec24_990c_434d_95b9_935802859152.slice/crio-4fd68dbd4eccebc6d93346cc2a7ab9133562144aa97e067d94b75a605642ccf4 WatchSource:0}: Error finding container 4fd68dbd4eccebc6d93346cc2a7ab9133562144aa97e067d94b75a605642ccf4: Status 404 returned error can't find the container with id 4fd68dbd4eccebc6d93346cc2a7ab9133562144aa97e067d94b75a605642ccf4 Jul 15 21:11:22.612290 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:11:22.612257 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5fcd48568d-wlklq" event={"ID":"cd06ec24-990c-434d-95b9-935802859152","Type":"ContainerStarted","Data":"a7958f7d1f1d15e9c05bc508311ad4ae1cb65d698e35cd8b0971dc1ab45924d9"} Jul 15 21:11:22.612290 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:11:22.612301 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5fcd48568d-wlklq" event={"ID":"cd06ec24-990c-434d-95b9-935802859152","Type":"ContainerStarted","Data":"4fd68dbd4eccebc6d93346cc2a7ab9133562144aa97e067d94b75a605642ccf4"} Jul 15 21:11:22.632901 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:11:22.632849 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5fcd48568d-wlklq" podStartSLOduration=1.23126304 podStartE2EDuration="1.632832246s" totalImagesPullingTime="401.569206ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:11:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:11:22.6299633 +0000 UTC m=+587.171275852" watchObservedRunningTime="2026-07-15 21:11:22.632832246 +0000 UTC m=+587.174144837" Jul 15 21:11:22.665144 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:11:22.665106 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-mmgmb"] Jul 15 21:11:22.665580 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:11:22.665538 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-mmgmb" podUID="699485ab-4d79-497a-ae7b-27e3bd042307" containerName="authorino" containerID="cri-o://1b88f6ac3fd609f65b0822711756fea8bd8b0a5f859a3affc4eb9c0b2c813697" gracePeriod=30 Jul 15 21:11:22.919898 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:11:22.919869 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-mmgmb" Jul 15 21:11:23.041604 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:11:23.041512 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mlbpn\" (UniqueName: \"kubernetes.io/projected/699485ab-4d79-497a-ae7b-27e3bd042307-kube-api-access-mlbpn\") pod \"699485ab-4d79-497a-ae7b-27e3bd042307\" (UID: \"699485ab-4d79-497a-ae7b-27e3bd042307\") " Jul 15 21:11:23.043587 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:11:23.043562 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/699485ab-4d79-497a-ae7b-27e3bd042307-kube-api-access-mlbpn" (OuterVolumeSpecName: "kube-api-access-mlbpn") pod "699485ab-4d79-497a-ae7b-27e3bd042307" (UID: "699485ab-4d79-497a-ae7b-27e3bd042307"). InnerVolumeSpecName "kube-api-access-mlbpn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:11:23.142286 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:11:23.142248 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mlbpn\" (UniqueName: \"kubernetes.io/projected/699485ab-4d79-497a-ae7b-27e3bd042307-kube-api-access-mlbpn\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:11:23.619563 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:11:23.619528 2579 generic.go:356] "Generic (PLEG): container finished" podID="699485ab-4d79-497a-ae7b-27e3bd042307" containerID="1b88f6ac3fd609f65b0822711756fea8bd8b0a5f859a3affc4eb9c0b2c813697" exitCode=0 Jul 15 21:11:23.619563 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:11:23.619565 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-mmgmb" Jul 15 21:11:23.619813 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:11:23.619609 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-mmgmb" event={"ID":"699485ab-4d79-497a-ae7b-27e3bd042307","Type":"ContainerDied","Data":"1b88f6ac3fd609f65b0822711756fea8bd8b0a5f859a3affc4eb9c0b2c813697"} Jul 15 21:11:23.619813 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:11:23.619649 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-mmgmb" event={"ID":"699485ab-4d79-497a-ae7b-27e3bd042307","Type":"ContainerDied","Data":"cdf2876a94aba93f3dff80de98f4cb5abc64dde2602443a40a92b64206445558"} Jul 15 21:11:23.619813 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:11:23.619666 2579 scope.go:117] "RemoveContainer" containerID="1b88f6ac3fd609f65b0822711756fea8bd8b0a5f859a3affc4eb9c0b2c813697" Jul 15 21:11:23.628631 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:11:23.628609 2579 scope.go:117] "RemoveContainer" containerID="1b88f6ac3fd609f65b0822711756fea8bd8b0a5f859a3affc4eb9c0b2c813697" Jul 15 21:11:23.628871 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:11:23.628853 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b88f6ac3fd609f65b0822711756fea8bd8b0a5f859a3affc4eb9c0b2c813697\": container with ID starting with 1b88f6ac3fd609f65b0822711756fea8bd8b0a5f859a3affc4eb9c0b2c813697 not found: ID does not exist" containerID="1b88f6ac3fd609f65b0822711756fea8bd8b0a5f859a3affc4eb9c0b2c813697" Jul 15 21:11:23.628918 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:11:23.628880 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b88f6ac3fd609f65b0822711756fea8bd8b0a5f859a3affc4eb9c0b2c813697"} err="failed to get container status \"1b88f6ac3fd609f65b0822711756fea8bd8b0a5f859a3affc4eb9c0b2c813697\": rpc error: code = NotFound desc = could not find container \"1b88f6ac3fd609f65b0822711756fea8bd8b0a5f859a3affc4eb9c0b2c813697\": container with ID starting with 1b88f6ac3fd609f65b0822711756fea8bd8b0a5f859a3affc4eb9c0b2c813697 not found: ID does not exist" Jul 15 21:11:23.641897 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:11:23.641871 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-mmgmb"] Jul 15 21:11:23.648937 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:11:23.648916 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-mmgmb"] Jul 15 21:11:23.997545 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:11:23.997464 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="699485ab-4d79-497a-ae7b-27e3bd042307" path="/var/lib/kubelet/pods/699485ab-4d79-497a-ae7b-27e3bd042307/volumes" Jul 15 21:11:26.233109 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:11:26.233050 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:11:27.345960 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:11:27.345918 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:11:27.445568 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:11:27.445530 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:11:27.549593 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:11:27.549549 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:11:27.752238 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:11:27.752205 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:11:30.742900 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:11:30.742858 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:11:30.872760 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:11:30.872722 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:11:35.939584 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:11:35.939551 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m5r2m_0db09c1e-d24d-4519-93c8-c185018c3d24/ovn-acl-logging/0.log" Jul 15 21:11:35.943828 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:11:35.943802 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m5r2m_0db09c1e-d24d-4519-93c8-c185018c3d24/ovn-acl-logging/0.log" Jul 15 21:12:01.600565 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:12:01.600474 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:12:01.698554 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:12:01.698514 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:12:02.288895 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:12:02.288838 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:12:02.436266 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:12:02.436228 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:12:02.735298 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:12:02.735268 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:12:02.900167 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:12:02.900133 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:12:03.004765 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:12:03.004682 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:12:05.360548 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:12:05.360512 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:12:05.461176 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:12:05.461139 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:12:10.874226 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:12:10.874189 2579 generic.go:356] "Generic (PLEG): container finished" podID="c9de6c58-b9b9-479a-b22a-57e8b98286be" containerID="8af80057af526697843d5eaaf27c5f8af9986578c265e5e74f5bc6216949fbec" exitCode=0 Jul 15 21:12:10.874653 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:12:10.874267 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-622mb" event={"ID":"c9de6c58-b9b9-479a-b22a-57e8b98286be","Type":"ContainerDied","Data":"8af80057af526697843d5eaaf27c5f8af9986578c265e5e74f5bc6216949fbec"} Jul 15 21:12:10.874716 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:12:10.874658 2579 scope.go:117] "RemoveContainer" containerID="8af80057af526697843d5eaaf27c5f8af9986578c265e5e74f5bc6216949fbec" Jul 15 21:12:10.875649 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:12:10.875625 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:12:10.875715 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:12:10.875693 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:12:10.875752 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:12:10.875726 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:12:11.882199 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:12:11.882156 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-622mb" event={"ID":"c9de6c58-b9b9-479a-b22a-57e8b98286be","Type":"ContainerStarted","Data":"21809ff71786b54d68a72c858ddfcf60e75abf99c9e31aaf1a192cafcbe2822c"} Jul 15 21:12:12.329675 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:12:12.329575 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-622mb" Jul 15 21:12:12.330081 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:12:12.329964 2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-622mb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 15 21:12:12.330276 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:12:12.330251 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-622mb" podUID="c9de6c58-b9b9-479a-b22a-57e8b98286be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 15 21:12:12.845134 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:12:12.845083 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:12:12.938838 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:12:12.938796 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:12:13.329561 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:12:13.329518 2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-622mb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 15 21:12:13.329776 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:12:13.329590 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-622mb" podUID="c9de6c58-b9b9-479a-b22a-57e8b98286be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 15 21:12:14.190422 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:12:14.190381 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:12:14.292377 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:12:14.292332 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:12:14.329572 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:12:14.329526 2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-622mb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 15 21:12:14.329771 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:12:14.329597 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-622mb" podUID="c9de6c58-b9b9-479a-b22a-57e8b98286be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 15 21:12:15.328923 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:12:15.328880 2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-622mb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 15 21:12:15.329560 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:12:15.329522 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-622mb" podUID="c9de6c58-b9b9-479a-b22a-57e8b98286be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 15 21:12:16.332877 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:12:16.332847 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-622mb" Jul 15 21:12:16.333293 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:12:16.333107 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-622mb" Jul 15 21:12:16.333892 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:12:16.333875 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-622mb" Jul 15 21:12:26.834265 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:12:26.834227 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:12:36.894203 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:12:36.894158 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:12:37.294633 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:12:37.294549 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:12:44.029562 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:12:44.029529 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:12:44.708472 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:12:44.708436 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:13:01.329243 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:13:01.329203 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:13:02.394368 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:13:02.394328 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:13:09.260996 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:13:09.260935 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:13:10.795389 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:13:10.795352 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:13:23.587732 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:13:23.587689 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:13:25.919863 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:13:25.919827 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:13:40.632793 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:13:40.632760 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:13:43.599720 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:13:43.599672 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:14:41.511910 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:14:41.511875 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-98759bf69-rll7s"] Jul 15 21:14:41.512402 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:14:41.512292 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="699485ab-4d79-497a-ae7b-27e3bd042307" containerName="authorino" Jul 15 21:14:41.512402 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:14:41.512308 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="699485ab-4d79-497a-ae7b-27e3bd042307" containerName="authorino" Jul 15 21:14:41.512402 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:14:41.512379 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="699485ab-4d79-497a-ae7b-27e3bd042307" containerName="authorino" Jul 15 21:14:41.515322 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:14:41.515304 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-98759bf69-rll7s" Jul 15 21:14:41.526738 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:14:41.526705 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-98759bf69-rll7s"] Jul 15 21:14:41.638133 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:14:41.638098 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/57610c56-136e-473b-9609-fc51ce6729dc-tls-cert\") pod \"authorino-98759bf69-rll7s\" (UID: \"57610c56-136e-473b-9609-fc51ce6729dc\") " pod="kuadrant-system/authorino-98759bf69-rll7s" Jul 15 21:14:41.638324 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:14:41.638149 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w78d8\" (UniqueName: \"kubernetes.io/projected/57610c56-136e-473b-9609-fc51ce6729dc-kube-api-access-w78d8\") pod \"authorino-98759bf69-rll7s\" (UID: \"57610c56-136e-473b-9609-fc51ce6729dc\") " pod="kuadrant-system/authorino-98759bf69-rll7s" Jul 15 21:14:41.738982 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:14:41.738945 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w78d8\" (UniqueName: \"kubernetes.io/projected/57610c56-136e-473b-9609-fc51ce6729dc-kube-api-access-w78d8\") pod \"authorino-98759bf69-rll7s\" (UID: \"57610c56-136e-473b-9609-fc51ce6729dc\") " pod="kuadrant-system/authorino-98759bf69-rll7s" Jul 15 21:14:41.739188 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:14:41.739046 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/57610c56-136e-473b-9609-fc51ce6729dc-tls-cert\") pod \"authorino-98759bf69-rll7s\" (UID: \"57610c56-136e-473b-9609-fc51ce6729dc\") " pod="kuadrant-system/authorino-98759bf69-rll7s" Jul 15 21:14:41.741607 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:14:41.741572 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/57610c56-136e-473b-9609-fc51ce6729dc-tls-cert\") pod \"authorino-98759bf69-rll7s\" (UID: \"57610c56-136e-473b-9609-fc51ce6729dc\") " pod="kuadrant-system/authorino-98759bf69-rll7s" Jul 15 21:14:41.754503 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:14:41.754468 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w78d8\" (UniqueName: \"kubernetes.io/projected/57610c56-136e-473b-9609-fc51ce6729dc-kube-api-access-w78d8\") pod \"authorino-98759bf69-rll7s\" (UID: \"57610c56-136e-473b-9609-fc51ce6729dc\") " pod="kuadrant-system/authorino-98759bf69-rll7s" Jul 15 21:14:41.817831 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:14:41.817736 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:14:41.824503 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:14:41.824475 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-98759bf69-rll7s" Jul 15 21:14:41.953806 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:14:41.953773 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:14:41.960173 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:14:41.960143 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-98759bf69-rll7s"] Jul 15 21:14:41.961495 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:14:41.961469 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod57610c56_136e_473b_9609_fc51ce6729dc.slice/crio-75fa99d7c88f47753c50cf9d0f3f81649d9ed531aaa96e2477c1cc651695620c WatchSource:0}: Error finding container 75fa99d7c88f47753c50cf9d0f3f81649d9ed531aaa96e2477c1cc651695620c: Status 404 returned error can't find the container with id 75fa99d7c88f47753c50cf9d0f3f81649d9ed531aaa96e2477c1cc651695620c Jul 15 21:14:41.962901 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:14:41.962881 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 21:14:42.667215 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:14:42.667177 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-98759bf69-rll7s" event={"ID":"57610c56-136e-473b-9609-fc51ce6729dc","Type":"ContainerStarted","Data":"73a245bf2a45b16ee131d89c281b772eb1d15ae4dbc1a0ce66e9692622a7961f"} Jul 15 21:14:42.667635 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:14:42.667227 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-98759bf69-rll7s" event={"ID":"57610c56-136e-473b-9609-fc51ce6729dc","Type":"ContainerStarted","Data":"75fa99d7c88f47753c50cf9d0f3f81649d9ed531aaa96e2477c1cc651695620c"} Jul 15 21:14:42.689406 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:14:42.689351 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-98759bf69-rll7s" podStartSLOduration=1.148134279 podStartE2EDuration="1.689334231s" totalImagesPullingTime="541.199952ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:14:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:14:42.68622466 +0000 UTC m=+787.227537208" watchObservedRunningTime="2026-07-15 21:14:42.689334231 +0000 UTC m=+787.230646781" Jul 15 21:14:42.730625 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:14:42.730584 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5fcd48568d-wlklq"] Jul 15 21:14:42.730839 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:14:42.730815 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5fcd48568d-wlklq" podUID="cd06ec24-990c-434d-95b9-935802859152" containerName="authorino" containerID="cri-o://a7958f7d1f1d15e9c05bc508311ad4ae1cb65d698e35cd8b0971dc1ab45924d9" gracePeriod=30 Jul 15 21:14:42.996774 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:14:42.996743 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5fcd48568d-wlklq" Jul 15 21:14:43.044796 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:14:43.044698 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:14:43.051019 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:14:43.050985 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/cd06ec24-990c-434d-95b9-935802859152-tls-cert\") pod \"cd06ec24-990c-434d-95b9-935802859152\" (UID: \"cd06ec24-990c-434d-95b9-935802859152\") " Jul 15 21:14:43.051241 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:14:43.051029 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dshdx\" (UniqueName: \"kubernetes.io/projected/cd06ec24-990c-434d-95b9-935802859152-kube-api-access-dshdx\") pod \"cd06ec24-990c-434d-95b9-935802859152\" (UID: \"cd06ec24-990c-434d-95b9-935802859152\") " Jul 15 21:14:43.053223 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:14:43.053192 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd06ec24-990c-434d-95b9-935802859152-kube-api-access-dshdx" (OuterVolumeSpecName: "kube-api-access-dshdx") pod "cd06ec24-990c-434d-95b9-935802859152" (UID: "cd06ec24-990c-434d-95b9-935802859152"). InnerVolumeSpecName "kube-api-access-dshdx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:14:43.061494 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:14:43.061467 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd06ec24-990c-434d-95b9-935802859152-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "cd06ec24-990c-434d-95b9-935802859152" (UID: "cd06ec24-990c-434d-95b9-935802859152"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 21:14:43.150304 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:14:43.150264 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:14:43.152551 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:14:43.152524 2579 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/cd06ec24-990c-434d-95b9-935802859152-tls-cert\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:14:43.152683 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:14:43.152554 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dshdx\" (UniqueName: \"kubernetes.io/projected/cd06ec24-990c-434d-95b9-935802859152-kube-api-access-dshdx\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:14:43.674767 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:14:43.674734 2579 generic.go:356] "Generic (PLEG): container finished" podID="cd06ec24-990c-434d-95b9-935802859152" containerID="a7958f7d1f1d15e9c05bc508311ad4ae1cb65d698e35cd8b0971dc1ab45924d9" exitCode=0 Jul 15 21:14:43.675246 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:14:43.674778 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5fcd48568d-wlklq" Jul 15 21:14:43.675246 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:14:43.674813 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5fcd48568d-wlklq" event={"ID":"cd06ec24-990c-434d-95b9-935802859152","Type":"ContainerDied","Data":"a7958f7d1f1d15e9c05bc508311ad4ae1cb65d698e35cd8b0971dc1ab45924d9"} Jul 15 21:14:43.675246 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:14:43.674845 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5fcd48568d-wlklq" event={"ID":"cd06ec24-990c-434d-95b9-935802859152","Type":"ContainerDied","Data":"4fd68dbd4eccebc6d93346cc2a7ab9133562144aa97e067d94b75a605642ccf4"} Jul 15 21:14:43.675246 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:14:43.674861 2579 scope.go:117] "RemoveContainer" containerID="a7958f7d1f1d15e9c05bc508311ad4ae1cb65d698e35cd8b0971dc1ab45924d9" Jul 15 21:14:43.684518 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:14:43.684498 2579 scope.go:117] "RemoveContainer" containerID="a7958f7d1f1d15e9c05bc508311ad4ae1cb65d698e35cd8b0971dc1ab45924d9" Jul 15 21:14:43.684842 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:14:43.684820 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7958f7d1f1d15e9c05bc508311ad4ae1cb65d698e35cd8b0971dc1ab45924d9\": container with ID starting with a7958f7d1f1d15e9c05bc508311ad4ae1cb65d698e35cd8b0971dc1ab45924d9 not found: ID does not exist" containerID="a7958f7d1f1d15e9c05bc508311ad4ae1cb65d698e35cd8b0971dc1ab45924d9" Jul 15 21:14:43.684920 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:14:43.684855 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7958f7d1f1d15e9c05bc508311ad4ae1cb65d698e35cd8b0971dc1ab45924d9"} err="failed to get container status \"a7958f7d1f1d15e9c05bc508311ad4ae1cb65d698e35cd8b0971dc1ab45924d9\": rpc error: code = NotFound desc = could not find container \"a7958f7d1f1d15e9c05bc508311ad4ae1cb65d698e35cd8b0971dc1ab45924d9\": container with ID starting with a7958f7d1f1d15e9c05bc508311ad4ae1cb65d698e35cd8b0971dc1ab45924d9 not found: ID does not exist" Jul 15 21:14:43.730118 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:14:43.730086 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5fcd48568d-wlklq"] Jul 15 21:14:43.736340 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:14:43.736309 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5fcd48568d-wlklq"] Jul 15 21:14:43.998113 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:14:43.998016 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd06ec24-990c-434d-95b9-935802859152" path="/var/lib/kubelet/pods/cd06ec24-990c-434d-95b9-935802859152/volumes" Jul 15 21:15:03.305133 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:15:03.305040 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:15:12.386963 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:15:12.386926 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:15:23.289966 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:15:23.289924 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:15:34.492523 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:15:34.492483 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:15:42.784600 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:15:42.784564 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:15:53.984937 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:15:53.984901 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:15:55.963809 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:15:55.963771 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:15:57.663966 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:15:57.663920 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:16:19.987811 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:16:19.987775 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:16:33.997055 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:16:33.997021 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:16:35.976481 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:16:35.976452 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m5r2m_0db09c1e-d24d-4519-93c8-c185018c3d24/ovn-acl-logging/0.log" Jul 15 21:16:35.977789 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:16:35.977765 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m5r2m_0db09c1e-d24d-4519-93c8-c185018c3d24/ovn-acl-logging/0.log" Jul 15 21:16:38.528403 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:16:38.528364 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:16:38.618358 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:16:38.618322 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:16:38.802868 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:16:38.802779 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:16:45.180709 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:16:45.180671 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:16:45.285669 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:16:45.285637 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:16:45.381881 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:16:45.381847 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:16:45.537571 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:16:45.537487 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:16:51.876733 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:16:51.876687 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:16:51.938956 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:16:51.938920 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:17:08.796904 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:17:08.796866 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:17:18.985679 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:17:18.985640 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:17:29.400082 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:17:29.400036 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:17:36.894566 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:17:36.894524 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:17:48.390468 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:17:48.390428 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:17:59.500703 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:17:59.500620 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:18:10.687787 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:18:10.687750 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:18:20.683829 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:18:20.683781 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:18:43.287477 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:18:43.287438 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:18:43.942872 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:18:43.942839 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:18:45.083398 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:18:45.083362 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:18:52.880166 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:18:52.880122 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:19:15.703769 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:19:15.703731 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:19:27.792988 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:19:27.792949 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:19:44.792774 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:19:44.792736 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:19:55.186148 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:19:55.186112 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:19:58.117397 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:19:58.117358 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:20:03.404890 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:20:03.404850 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:20:05.298908 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:20:05.298872 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:20:14.184553 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:20:14.184520 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:20:14.290462 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:20:14.290427 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:20:15.301868 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:20:15.301820 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:20:25.189799 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:20:25.189763 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:20:50.220411 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:20:50.220375 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vbm4v"] Jul 15 21:20:50.220895 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:20:50.220622 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vbm4v" podUID="a7538513-8059-4dc3-b3c5-41fcbab87240" containerName="manager" containerID="cri-o://21e1f5d751ae65106f2a37b3075c8bd6bba4714b96d240f3c7bca91c6382484c" gracePeriod=10 Jul 15 21:20:52.273615 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:20:52.273585 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vbm4v" Jul 15 21:20:52.369436 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:20:52.369402 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d7vzh\" (UniqueName: \"kubernetes.io/projected/a7538513-8059-4dc3-b3c5-41fcbab87240-kube-api-access-d7vzh\") pod \"a7538513-8059-4dc3-b3c5-41fcbab87240\" (UID: \"a7538513-8059-4dc3-b3c5-41fcbab87240\") " Jul 15 21:20:52.369636 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:20:52.369495 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a7538513-8059-4dc3-b3c5-41fcbab87240-extensions-socket-volume\") pod \"a7538513-8059-4dc3-b3c5-41fcbab87240\" (UID: \"a7538513-8059-4dc3-b3c5-41fcbab87240\") " Jul 15 21:20:52.369848 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:20:52.369825 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7538513-8059-4dc3-b3c5-41fcbab87240-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a7538513-8059-4dc3-b3c5-41fcbab87240" (UID: "a7538513-8059-4dc3-b3c5-41fcbab87240"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:20:52.371394 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:20:52.371368 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7538513-8059-4dc3-b3c5-41fcbab87240-kube-api-access-d7vzh" (OuterVolumeSpecName: "kube-api-access-d7vzh") pod "a7538513-8059-4dc3-b3c5-41fcbab87240" (UID: "a7538513-8059-4dc3-b3c5-41fcbab87240"). InnerVolumeSpecName "kube-api-access-d7vzh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:20:52.470439 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:20:52.470401 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d7vzh\" (UniqueName: \"kubernetes.io/projected/a7538513-8059-4dc3-b3c5-41fcbab87240-kube-api-access-d7vzh\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:20:52.470439 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:20:52.470433 2579 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a7538513-8059-4dc3-b3c5-41fcbab87240-extensions-socket-volume\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:20:52.676772 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:20:52.676676 2579 generic.go:356] "Generic (PLEG): container finished" podID="a7538513-8059-4dc3-b3c5-41fcbab87240" containerID="21e1f5d751ae65106f2a37b3075c8bd6bba4714b96d240f3c7bca91c6382484c" exitCode=0 Jul 15 21:20:52.676772 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:20:52.676720 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vbm4v" Jul 15 21:20:52.676979 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:20:52.676762 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vbm4v" event={"ID":"a7538513-8059-4dc3-b3c5-41fcbab87240","Type":"ContainerDied","Data":"21e1f5d751ae65106f2a37b3075c8bd6bba4714b96d240f3c7bca91c6382484c"} Jul 15 21:20:52.676979 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:20:52.676818 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vbm4v" event={"ID":"a7538513-8059-4dc3-b3c5-41fcbab87240","Type":"ContainerDied","Data":"5fbc6e997cf5e20e7c078b535a08bd469fd933e2302dbb4f4757e84df3039623"} Jul 15 21:20:52.676979 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:20:52.676847 2579 scope.go:117] "RemoveContainer" containerID="21e1f5d751ae65106f2a37b3075c8bd6bba4714b96d240f3c7bca91c6382484c" Jul 15 21:20:52.686912 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:20:52.686897 2579 scope.go:117] "RemoveContainer" containerID="21e1f5d751ae65106f2a37b3075c8bd6bba4714b96d240f3c7bca91c6382484c" Jul 15 21:20:52.687176 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:20:52.687160 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21e1f5d751ae65106f2a37b3075c8bd6bba4714b96d240f3c7bca91c6382484c\": container with ID starting with 21e1f5d751ae65106f2a37b3075c8bd6bba4714b96d240f3c7bca91c6382484c not found: ID does not exist" containerID="21e1f5d751ae65106f2a37b3075c8bd6bba4714b96d240f3c7bca91c6382484c" Jul 15 21:20:52.687239 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:20:52.687183 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21e1f5d751ae65106f2a37b3075c8bd6bba4714b96d240f3c7bca91c6382484c"} err="failed to get container status \"21e1f5d751ae65106f2a37b3075c8bd6bba4714b96d240f3c7bca91c6382484c\": rpc error: code = NotFound desc = could not find container \"21e1f5d751ae65106f2a37b3075c8bd6bba4714b96d240f3c7bca91c6382484c\": container with ID starting with 21e1f5d751ae65106f2a37b3075c8bd6bba4714b96d240f3c7bca91c6382484c not found: ID does not exist" Jul 15 21:20:52.702719 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:20:52.702689 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vbm4v"] Jul 15 21:20:52.709079 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:20:52.709039 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vbm4v"] Jul 15 21:20:53.997412 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:20:53.997374 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7538513-8059-4dc3-b3c5-41fcbab87240" path="/var/lib/kubelet/pods/a7538513-8059-4dc3-b3c5-41fcbab87240/volumes" Jul 15 21:21:36.010496 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:21:36.010464 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m5r2m_0db09c1e-d24d-4519-93c8-c185018c3d24/ovn-acl-logging/0.log" Jul 15 21:21:36.011092 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:21:36.010671 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m5r2m_0db09c1e-d24d-4519-93c8-c185018c3d24/ovn-acl-logging/0.log" Jul 15 21:21:51.366122 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:21:51.366052 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t8w9q"] Jul 15 21:21:51.366531 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:21:51.366431 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd06ec24-990c-434d-95b9-935802859152" containerName="authorino" Jul 15 21:21:51.366531 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:21:51.366442 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd06ec24-990c-434d-95b9-935802859152" containerName="authorino" Jul 15 21:21:51.366531 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:21:51.366452 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7538513-8059-4dc3-b3c5-41fcbab87240" containerName="manager" Jul 15 21:21:51.366531 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:21:51.366458 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7538513-8059-4dc3-b3c5-41fcbab87240" containerName="manager" Jul 15 21:21:51.366531 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:21:51.366512 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd06ec24-990c-434d-95b9-935802859152" containerName="authorino" Jul 15 21:21:51.366531 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:21:51.366520 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7538513-8059-4dc3-b3c5-41fcbab87240" containerName="manager" Jul 15 21:21:51.370627 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:21:51.370608 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t8w9q" Jul 15 21:21:51.373921 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:21:51.373902 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-jn62q\"" Jul 15 21:21:51.390445 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:21:51.390418 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t8w9q"] Jul 15 21:21:51.467818 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:21:51.467777 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ee17d5c5-5655-460e-a148-0e4b4e95ee2a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-t8w9q\" (UID: \"ee17d5c5-5655-460e-a148-0e4b4e95ee2a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t8w9q" Jul 15 21:21:51.468008 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:21:51.467892 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnkl8\" (UniqueName: \"kubernetes.io/projected/ee17d5c5-5655-460e-a148-0e4b4e95ee2a-kube-api-access-tnkl8\") pod \"kuadrant-operator-controller-manager-86878c769f-t8w9q\" (UID: \"ee17d5c5-5655-460e-a148-0e4b4e95ee2a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t8w9q" Jul 15 21:21:51.569113 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:21:51.569050 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnkl8\" (UniqueName: \"kubernetes.io/projected/ee17d5c5-5655-460e-a148-0e4b4e95ee2a-kube-api-access-tnkl8\") pod \"kuadrant-operator-controller-manager-86878c769f-t8w9q\" (UID: \"ee17d5c5-5655-460e-a148-0e4b4e95ee2a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t8w9q" Jul 15 21:21:51.569330 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:21:51.569130 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ee17d5c5-5655-460e-a148-0e4b4e95ee2a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-t8w9q\" (UID: \"ee17d5c5-5655-460e-a148-0e4b4e95ee2a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t8w9q" Jul 15 21:21:51.569560 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:21:51.569537 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ee17d5c5-5655-460e-a148-0e4b4e95ee2a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-t8w9q\" (UID: \"ee17d5c5-5655-460e-a148-0e4b4e95ee2a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t8w9q" Jul 15 21:21:51.579295 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:21:51.579264 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnkl8\" (UniqueName: \"kubernetes.io/projected/ee17d5c5-5655-460e-a148-0e4b4e95ee2a-kube-api-access-tnkl8\") pod \"kuadrant-operator-controller-manager-86878c769f-t8w9q\" (UID: \"ee17d5c5-5655-460e-a148-0e4b4e95ee2a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t8w9q" Jul 15 21:21:51.680953 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:21:51.680861 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t8w9q" Jul 15 21:21:51.816899 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:21:51.816872 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t8w9q"] Jul 15 21:21:51.818795 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:21:51.818769 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee17d5c5_5655_460e_a148_0e4b4e95ee2a.slice/crio-bc5d909bccda71aecfde06e385e73a29f4f22492b6feede9bfcec94993c27af2 WatchSource:0}: Error finding container bc5d909bccda71aecfde06e385e73a29f4f22492b6feede9bfcec94993c27af2: Status 404 returned error can't find the container with id bc5d909bccda71aecfde06e385e73a29f4f22492b6feede9bfcec94993c27af2 Jul 15 21:21:51.990498 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:21:51.990390 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t8w9q" event={"ID":"ee17d5c5-5655-460e-a148-0e4b4e95ee2a","Type":"ContainerStarted","Data":"bd4b1b26df57e000cda9b65e9f97e00e8a84b03431a1176c965c796d2fcef14d"} Jul 15 21:21:51.990498 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:21:51.990453 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t8w9q" event={"ID":"ee17d5c5-5655-460e-a148-0e4b4e95ee2a","Type":"ContainerStarted","Data":"bc5d909bccda71aecfde06e385e73a29f4f22492b6feede9bfcec94993c27af2"} Jul 15 21:21:51.990738 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:21:51.990536 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t8w9q" Jul 15 21:21:52.018028 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:21:52.017978 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t8w9q" podStartSLOduration=1.017960602 podStartE2EDuration="1.017960602s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:21:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:21:52.014644996 +0000 UTC m=+1216.555957545" watchObservedRunningTime="2026-07-15 21:21:52.017960602 +0000 UTC m=+1216.559273154" Jul 15 21:22:02.998208 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:22:02.998172 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t8w9q" Jul 15 21:22:13.715667 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:22:13.715624 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:22:24.401461 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:22:24.401425 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:22:33.100870 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:22:33.100788 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:22:47.802743 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:22:47.802708 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:22:57.695829 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:22:57.695790 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:23:07.322956 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:23:07.322920 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:23:17.620285 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:23:17.620250 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:23:27.002832 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:23:27.002794 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:23:39.721343 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:23:39.721283 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:23:48.104006 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:23:48.103970 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:24:00.596387 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:24:00.596306 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:24:25.260507 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:24:25.260464 2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-07-15T21:24:15Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-15T21:24:15Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-15T21:24:15Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-15T21:24:15Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"}]}}\" for node \"ip-10-0-137-185.ec2.internal\": Patch \"https://a9c0189c76ffa414bbe64782f22ffef9-e324037f91a3c3d1.elb.us-east-1.amazonaws.com:6443/api/v1/nodes/ip-10-0-137-185.ec2.internal/status?timeout=10s\": context deadline exceeded" Jul 15 21:24:27.030499 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:24:27.030461 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:24:40.407444 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:24:40.407399 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:25:03.573340 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:03.573299 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/opendatahub-operator-controller-manager-65c67f598f-wb248" podUID="7193df13-36d0-4f6b-826b-4c876d9c2299" containerName="manager" probeResult="failure" output="Get \"http://10.132.0.20:8081/readyz\": dial tcp 10.132.0.20:8081: connect: connection refused" Jul 15 21:25:03.733112 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:03.733002 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mcldd" podUID="ccbdd9b9-e1f6-4075-b80d-9ea7827580cf" containerName="manager" probeResult="failure" output="Get \"http://10.132.0.25:8081/readyz\": dial tcp 10.132.0.25:8081: connect: connection refused" Jul 15 21:25:03.999250 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:03.999167 2579 generic.go:356] "Generic (PLEG): container finished" podID="cd1829fe-534c-44fc-b6cf-5761acfa46a0" containerID="37e519e0e4cd260d08da16d5acb9a7a27d3b7cea2535ede53350385e3d03ccb2" exitCode=1 Jul 15 21:25:03.999411 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:03.999242 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7bzzs" event={"ID":"cd1829fe-534c-44fc-b6cf-5761acfa46a0","Type":"ContainerDied","Data":"37e519e0e4cd260d08da16d5acb9a7a27d3b7cea2535ede53350385e3d03ccb2"} Jul 15 21:25:03.999620 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:03.999603 2579 scope.go:117] "RemoveContainer" containerID="37e519e0e4cd260d08da16d5acb9a7a27d3b7cea2535ede53350385e3d03ccb2" Jul 15 21:25:04.001268 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:04.001246 2579 generic.go:356] "Generic (PLEG): container finished" podID="7193df13-36d0-4f6b-826b-4c876d9c2299" containerID="a4f8b35df3b1028bb04d3f827a82449bddfdb71279fba88ac318287bcac4b930" exitCode=1 Jul 15 21:25:04.001372 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:04.001321 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65c67f598f-wb248" event={"ID":"7193df13-36d0-4f6b-826b-4c876d9c2299","Type":"ContainerDied","Data":"a4f8b35df3b1028bb04d3f827a82449bddfdb71279fba88ac318287bcac4b930"} Jul 15 21:25:04.001681 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:04.001661 2579 scope.go:117] "RemoveContainer" containerID="a4f8b35df3b1028bb04d3f827a82449bddfdb71279fba88ac318287bcac4b930" Jul 15 21:25:04.002015 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:04.001999 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 21:25:04.002898 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:04.002880 2579 generic.go:356] "Generic (PLEG): container finished" podID="c75adccd-fdbc-47a6-a744-c7020160ebb5" containerID="0d1bc41716da076f771d8b46d81c0cd7f579c2ce52d527173f6cbde423967cee" exitCode=1 Jul 15 21:25:04.002991 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:04.002964 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-2mq85" event={"ID":"c75adccd-fdbc-47a6-a744-c7020160ebb5","Type":"ContainerDied","Data":"0d1bc41716da076f771d8b46d81c0cd7f579c2ce52d527173f6cbde423967cee"} Jul 15 21:25:04.003321 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:04.003305 2579 scope.go:117] "RemoveContainer" containerID="0d1bc41716da076f771d8b46d81c0cd7f579c2ce52d527173f6cbde423967cee" Jul 15 21:25:04.004839 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:04.004818 2579 generic.go:356] "Generic (PLEG): container finished" podID="46fc8345-bb02-4935-bfa9-227aba6b8077" containerID="ee64779c84f1c3235c4b249811eab98222058861d319c3d0d5b2afab2e129dc7" exitCode=1 Jul 15 21:25:04.004945 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:04.004896 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gxtc2" event={"ID":"46fc8345-bb02-4935-bfa9-227aba6b8077","Type":"ContainerDied","Data":"ee64779c84f1c3235c4b249811eab98222058861d319c3d0d5b2afab2e129dc7"} Jul 15 21:25:04.005007 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:04.004944 2579 scope.go:117] "RemoveContainer" containerID="ec2657694d8391fad7704d004bd1b4f7aa8fff8f1a175a9ff222d3b3065d72d4" Jul 15 21:25:04.005231 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:04.005212 2579 scope.go:117] "RemoveContainer" containerID="ee64779c84f1c3235c4b249811eab98222058861d319c3d0d5b2afab2e129dc7" Jul 15 21:25:04.007362 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:04.007336 2579 generic.go:356] "Generic (PLEG): container finished" podID="ee17d5c5-5655-460e-a148-0e4b4e95ee2a" containerID="bd4b1b26df57e000cda9b65e9f97e00e8a84b03431a1176c965c796d2fcef14d" exitCode=0 Jul 15 21:25:04.007455 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:04.007372 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t8w9q" event={"ID":"ee17d5c5-5655-460e-a148-0e4b4e95ee2a","Type":"ContainerDied","Data":"bd4b1b26df57e000cda9b65e9f97e00e8a84b03431a1176c965c796d2fcef14d"} Jul 15 21:25:04.007680 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:04.007664 2579 scope.go:117] "RemoveContainer" containerID="bd4b1b26df57e000cda9b65e9f97e00e8a84b03431a1176c965c796d2fcef14d" Jul 15 21:25:04.011376 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:04.010932 2579 generic.go:356] "Generic (PLEG): container finished" podID="86c51d13-9e24-46d2-af54-3002ca89dec3" containerID="359ab3592d84f1d0bb73acc135bd3113de6165448712cd1324b870b792a9262a" exitCode=1 Jul 15 21:25:04.011376 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:04.011027 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fpp7b" event={"ID":"86c51d13-9e24-46d2-af54-3002ca89dec3","Type":"ContainerDied","Data":"359ab3592d84f1d0bb73acc135bd3113de6165448712cd1324b870b792a9262a"} Jul 15 21:25:04.011376 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:04.011333 2579 scope.go:117] "RemoveContainer" containerID="359ab3592d84f1d0bb73acc135bd3113de6165448712cd1324b870b792a9262a" Jul 15 21:25:04.014684 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:04.014663 2579 generic.go:356] "Generic (PLEG): container finished" podID="799804cd-f071-4c3f-8217-7a20956aeac7" containerID="7795763290e0b3d9ecdc5528a1e596e8e9c70b86209a1f195627e6f11bdf00e9" exitCode=1 Jul 15 21:25:04.014785 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:04.014742 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4cj4n" event={"ID":"799804cd-f071-4c3f-8217-7a20956aeac7","Type":"ContainerDied","Data":"7795763290e0b3d9ecdc5528a1e596e8e9c70b86209a1f195627e6f11bdf00e9"} Jul 15 21:25:04.015037 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:04.015022 2579 scope.go:117] "RemoveContainer" containerID="7795763290e0b3d9ecdc5528a1e596e8e9c70b86209a1f195627e6f11bdf00e9" Jul 15 21:25:04.016639 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:04.016615 2579 generic.go:356] "Generic (PLEG): container finished" podID="ccbdd9b9-e1f6-4075-b80d-9ea7827580cf" containerID="0d45cfc026d5db823492e31b8cd7f8fa89e5b1abbcfed0f598b4f0604475a7fd" exitCode=1 Jul 15 21:25:04.016736 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:04.016683 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mcldd" event={"ID":"ccbdd9b9-e1f6-4075-b80d-9ea7827580cf","Type":"ContainerDied","Data":"0d45cfc026d5db823492e31b8cd7f8fa89e5b1abbcfed0f598b4f0604475a7fd"} Jul 15 21:25:04.016961 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:04.016944 2579 scope.go:117] "RemoveContainer" containerID="0d45cfc026d5db823492e31b8cd7f8fa89e5b1abbcfed0f598b4f0604475a7fd" Jul 15 21:25:04.022546 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:04.022524 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 15 21:25:04.168111 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:04.168024 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4cj4n" Jul 15 21:25:04.787281 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:04.787243 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gxtc2" Jul 15 21:25:05.023778 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:05.023741 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4cj4n" event={"ID":"799804cd-f071-4c3f-8217-7a20956aeac7","Type":"ContainerStarted","Data":"dd9e140a79b60067fde5012ec6676b857adfb67ca9e0d91b054da900c703e0ea"} Jul 15 21:25:05.023989 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:05.023845 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4cj4n" Jul 15 21:25:05.025872 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:05.025846 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7bzzs" event={"ID":"cd1829fe-534c-44fc-b6cf-5761acfa46a0","Type":"ContainerStarted","Data":"e7c779d1161450b0855c13d0db793eaeb47f632578ea9a9ead938e6b1f34c196"} Jul 15 21:25:05.026082 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:05.026044 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7bzzs" Jul 15 21:25:05.028330 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:05.028307 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65c67f598f-wb248" event={"ID":"7193df13-36d0-4f6b-826b-4c876d9c2299","Type":"ContainerStarted","Data":"c68a4e85fdcaef8b1a13e493b126cbf7c37fb514f37e873e2fc659dec1d91d53"} Jul 15 21:25:05.028509 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:05.028488 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-65c67f598f-wb248" Jul 15 21:25:05.030451 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:05.030393 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-2mq85" event={"ID":"c75adccd-fdbc-47a6-a744-c7020160ebb5","Type":"ContainerStarted","Data":"cff92fbe466b60f8d4644c78568759bb5bf72149c78d5aacec6601d1c982cd5f"} Jul 15 21:25:05.030639 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:05.030618 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-2mq85" Jul 15 21:25:05.032447 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:05.032425 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gxtc2" event={"ID":"46fc8345-bb02-4935-bfa9-227aba6b8077","Type":"ContainerStarted","Data":"ecdca68f1625037f54c5c904db3b6d8705bc5286d4250e16f583e7454477cc43"} Jul 15 21:25:05.032553 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:05.032468 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gxtc2" Jul 15 21:25:05.034280 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:05.034261 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t8w9q" event={"ID":"ee17d5c5-5655-460e-a148-0e4b4e95ee2a","Type":"ContainerStarted","Data":"b7f9df140befc7e3b012bff4298186e604d37adc8d251fa0a8b2f192f1f7ae5f"} Jul 15 21:25:05.034447 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:05.034431 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t8w9q" Jul 15 21:25:05.035909 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:05.035889 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fpp7b" event={"ID":"86c51d13-9e24-46d2-af54-3002ca89dec3","Type":"ContainerStarted","Data":"3f380b35ee80b609ea22ea1d7ab483a01c196a6e0f691a012fd3f08fc4c12a27"} Jul 15 21:25:05.036100 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:05.036082 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fpp7b" Jul 15 21:25:06.043134 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:06.043096 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mcldd" event={"ID":"ccbdd9b9-e1f6-4075-b80d-9ea7827580cf","Type":"ContainerStarted","Data":"55c4685eff8daefa1248ab0124131771ea7c739bc629b0858b593b550e9e38b3"} Jul 15 21:25:06.043563 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:06.043541 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mcldd" Jul 15 21:25:07.048579 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:07.048541 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mcldd" Jul 15 21:25:07.838794 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:25:07.838749 2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-07-15T21:24:57Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-15T21:24:57Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-15T21:24:57Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-15T21:24:57Z\\\",\\\"type\\\":\\\"Ready\\\"}]}}\" for node \"ip-10-0-137-185.ec2.internal\": Patch \"https://a9c0189c76ffa414bbe64782f22ffef9-e324037f91a3c3d1.elb.us-east-1.amazonaws.com:6443/api/v1/nodes/ip-10-0-137-185.ec2.internal/status?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Jul 15 21:25:13.502654 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:25:13.502596 2579 controller.go:195] "Failed to update lease" err="Put \"https://a9c0189c76ffa414bbe64782f22ffef9-e324037f91a3c3d1.elb.us-east-1.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-137-185.ec2.internal?timeout=10s\": context deadline exceeded" Jul 15 21:25:16.044940 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:16.044892 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/opendatahub-operator-controller-manager-65c67f598f-wb248" podUID="7193df13-36d0-4f6b-826b-4c876d9c2299" containerName="manager" probeResult="failure" output="Get \"http://10.132.0.20:8081/readyz\": dial tcp 10.132.0.20:8081: connect: connection refused" Jul 15 21:25:16.045558 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:16.045360 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fpp7b" Jul 15 21:25:16.045742 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:16.045594 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4cj4n" Jul 15 21:25:16.045742 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:16.045632 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t8w9q" Jul 15 21:25:16.045742 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:16.045679 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7bzzs" Jul 15 21:25:16.046181 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:16.046161 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-2mq85" Jul 15 21:25:17.603159 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:25:17.603119 2579 controller.go:195] "Failed to update lease" err="Operation cannot be fulfilled on leases.coordination.k8s.io \"ip-10-0-137-185.ec2.internal\": the object has been modified; please apply your changes to the latest version and try again" Jul 15 21:25:24.068241 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:24.068204 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:25:26.045182 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:26.045152 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-65c67f598f-wb248" Jul 15 21:25:27.524233 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:25:27.524189 2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-07-15T21:25:17Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-15T21:25:17Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-15T21:25:17Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-15T21:25:17Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"}]}}\" for node \"ip-10-0-137-185.ec2.internal\": Patch \"https://a9c0189c76ffa414bbe64782f22ffef9-e324037f91a3c3d1.elb.us-east-1.amazonaws.com:6443/api/v1/nodes/ip-10-0-137-185.ec2.internal/status?timeout=10s\": context deadline exceeded" Jul 15 21:25:36.048852 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:36.048823 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gxtc2" Jul 15 21:25:37.525215 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:25:37.525175 2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-185.ec2.internal\": Get \"https://a9c0189c76ffa414bbe64782f22ffef9-e324037f91a3c3d1.elb.us-east-1.amazonaws.com:6443/api/v1/nodes/ip-10-0-137-185.ec2.internal?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Jul 15 21:25:38.056157 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:38.056117 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mcldd" Jul 15 21:25:38.999496 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:25:38.999453 2579 controller.go:195] "Failed to update lease" err="Put \"https://a9c0189c76ffa414bbe64782f22ffef9-e324037f91a3c3d1.elb.us-east-1.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-137-185.ec2.internal?timeout=10s\": context deadline exceeded" Jul 15 21:25:47.525685 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:25:47.525642 2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-185.ec2.internal\": Get \"https://a9c0189c76ffa414bbe64782f22ffef9-e324037f91a3c3d1.elb.us-east-1.amazonaws.com:6443/api/v1/nodes/ip-10-0-137-185.ec2.internal?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Jul 15 21:25:49.000637 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:25:49.000589 2579 controller.go:195] "Failed to update lease" err="Put \"https://a9c0189c76ffa414bbe64782f22ffef9-e324037f91a3c3d1.elb.us-east-1.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-137-185.ec2.internal?timeout=10s\": context deadline exceeded" Jul 15 21:25:49.307366 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:25:49.307270 2579 controller.go:195] "Failed to update lease" err="Operation cannot be fulfilled on leases.coordination.k8s.io \"ip-10-0-137-185.ec2.internal\": the object has been modified; please apply your changes to the latest version and try again" Jul 15 21:25:49.734034 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:25:49.733995 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:26:01.579979 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:26:01.579941 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:26:08.294163 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:26:08.294121 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:26:12.322919 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:26:12.322878 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:26:29.116700 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:26:29.116660 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:26:35.981517 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:26:35.981475 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:26:36.054961 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:26:36.054928 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m5r2m_0db09c1e-d24d-4519-93c8-c185018c3d24/ovn-acl-logging/0.log" Jul 15 21:26:36.054961 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:26:36.054970 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m5r2m_0db09c1e-d24d-4519-93c8-c185018c3d24/ovn-acl-logging/0.log" Jul 15 21:26:44.610577 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:26:44.610536 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:26:55.599340 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:26:55.599301 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:27:03.499280 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:27:03.499192 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:27:14.456864 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:27:14.456826 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:27:26.135883 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:27:26.135837 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:27:29.335549 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:27:29.335506 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:27:37.648414 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:27:37.648376 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:27:54.095903 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:27:54.095863 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:28:08.594954 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:28:08.594915 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:28:17.997524 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:28:17.997496 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:28:27.368928 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:28:27.368888 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:28:37.595539 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:28:37.595496 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:28:39.365694 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:28:39.365635 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:28:39.611585 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:28:39.611544 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:28:43.914262 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:28:43.914224 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:28:51.100387 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:28:51.100354 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:29:09.093532 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:09.093498 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:29:11.328259 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:11.328209 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:29:20.999617 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:20.999572 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:29:21.500107 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:21.500056 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:29:29.191552 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:29.191508 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:29:34.338504 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:34.338463 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:29:37.343376 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:37.343337 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:29:37.443904 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:37.443867 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:29:37.867708 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:37.867665 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:29:38.593642 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:38.593610 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:29:44.048589 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:44.048554 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:29:44.192383 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:44.192344 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:29:44.349624 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:44.349586 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n"] Jul 15 21:29:44.353166 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:44.353140 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n" Jul 15 21:29:44.356271 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:44.356247 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-8cee6bf5-openshift-default-dockercfg-lls2b\"" Jul 15 21:29:44.371914 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:44.371880 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:29:44.436152 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:44.436116 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n"] Jul 15 21:29:44.468759 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:44.468723 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:29:44.481295 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:44.481257 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/794548fa-1781-456e-9d3e-39172e4cd4e4-workload-certs\") pod \"e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n\" (UID: \"794548fa-1781-456e-9d3e-39172e4cd4e4\") " pod="openshift-ingress/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n" Jul 15 21:29:44.481476 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:44.481302 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fm24h\" (UniqueName: \"kubernetes.io/projected/794548fa-1781-456e-9d3e-39172e4cd4e4-kube-api-access-fm24h\") pod \"e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n\" (UID: \"794548fa-1781-456e-9d3e-39172e4cd4e4\") " pod="openshift-ingress/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n" Jul 15 21:29:44.481476 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:44.481365 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/794548fa-1781-456e-9d3e-39172e4cd4e4-istio-podinfo\") pod \"e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n\" (UID: \"794548fa-1781-456e-9d3e-39172e4cd4e4\") " pod="openshift-ingress/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n" Jul 15 21:29:44.481476 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:44.481397 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/794548fa-1781-456e-9d3e-39172e4cd4e4-istiod-ca-cert\") pod \"e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n\" (UID: \"794548fa-1781-456e-9d3e-39172e4cd4e4\") " pod="openshift-ingress/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n" Jul 15 21:29:44.481476 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:44.481415 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/794548fa-1781-456e-9d3e-39172e4cd4e4-istio-token\") pod \"e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n\" (UID: \"794548fa-1781-456e-9d3e-39172e4cd4e4\") " pod="openshift-ingress/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n" Jul 15 21:29:44.481476 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:44.481451 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/794548fa-1781-456e-9d3e-39172e4cd4e4-workload-socket\") pod \"e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n\" (UID: \"794548fa-1781-456e-9d3e-39172e4cd4e4\") " pod="openshift-ingress/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n" Jul 15 21:29:44.481733 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:44.481539 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/794548fa-1781-456e-9d3e-39172e4cd4e4-istio-data\") pod \"e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n\" (UID: \"794548fa-1781-456e-9d3e-39172e4cd4e4\") " pod="openshift-ingress/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n" Jul 15 21:29:44.481733 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:44.481591 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/794548fa-1781-456e-9d3e-39172e4cd4e4-istio-envoy\") pod \"e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n\" (UID: \"794548fa-1781-456e-9d3e-39172e4cd4e4\") " pod="openshift-ingress/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n" Jul 15 21:29:44.481733 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:44.481633 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/794548fa-1781-456e-9d3e-39172e4cd4e4-credential-socket\") pod \"e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n\" (UID: \"794548fa-1781-456e-9d3e-39172e4cd4e4\") " pod="openshift-ingress/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n" Jul 15 21:29:44.582908 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:44.582809 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/794548fa-1781-456e-9d3e-39172e4cd4e4-istio-podinfo\") pod \"e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n\" (UID: \"794548fa-1781-456e-9d3e-39172e4cd4e4\") " pod="openshift-ingress/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n" Jul 15 21:29:44.582908 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:44.582849 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/794548fa-1781-456e-9d3e-39172e4cd4e4-istiod-ca-cert\") pod \"e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n\" (UID: \"794548fa-1781-456e-9d3e-39172e4cd4e4\") " pod="openshift-ingress/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n" Jul 15 21:29:44.582908 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:44.582870 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/794548fa-1781-456e-9d3e-39172e4cd4e4-istio-token\") pod \"e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n\" (UID: \"794548fa-1781-456e-9d3e-39172e4cd4e4\") " pod="openshift-ingress/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n" Jul 15 21:29:44.583244 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:44.582930 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/794548fa-1781-456e-9d3e-39172e4cd4e4-workload-socket\") pod \"e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n\" (UID: \"794548fa-1781-456e-9d3e-39172e4cd4e4\") " pod="openshift-ingress/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n" Jul 15 21:29:44.583244 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:44.582994 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/794548fa-1781-456e-9d3e-39172e4cd4e4-istio-data\") pod \"e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n\" (UID: \"794548fa-1781-456e-9d3e-39172e4cd4e4\") " pod="openshift-ingress/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n" Jul 15 21:29:44.583244 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:44.583032 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/794548fa-1781-456e-9d3e-39172e4cd4e4-istio-envoy\") pod \"e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n\" (UID: \"794548fa-1781-456e-9d3e-39172e4cd4e4\") " pod="openshift-ingress/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n" Jul 15 21:29:44.583244 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:44.583089 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/794548fa-1781-456e-9d3e-39172e4cd4e4-credential-socket\") pod \"e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n\" (UID: \"794548fa-1781-456e-9d3e-39172e4cd4e4\") " pod="openshift-ingress/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n" Jul 15 21:29:44.583244 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:44.583122 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/794548fa-1781-456e-9d3e-39172e4cd4e4-workload-certs\") pod \"e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n\" (UID: \"794548fa-1781-456e-9d3e-39172e4cd4e4\") " pod="openshift-ingress/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n" Jul 15 21:29:44.583244 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:44.583152 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fm24h\" (UniqueName: \"kubernetes.io/projected/794548fa-1781-456e-9d3e-39172e4cd4e4-kube-api-access-fm24h\") pod \"e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n\" (UID: \"794548fa-1781-456e-9d3e-39172e4cd4e4\") " pod="openshift-ingress/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n" Jul 15 21:29:44.583575 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:44.583549 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/794548fa-1781-456e-9d3e-39172e4cd4e4-istio-data\") pod \"e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n\" (UID: \"794548fa-1781-456e-9d3e-39172e4cd4e4\") " pod="openshift-ingress/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n" Jul 15 21:29:44.583697 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:44.583659 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/794548fa-1781-456e-9d3e-39172e4cd4e4-credential-socket\") pod \"e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n\" (UID: \"794548fa-1781-456e-9d3e-39172e4cd4e4\") " pod="openshift-ingress/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n" Jul 15 21:29:44.583785 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:44.583692 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/794548fa-1781-456e-9d3e-39172e4cd4e4-workload-socket\") pod \"e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n\" (UID: \"794548fa-1781-456e-9d3e-39172e4cd4e4\") " pod="openshift-ingress/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n" Jul 15 21:29:44.583785 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:44.583761 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/794548fa-1781-456e-9d3e-39172e4cd4e4-workload-certs\") pod \"e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n\" (UID: \"794548fa-1781-456e-9d3e-39172e4cd4e4\") " pod="openshift-ingress/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n" Jul 15 21:29:44.583895 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:44.583868 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/794548fa-1781-456e-9d3e-39172e4cd4e4-istiod-ca-cert\") pod \"e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n\" (UID: \"794548fa-1781-456e-9d3e-39172e4cd4e4\") " pod="openshift-ingress/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n" Jul 15 21:29:44.585880 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:44.585855 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/794548fa-1781-456e-9d3e-39172e4cd4e4-istio-envoy\") pod \"e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n\" (UID: \"794548fa-1781-456e-9d3e-39172e4cd4e4\") " pod="openshift-ingress/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n" Jul 15 21:29:44.586029 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:44.586012 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/794548fa-1781-456e-9d3e-39172e4cd4e4-istio-podinfo\") pod \"e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n\" (UID: \"794548fa-1781-456e-9d3e-39172e4cd4e4\") " pod="openshift-ingress/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n" Jul 15 21:29:44.595037 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:44.595014 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fm24h\" (UniqueName: \"kubernetes.io/projected/794548fa-1781-456e-9d3e-39172e4cd4e4-kube-api-access-fm24h\") pod \"e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n\" (UID: \"794548fa-1781-456e-9d3e-39172e4cd4e4\") " pod="openshift-ingress/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n" Jul 15 21:29:44.595037 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:44.595028 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/794548fa-1781-456e-9d3e-39172e4cd4e4-istio-token\") pod \"e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n\" (UID: \"794548fa-1781-456e-9d3e-39172e4cd4e4\") " pod="openshift-ingress/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n" Jul 15 21:29:44.663786 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:44.663749 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n" Jul 15 21:29:44.816197 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:44.816172 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n"] Jul 15 21:29:44.818366 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:29:44.818336 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod794548fa_1781_456e_9d3e_39172e4cd4e4.slice/crio-47ab8c5ee8a02cff6a13d9ce54f33690875d3d4cbed8d5893732abc71a33f392 WatchSource:0}: Error finding container 47ab8c5ee8a02cff6a13d9ce54f33690875d3d4cbed8d5893732abc71a33f392: Status 404 returned error can't find the container with id 47ab8c5ee8a02cff6a13d9ce54f33690875d3d4cbed8d5893732abc71a33f392 Jul 15 21:29:44.820447 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:44.820423 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:29:44.820523 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:44.820483 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:29:44.820523 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:44.820511 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:29:45.556147 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:45.556113 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:29:45.592403 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:45.592372 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n" event={"ID":"794548fa-1781-456e-9d3e-39172e4cd4e4","Type":"ContainerStarted","Data":"89c5ce7f47b5df5916a0e22c8a18fa822dd8921e1ca1116de2ab5eaaa6f6c974"} Jul 15 21:29:45.592575 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:45.592424 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n" event={"ID":"794548fa-1781-456e-9d3e-39172e4cd4e4","Type":"ContainerStarted","Data":"47ab8c5ee8a02cff6a13d9ce54f33690875d3d4cbed8d5893732abc71a33f392"} Jul 15 21:29:45.623874 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:45.623820 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n" podStartSLOduration=1.62380078 podStartE2EDuration="1.62380078s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:29:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:29:45.621555573 +0000 UTC m=+1690.162868122" watchObservedRunningTime="2026-07-15 21:29:45.62380078 +0000 UTC m=+1690.165113329" Jul 15 21:29:45.664485 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:45.664451 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n" Jul 15 21:29:45.669818 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:45.669791 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n" Jul 15 21:29:45.701056 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:45.701022 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:29:45.833035 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:45.832933 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:29:46.406953 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:46.406917 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:29:46.597364 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:46.597335 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n" Jul 15 21:29:46.598304 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:46.598271 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n" Jul 15 21:29:47.264859 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:47.264820 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:29:47.370261 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:47.370221 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:29:48.293463 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:48.293423 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:29:49.396189 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:49.396152 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:29:57.663911 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:57.663875 2579 generic.go:356] "Generic (PLEG): container finished" podID="794548fa-1781-456e-9d3e-39172e4cd4e4" containerID="89c5ce7f47b5df5916a0e22c8a18fa822dd8921e1ca1116de2ab5eaaa6f6c974" exitCode=0 Jul 15 21:29:57.664373 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:57.663950 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n" event={"ID":"794548fa-1781-456e-9d3e-39172e4cd4e4","Type":"ContainerDied","Data":"89c5ce7f47b5df5916a0e22c8a18fa822dd8921e1ca1116de2ab5eaaa6f6c974"} Jul 15 21:29:57.664373 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:57.664357 2579 scope.go:117] "RemoveContainer" containerID="89c5ce7f47b5df5916a0e22c8a18fa822dd8921e1ca1116de2ab5eaaa6f6c974" Jul 15 21:29:57.665447 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:57.665419 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:29:57.665553 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:57.665488 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:29:57.665553 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:57.665520 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:29:58.672109 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:58.672005 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n" event={"ID":"794548fa-1781-456e-9d3e-39172e4cd4e4","Type":"ContainerStarted","Data":"eb137afa91dbd92dded498042b5de04ca50ef609940870cc7077f009c5ffdab3"} Jul 15 21:29:59.664839 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:59.664797 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n" Jul 15 21:29:59.665984 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:59.665956 2579 patch_prober.go:28] interesting pod/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 15 21:29:59.666134 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:29:59.666019 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n" podUID="794548fa-1781-456e-9d3e-39172e4cd4e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 15 21:30:00.664198 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:30:00.664156 2579 patch_prober.go:28] interesting pod/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 15 21:30:00.664687 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:30:00.664221 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n" podUID="794548fa-1781-456e-9d3e-39172e4cd4e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 15 21:30:01.597835 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:30:01.597792 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n" Jul 15 21:30:01.664551 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:30:01.664508 2579 patch_prober.go:28] interesting pod/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 15 21:30:01.665025 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:30:01.664572 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n" podUID="794548fa-1781-456e-9d3e-39172e4cd4e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 15 21:30:02.665017 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:30:02.664982 2579 patch_prober.go:28] interesting pod/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 15 21:30:02.665425 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:30:02.665042 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n" podUID="794548fa-1781-456e-9d3e-39172e4cd4e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 15 21:30:03.668124 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:30:03.668086 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n" Jul 15 21:30:03.669047 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:30:03.669025 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n" Jul 15 21:30:51.853887 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:30:51.853843 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:31:02.064903 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:02.064856 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:31:03.200577 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:03.200538 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:31:08.822824 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:08.822783 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:31:08.909221 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:08.909186 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:31:12.279335 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:12.279281 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n"] Jul 15 21:31:12.279813 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:12.279519 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n" podUID="794548fa-1781-456e-9d3e-39172e4cd4e4" containerName="istio-proxy" containerID="cri-o://eb137afa91dbd92dded498042b5de04ca50ef609940870cc7077f009c5ffdab3" gracePeriod=30 Jul 15 21:31:12.894293 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:12.894260 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg"] Jul 15 21:31:12.896983 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:12.896959 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg" Jul 15 21:31:12.902874 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:12.902850 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-18bc3701-gw-openshift-default-dockercfg-d6657\"" Jul 15 21:31:12.913943 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:12.913912 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg"] Jul 15 21:31:12.949889 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:12.949850 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb72104c-e8a8-4bf1-b48e-b5dbcb200d70-istiod-ca-cert\") pod \"e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg\" (UID: \"cb72104c-e8a8-4bf1-b48e-b5dbcb200d70\") " pod="openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg" Jul 15 21:31:12.950082 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:12.949916 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb72104c-e8a8-4bf1-b48e-b5dbcb200d70-istio-envoy\") pod \"e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg\" (UID: \"cb72104c-e8a8-4bf1-b48e-b5dbcb200d70\") " pod="openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg" Jul 15 21:31:12.950082 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:12.949962 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb72104c-e8a8-4bf1-b48e-b5dbcb200d70-istio-token\") pod \"e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg\" (UID: \"cb72104c-e8a8-4bf1-b48e-b5dbcb200d70\") " pod="openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg" Jul 15 21:31:12.950082 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:12.949991 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb72104c-e8a8-4bf1-b48e-b5dbcb200d70-workload-certs\") pod \"e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg\" (UID: \"cb72104c-e8a8-4bf1-b48e-b5dbcb200d70\") " pod="openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg" Jul 15 21:31:12.950082 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:12.950011 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb72104c-e8a8-4bf1-b48e-b5dbcb200d70-istio-data\") pod \"e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg\" (UID: \"cb72104c-e8a8-4bf1-b48e-b5dbcb200d70\") " pod="openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg" Jul 15 21:31:12.950082 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:12.950029 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9fpr\" (UniqueName: \"kubernetes.io/projected/cb72104c-e8a8-4bf1-b48e-b5dbcb200d70-kube-api-access-r9fpr\") pod \"e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg\" (UID: \"cb72104c-e8a8-4bf1-b48e-b5dbcb200d70\") " pod="openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg" Jul 15 21:31:12.950272 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:12.950057 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb72104c-e8a8-4bf1-b48e-b5dbcb200d70-credential-socket\") pod \"e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg\" (UID: \"cb72104c-e8a8-4bf1-b48e-b5dbcb200d70\") " pod="openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg" Jul 15 21:31:12.950272 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:12.950129 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb72104c-e8a8-4bf1-b48e-b5dbcb200d70-workload-socket\") pod \"e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg\" (UID: \"cb72104c-e8a8-4bf1-b48e-b5dbcb200d70\") " pod="openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg" Jul 15 21:31:12.950272 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:12.950163 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb72104c-e8a8-4bf1-b48e-b5dbcb200d70-istio-podinfo\") pod \"e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg\" (UID: \"cb72104c-e8a8-4bf1-b48e-b5dbcb200d70\") " pod="openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg" Jul 15 21:31:13.033363 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:13.033322 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:31:13.051450 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:13.051412 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb72104c-e8a8-4bf1-b48e-b5dbcb200d70-istio-podinfo\") pod \"e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg\" (UID: \"cb72104c-e8a8-4bf1-b48e-b5dbcb200d70\") " pod="openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg" Jul 15 21:31:13.051658 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:13.051460 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb72104c-e8a8-4bf1-b48e-b5dbcb200d70-istiod-ca-cert\") pod \"e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg\" (UID: \"cb72104c-e8a8-4bf1-b48e-b5dbcb200d70\") " pod="openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg" Jul 15 21:31:13.051658 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:13.051512 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb72104c-e8a8-4bf1-b48e-b5dbcb200d70-istio-envoy\") pod \"e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg\" (UID: \"cb72104c-e8a8-4bf1-b48e-b5dbcb200d70\") " pod="openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg" Jul 15 21:31:13.051658 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:13.051549 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb72104c-e8a8-4bf1-b48e-b5dbcb200d70-istio-token\") pod \"e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg\" (UID: \"cb72104c-e8a8-4bf1-b48e-b5dbcb200d70\") " pod="openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg" Jul 15 21:31:13.051820 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:13.051679 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb72104c-e8a8-4bf1-b48e-b5dbcb200d70-workload-certs\") pod \"e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg\" (UID: \"cb72104c-e8a8-4bf1-b48e-b5dbcb200d70\") " pod="openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg" Jul 15 21:31:13.051820 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:13.051724 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb72104c-e8a8-4bf1-b48e-b5dbcb200d70-istio-data\") pod \"e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg\" (UID: \"cb72104c-e8a8-4bf1-b48e-b5dbcb200d70\") " pod="openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg" Jul 15 21:31:13.051820 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:13.051748 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9fpr\" (UniqueName: \"kubernetes.io/projected/cb72104c-e8a8-4bf1-b48e-b5dbcb200d70-kube-api-access-r9fpr\") pod \"e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg\" (UID: \"cb72104c-e8a8-4bf1-b48e-b5dbcb200d70\") " pod="openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg" Jul 15 21:31:13.051820 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:13.051790 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb72104c-e8a8-4bf1-b48e-b5dbcb200d70-credential-socket\") pod \"e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg\" (UID: \"cb72104c-e8a8-4bf1-b48e-b5dbcb200d70\") " pod="openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg" Jul 15 21:31:13.052166 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:13.051825 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb72104c-e8a8-4bf1-b48e-b5dbcb200d70-workload-socket\") pod \"e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg\" (UID: \"cb72104c-e8a8-4bf1-b48e-b5dbcb200d70\") " pod="openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg" Jul 15 21:31:13.052166 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:13.052082 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb72104c-e8a8-4bf1-b48e-b5dbcb200d70-workload-certs\") pod \"e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg\" (UID: \"cb72104c-e8a8-4bf1-b48e-b5dbcb200d70\") " pod="openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg" Jul 15 21:31:13.052306 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:13.052285 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb72104c-e8a8-4bf1-b48e-b5dbcb200d70-istiod-ca-cert\") pod \"e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg\" (UID: \"cb72104c-e8a8-4bf1-b48e-b5dbcb200d70\") " pod="openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg" Jul 15 21:31:13.052369 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:13.052331 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb72104c-e8a8-4bf1-b48e-b5dbcb200d70-credential-socket\") pod \"e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg\" (UID: \"cb72104c-e8a8-4bf1-b48e-b5dbcb200d70\") " pod="openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg" Jul 15 21:31:13.052425 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:13.052389 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb72104c-e8a8-4bf1-b48e-b5dbcb200d70-istio-data\") pod \"e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg\" (UID: \"cb72104c-e8a8-4bf1-b48e-b5dbcb200d70\") " pod="openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg" Jul 15 21:31:13.052670 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:13.052450 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb72104c-e8a8-4bf1-b48e-b5dbcb200d70-workload-socket\") pod \"e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg\" (UID: \"cb72104c-e8a8-4bf1-b48e-b5dbcb200d70\") " pod="openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg" Jul 15 21:31:13.053883 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:13.053857 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb72104c-e8a8-4bf1-b48e-b5dbcb200d70-istio-envoy\") pod \"e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg\" (UID: \"cb72104c-e8a8-4bf1-b48e-b5dbcb200d70\") " pod="openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg" Jul 15 21:31:13.054118 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:13.054101 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb72104c-e8a8-4bf1-b48e-b5dbcb200d70-istio-podinfo\") pod \"e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg\" (UID: \"cb72104c-e8a8-4bf1-b48e-b5dbcb200d70\") " pod="openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg" Jul 15 21:31:13.062561 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:13.062533 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb72104c-e8a8-4bf1-b48e-b5dbcb200d70-istio-token\") pod \"e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg\" (UID: \"cb72104c-e8a8-4bf1-b48e-b5dbcb200d70\") " pod="openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg" Jul 15 21:31:13.062659 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:13.062591 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9fpr\" (UniqueName: \"kubernetes.io/projected/cb72104c-e8a8-4bf1-b48e-b5dbcb200d70-kube-api-access-r9fpr\") pod \"e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg\" (UID: \"cb72104c-e8a8-4bf1-b48e-b5dbcb200d70\") " pod="openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg" Jul 15 21:31:13.208700 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:13.208591 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg" Jul 15 21:31:13.361193 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:13.361164 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg"] Jul 15 21:31:13.363385 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:31:13.363357 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb72104c_e8a8_4bf1_b48e_b5dbcb200d70.slice/crio-ed1220bd0b8addf7b6732a9a7e1c30484f91c068a8a4e7f561a7653acb754927 WatchSource:0}: Error finding container ed1220bd0b8addf7b6732a9a7e1c30484f91c068a8a4e7f561a7653acb754927: Status 404 returned error can't find the container with id ed1220bd0b8addf7b6732a9a7e1c30484f91c068a8a4e7f561a7653acb754927 Jul 15 21:31:13.365493 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:13.365463 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:31:13.365585 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:13.365522 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:31:13.365585 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:13.365554 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:31:13.407446 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:13.407419 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:31:14.087477 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:14.087438 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg" event={"ID":"cb72104c-e8a8-4bf1-b48e-b5dbcb200d70","Type":"ContainerStarted","Data":"e0e126c42850886502e31ed680509bb05220d01fb9af093d12d9a0934e922c10"} Jul 15 21:31:14.087651 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:14.087499 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg" event={"ID":"cb72104c-e8a8-4bf1-b48e-b5dbcb200d70","Type":"ContainerStarted","Data":"ed1220bd0b8addf7b6732a9a7e1c30484f91c068a8a4e7f561a7653acb754927"} Jul 15 21:31:14.118426 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:14.118372 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg" podStartSLOduration=2.118352491 podStartE2EDuration="2.118352491s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:31:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:31:14.116023903 +0000 UTC m=+1778.657336452" watchObservedRunningTime="2026-07-15 21:31:14.118352491 +0000 UTC m=+1778.659665044" Jul 15 21:31:14.169709 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:14.169676 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:31:14.209362 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:14.209319 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg" Jul 15 21:31:14.214315 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:14.214289 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg" Jul 15 21:31:14.343232 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:14.343135 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:31:14.546435 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:14.546399 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:31:14.855781 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:14.855747 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:31:14.958612 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:14.958575 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:31:15.093049 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:15.093019 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg" Jul 15 21:31:15.094195 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:15.094166 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg" Jul 15 21:31:17.080980 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:17.080941 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:31:17.336932 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:17.336838 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:31:17.541276 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:17.541247 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n" Jul 15 21:31:17.695377 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:17.695342 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/794548fa-1781-456e-9d3e-39172e4cd4e4-istio-data\") pod \"794548fa-1781-456e-9d3e-39172e4cd4e4\" (UID: \"794548fa-1781-456e-9d3e-39172e4cd4e4\") " Jul 15 21:31:17.695543 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:17.695392 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/794548fa-1781-456e-9d3e-39172e4cd4e4-istiod-ca-cert\") pod \"794548fa-1781-456e-9d3e-39172e4cd4e4\" (UID: \"794548fa-1781-456e-9d3e-39172e4cd4e4\") " Jul 15 21:31:17.695543 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:17.695418 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/794548fa-1781-456e-9d3e-39172e4cd4e4-istio-envoy\") pod \"794548fa-1781-456e-9d3e-39172e4cd4e4\" (UID: \"794548fa-1781-456e-9d3e-39172e4cd4e4\") " Jul 15 21:31:17.695543 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:17.695458 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/794548fa-1781-456e-9d3e-39172e4cd4e4-credential-socket\") pod \"794548fa-1781-456e-9d3e-39172e4cd4e4\" (UID: \"794548fa-1781-456e-9d3e-39172e4cd4e4\") " Jul 15 21:31:17.695543 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:17.695506 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/794548fa-1781-456e-9d3e-39172e4cd4e4-workload-socket\") pod \"794548fa-1781-456e-9d3e-39172e4cd4e4\" (UID: \"794548fa-1781-456e-9d3e-39172e4cd4e4\") " Jul 15 21:31:17.695759 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:17.695574 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/794548fa-1781-456e-9d3e-39172e4cd4e4-workload-certs\") pod \"794548fa-1781-456e-9d3e-39172e4cd4e4\" (UID: \"794548fa-1781-456e-9d3e-39172e4cd4e4\") " Jul 15 21:31:17.695759 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:17.695598 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fm24h\" (UniqueName: \"kubernetes.io/projected/794548fa-1781-456e-9d3e-39172e4cd4e4-kube-api-access-fm24h\") pod \"794548fa-1781-456e-9d3e-39172e4cd4e4\" (UID: \"794548fa-1781-456e-9d3e-39172e4cd4e4\") " Jul 15 21:31:17.695759 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:17.695661 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/794548fa-1781-456e-9d3e-39172e4cd4e4-istio-token\") pod \"794548fa-1781-456e-9d3e-39172e4cd4e4\" (UID: \"794548fa-1781-456e-9d3e-39172e4cd4e4\") " Jul 15 21:31:17.695863 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:17.695777 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/794548fa-1781-456e-9d3e-39172e4cd4e4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "794548fa-1781-456e-9d3e-39172e4cd4e4" (UID: "794548fa-1781-456e-9d3e-39172e4cd4e4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:31:17.695863 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:17.695817 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/794548fa-1781-456e-9d3e-39172e4cd4e4-istio-podinfo\") pod \"794548fa-1781-456e-9d3e-39172e4cd4e4\" (UID: \"794548fa-1781-456e-9d3e-39172e4cd4e4\") " Jul 15 21:31:17.695863 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:17.695823 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/794548fa-1781-456e-9d3e-39172e4cd4e4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "794548fa-1781-456e-9d3e-39172e4cd4e4" (UID: "794548fa-1781-456e-9d3e-39172e4cd4e4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 21:31:17.696145 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:17.696120 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/794548fa-1781-456e-9d3e-39172e4cd4e4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "794548fa-1781-456e-9d3e-39172e4cd4e4" (UID: "794548fa-1781-456e-9d3e-39172e4cd4e4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:31:17.696145 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:17.696133 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/794548fa-1781-456e-9d3e-39172e4cd4e4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "794548fa-1781-456e-9d3e-39172e4cd4e4" (UID: "794548fa-1781-456e-9d3e-39172e4cd4e4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:31:17.696300 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:17.696161 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/794548fa-1781-456e-9d3e-39172e4cd4e4-istio-data" (OuterVolumeSpecName: "istio-data") pod "794548fa-1781-456e-9d3e-39172e4cd4e4" (UID: "794548fa-1781-456e-9d3e-39172e4cd4e4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:31:17.696300 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:17.696225 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/794548fa-1781-456e-9d3e-39172e4cd4e4-istiod-ca-cert\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:31:17.696300 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:17.696247 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/794548fa-1781-456e-9d3e-39172e4cd4e4-credential-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:31:17.696300 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:17.696261 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/794548fa-1781-456e-9d3e-39172e4cd4e4-workload-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:31:17.697825 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:17.697802 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/794548fa-1781-456e-9d3e-39172e4cd4e4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "794548fa-1781-456e-9d3e-39172e4cd4e4" (UID: "794548fa-1781-456e-9d3e-39172e4cd4e4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 21:31:17.697932 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:17.697872 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/794548fa-1781-456e-9d3e-39172e4cd4e4-kube-api-access-fm24h" (OuterVolumeSpecName: "kube-api-access-fm24h") pod "794548fa-1781-456e-9d3e-39172e4cd4e4" (UID: "794548fa-1781-456e-9d3e-39172e4cd4e4"). InnerVolumeSpecName "kube-api-access-fm24h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:31:17.697932 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:17.697888 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/794548fa-1781-456e-9d3e-39172e4cd4e4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "794548fa-1781-456e-9d3e-39172e4cd4e4" (UID: "794548fa-1781-456e-9d3e-39172e4cd4e4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:31:17.698023 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:17.697929 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/794548fa-1781-456e-9d3e-39172e4cd4e4-istio-token" (OuterVolumeSpecName: "istio-token") pod "794548fa-1781-456e-9d3e-39172e4cd4e4" (UID: "794548fa-1781-456e-9d3e-39172e4cd4e4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:31:17.797573 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:17.797523 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/794548fa-1781-456e-9d3e-39172e4cd4e4-istio-podinfo\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:31:17.797573 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:17.797567 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/794548fa-1781-456e-9d3e-39172e4cd4e4-istio-data\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:31:17.797573 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:17.797577 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/794548fa-1781-456e-9d3e-39172e4cd4e4-istio-envoy\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:31:17.797573 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:17.797585 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/794548fa-1781-456e-9d3e-39172e4cd4e4-workload-certs\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:31:17.797573 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:17.797593 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fm24h\" (UniqueName: \"kubernetes.io/projected/794548fa-1781-456e-9d3e-39172e4cd4e4-kube-api-access-fm24h\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:31:17.797891 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:17.797602 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/794548fa-1781-456e-9d3e-39172e4cd4e4-istio-token\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:31:18.109165 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:18.109126 2579 generic.go:356] "Generic (PLEG): container finished" podID="794548fa-1781-456e-9d3e-39172e4cd4e4" containerID="eb137afa91dbd92dded498042b5de04ca50ef609940870cc7077f009c5ffdab3" exitCode=0 Jul 15 21:31:18.109543 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:18.109179 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n" Jul 15 21:31:18.109543 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:18.109202 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n" event={"ID":"794548fa-1781-456e-9d3e-39172e4cd4e4","Type":"ContainerDied","Data":"eb137afa91dbd92dded498042b5de04ca50ef609940870cc7077f009c5ffdab3"} Jul 15 21:31:18.109543 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:18.109266 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n" event={"ID":"794548fa-1781-456e-9d3e-39172e4cd4e4","Type":"ContainerDied","Data":"47ab8c5ee8a02cff6a13d9ce54f33690875d3d4cbed8d5893732abc71a33f392"} Jul 15 21:31:18.109543 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:18.109290 2579 scope.go:117] "RemoveContainer" containerID="eb137afa91dbd92dded498042b5de04ca50ef609940870cc7077f009c5ffdab3" Jul 15 21:31:18.118325 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:18.118306 2579 scope.go:117] "RemoveContainer" containerID="89c5ce7f47b5df5916a0e22c8a18fa822dd8921e1ca1116de2ab5eaaa6f6c974" Jul 15 21:31:18.126900 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:18.126878 2579 scope.go:117] "RemoveContainer" containerID="eb137afa91dbd92dded498042b5de04ca50ef609940870cc7077f009c5ffdab3" Jul 15 21:31:18.127229 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:31:18.127201 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb137afa91dbd92dded498042b5de04ca50ef609940870cc7077f009c5ffdab3\": container with ID starting with eb137afa91dbd92dded498042b5de04ca50ef609940870cc7077f009c5ffdab3 not found: ID does not exist" containerID="eb137afa91dbd92dded498042b5de04ca50ef609940870cc7077f009c5ffdab3" Jul 15 21:31:18.127360 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:18.127235 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb137afa91dbd92dded498042b5de04ca50ef609940870cc7077f009c5ffdab3"} err="failed to get container status \"eb137afa91dbd92dded498042b5de04ca50ef609940870cc7077f009c5ffdab3\": rpc error: code = NotFound desc = could not find container \"eb137afa91dbd92dded498042b5de04ca50ef609940870cc7077f009c5ffdab3\": container with ID starting with eb137afa91dbd92dded498042b5de04ca50ef609940870cc7077f009c5ffdab3 not found: ID does not exist" Jul 15 21:31:18.127360 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:18.127255 2579 scope.go:117] "RemoveContainer" containerID="89c5ce7f47b5df5916a0e22c8a18fa822dd8921e1ca1116de2ab5eaaa6f6c974" Jul 15 21:31:18.127475 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:31:18.127454 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89c5ce7f47b5df5916a0e22c8a18fa822dd8921e1ca1116de2ab5eaaa6f6c974\": container with ID starting with 89c5ce7f47b5df5916a0e22c8a18fa822dd8921e1ca1116de2ab5eaaa6f6c974 not found: ID does not exist" containerID="89c5ce7f47b5df5916a0e22c8a18fa822dd8921e1ca1116de2ab5eaaa6f6c974" Jul 15 21:31:18.127528 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:18.127473 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89c5ce7f47b5df5916a0e22c8a18fa822dd8921e1ca1116de2ab5eaaa6f6c974"} err="failed to get container status \"89c5ce7f47b5df5916a0e22c8a18fa822dd8921e1ca1116de2ab5eaaa6f6c974\": rpc error: code = NotFound desc = could not find container \"89c5ce7f47b5df5916a0e22c8a18fa822dd8921e1ca1116de2ab5eaaa6f6c974\": container with ID starting with 89c5ce7f47b5df5916a0e22c8a18fa822dd8921e1ca1116de2ab5eaaa6f6c974 not found: ID does not exist" Jul 15 21:31:18.147311 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:18.147278 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n"] Jul 15 21:31:18.159757 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:18.159726 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-8cee6bf5-openshift-default-7f467bc47-ppj4n"] Jul 15 21:31:19.557984 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:19.557938 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:31:19.716265 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:19.716230 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:31:19.728314 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:19.728262 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:31:19.998277 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:19.998246 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="794548fa-1781-456e-9d3e-39172e4cd4e4" path="/var/lib/kubelet/pods/794548fa-1781-456e-9d3e-39172e4cd4e4/volumes" Jul 15 21:31:28.176916 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:28.176833 2579 generic.go:356] "Generic (PLEG): container finished" podID="cb72104c-e8a8-4bf1-b48e-b5dbcb200d70" containerID="e0e126c42850886502e31ed680509bb05220d01fb9af093d12d9a0934e922c10" exitCode=0 Jul 15 21:31:28.177366 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:28.176912 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg" event={"ID":"cb72104c-e8a8-4bf1-b48e-b5dbcb200d70","Type":"ContainerDied","Data":"e0e126c42850886502e31ed680509bb05220d01fb9af093d12d9a0934e922c10"} Jul 15 21:31:28.177366 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:28.177348 2579 scope.go:117] "RemoveContainer" containerID="e0e126c42850886502e31ed680509bb05220d01fb9af093d12d9a0934e922c10" Jul 15 21:31:28.178384 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:28.178354 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:31:28.178496 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:28.178440 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:31:28.178496 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:28.178489 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:31:29.190169 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:29.190127 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg" event={"ID":"cb72104c-e8a8-4bf1-b48e-b5dbcb200d70","Type":"ContainerStarted","Data":"f910a96538628ccb38026f950b604b2a66f0e17736fb0a0058738d4d797435e8"} Jul 15 21:31:29.209006 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:29.208972 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg" Jul 15 21:31:29.210252 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:29.210223 2579 patch_prober.go:28] interesting pod/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 15 21:31:29.210379 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:29.210277 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg" podUID="cb72104c-e8a8-4bf1-b48e-b5dbcb200d70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 15 21:31:30.062871 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:30.062800 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:31:30.093515 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:30.093481 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg" Jul 15 21:31:30.163570 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:30.163540 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:31:30.210012 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:30.209976 2579 patch_prober.go:28] interesting pod/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 15 21:31:30.210503 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:30.210059 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg" podUID="cb72104c-e8a8-4bf1-b48e-b5dbcb200d70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 15 21:31:31.209106 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:31.209053 2579 patch_prober.go:28] interesting pod/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 15 21:31:31.209320 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:31.209162 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg" podUID="cb72104c-e8a8-4bf1-b48e-b5dbcb200d70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 15 21:31:32.209620 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:32.209587 2579 patch_prober.go:28] interesting pod/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 15 21:31:32.210115 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:32.209648 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg" podUID="cb72104c-e8a8-4bf1-b48e-b5dbcb200d70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 15 21:31:33.208977 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:33.208939 2579 patch_prober.go:28] interesting pod/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 15 21:31:33.209296 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:33.209115 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg" podUID="cb72104c-e8a8-4bf1-b48e-b5dbcb200d70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 15 21:31:33.416800 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:33.416762 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:31:34.213693 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:34.213670 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg" Jul 15 21:31:34.214665 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:34.214643 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg" Jul 15 21:31:36.093242 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:36.093209 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m5r2m_0db09c1e-d24d-4519-93c8-c185018c3d24/ovn-acl-logging/0.log" Jul 15 21:31:36.093944 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:36.093924 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m5r2m_0db09c1e-d24d-4519-93c8-c185018c3d24/ovn-acl-logging/0.log" Jul 15 21:31:55.587974 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:55.587918 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:31:55.670490 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:31:55.670452 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:32:18.875599 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:18.875564 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg"] Jul 15 21:32:18.876153 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:18.875769 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg" podUID="cb72104c-e8a8-4bf1-b48e-b5dbcb200d70" containerName="istio-proxy" containerID="cri-o://f910a96538628ccb38026f950b604b2a66f0e17736fb0a0058738d4d797435e8" gracePeriod=30 Jul 15 21:32:18.950591 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:18.950551 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:32:19.068802 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:19.068753 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:32:19.216919 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:19.216816 2579 patch_prober.go:28] interesting pod/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused Jul 15 21:32:19.216919 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:19.216900 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg" podUID="cb72104c-e8a8-4bf1-b48e-b5dbcb200d70" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 21:32:20.625608 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:20.625565 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:32:24.129485 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:24.129458 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg" Jul 15 21:32:24.191704 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:24.191612 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb72104c-e8a8-4bf1-b48e-b5dbcb200d70-istio-data\") pod \"cb72104c-e8a8-4bf1-b48e-b5dbcb200d70\" (UID: \"cb72104c-e8a8-4bf1-b48e-b5dbcb200d70\") " Jul 15 21:32:24.191704 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:24.191653 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb72104c-e8a8-4bf1-b48e-b5dbcb200d70-credential-socket\") pod \"cb72104c-e8a8-4bf1-b48e-b5dbcb200d70\" (UID: \"cb72104c-e8a8-4bf1-b48e-b5dbcb200d70\") " Jul 15 21:32:24.191704 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:24.191697 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb72104c-e8a8-4bf1-b48e-b5dbcb200d70-istiod-ca-cert\") pod \"cb72104c-e8a8-4bf1-b48e-b5dbcb200d70\" (UID: \"cb72104c-e8a8-4bf1-b48e-b5dbcb200d70\") " Jul 15 21:32:24.191983 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:24.191748 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r9fpr\" (UniqueName: \"kubernetes.io/projected/cb72104c-e8a8-4bf1-b48e-b5dbcb200d70-kube-api-access-r9fpr\") pod \"cb72104c-e8a8-4bf1-b48e-b5dbcb200d70\" (UID: \"cb72104c-e8a8-4bf1-b48e-b5dbcb200d70\") " Jul 15 21:32:24.191983 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:24.191776 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb72104c-e8a8-4bf1-b48e-b5dbcb200d70-workload-certs\") pod \"cb72104c-e8a8-4bf1-b48e-b5dbcb200d70\" (UID: \"cb72104c-e8a8-4bf1-b48e-b5dbcb200d70\") " Jul 15 21:32:24.191983 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:24.191850 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb72104c-e8a8-4bf1-b48e-b5dbcb200d70-workload-socket\") pod \"cb72104c-e8a8-4bf1-b48e-b5dbcb200d70\" (UID: \"cb72104c-e8a8-4bf1-b48e-b5dbcb200d70\") " Jul 15 21:32:24.191983 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:24.191891 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb72104c-e8a8-4bf1-b48e-b5dbcb200d70-istio-token\") pod \"cb72104c-e8a8-4bf1-b48e-b5dbcb200d70\" (UID: \"cb72104c-e8a8-4bf1-b48e-b5dbcb200d70\") " Jul 15 21:32:24.191983 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:24.191942 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb72104c-e8a8-4bf1-b48e-b5dbcb200d70-istio-podinfo\") pod \"cb72104c-e8a8-4bf1-b48e-b5dbcb200d70\" (UID: \"cb72104c-e8a8-4bf1-b48e-b5dbcb200d70\") " Jul 15 21:32:24.191983 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:24.191973 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb72104c-e8a8-4bf1-b48e-b5dbcb200d70-istio-envoy\") pod \"cb72104c-e8a8-4bf1-b48e-b5dbcb200d70\" (UID: \"cb72104c-e8a8-4bf1-b48e-b5dbcb200d70\") " Jul 15 21:32:24.192337 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:24.191962 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb72104c-e8a8-4bf1-b48e-b5dbcb200d70-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cb72104c-e8a8-4bf1-b48e-b5dbcb200d70" (UID: "cb72104c-e8a8-4bf1-b48e-b5dbcb200d70"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:32:24.192337 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:24.192201 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb72104c-e8a8-4bf1-b48e-b5dbcb200d70-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cb72104c-e8a8-4bf1-b48e-b5dbcb200d70" (UID: "cb72104c-e8a8-4bf1-b48e-b5dbcb200d70"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:32:24.192450 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:24.192395 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb72104c-e8a8-4bf1-b48e-b5dbcb200d70-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cb72104c-e8a8-4bf1-b48e-b5dbcb200d70" (UID: "cb72104c-e8a8-4bf1-b48e-b5dbcb200d70"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:32:24.192704 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:24.192674 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb72104c-e8a8-4bf1-b48e-b5dbcb200d70-istio-data" (OuterVolumeSpecName: "istio-data") pod "cb72104c-e8a8-4bf1-b48e-b5dbcb200d70" (UID: "cb72104c-e8a8-4bf1-b48e-b5dbcb200d70"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:32:24.192957 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:24.192932 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb72104c-e8a8-4bf1-b48e-b5dbcb200d70-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cb72104c-e8a8-4bf1-b48e-b5dbcb200d70" (UID: "cb72104c-e8a8-4bf1-b48e-b5dbcb200d70"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 21:32:24.194977 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:24.194939 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb72104c-e8a8-4bf1-b48e-b5dbcb200d70-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cb72104c-e8a8-4bf1-b48e-b5dbcb200d70" (UID: "cb72104c-e8a8-4bf1-b48e-b5dbcb200d70"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:32:24.195106 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:24.194973 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cb72104c-e8a8-4bf1-b48e-b5dbcb200d70-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cb72104c-e8a8-4bf1-b48e-b5dbcb200d70" (UID: "cb72104c-e8a8-4bf1-b48e-b5dbcb200d70"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 21:32:24.195106 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:24.195024 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb72104c-e8a8-4bf1-b48e-b5dbcb200d70-istio-token" (OuterVolumeSpecName: "istio-token") pod "cb72104c-e8a8-4bf1-b48e-b5dbcb200d70" (UID: "cb72104c-e8a8-4bf1-b48e-b5dbcb200d70"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:32:24.195306 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:24.195276 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb72104c-e8a8-4bf1-b48e-b5dbcb200d70-kube-api-access-r9fpr" (OuterVolumeSpecName: "kube-api-access-r9fpr") pod "cb72104c-e8a8-4bf1-b48e-b5dbcb200d70" (UID: "cb72104c-e8a8-4bf1-b48e-b5dbcb200d70"). InnerVolumeSpecName "kube-api-access-r9fpr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:32:24.292688 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:24.292655 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb72104c-e8a8-4bf1-b48e-b5dbcb200d70-istiod-ca-cert\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:32:24.292688 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:24.292684 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r9fpr\" (UniqueName: \"kubernetes.io/projected/cb72104c-e8a8-4bf1-b48e-b5dbcb200d70-kube-api-access-r9fpr\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:32:24.292688 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:24.292693 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb72104c-e8a8-4bf1-b48e-b5dbcb200d70-workload-certs\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:32:24.292927 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:24.292702 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb72104c-e8a8-4bf1-b48e-b5dbcb200d70-workload-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:32:24.292927 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:24.292710 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb72104c-e8a8-4bf1-b48e-b5dbcb200d70-istio-token\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:32:24.292927 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:24.292719 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb72104c-e8a8-4bf1-b48e-b5dbcb200d70-istio-podinfo\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:32:24.292927 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:24.292726 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb72104c-e8a8-4bf1-b48e-b5dbcb200d70-istio-envoy\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:32:24.292927 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:24.292735 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb72104c-e8a8-4bf1-b48e-b5dbcb200d70-istio-data\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:32:24.292927 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:24.292744 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb72104c-e8a8-4bf1-b48e-b5dbcb200d70-credential-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:32:24.486516 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:24.486423 2579 generic.go:356] "Generic (PLEG): container finished" podID="cb72104c-e8a8-4bf1-b48e-b5dbcb200d70" containerID="f910a96538628ccb38026f950b604b2a66f0e17736fb0a0058738d4d797435e8" exitCode=0 Jul 15 21:32:24.486516 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:24.486466 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg" event={"ID":"cb72104c-e8a8-4bf1-b48e-b5dbcb200d70","Type":"ContainerDied","Data":"f910a96538628ccb38026f950b604b2a66f0e17736fb0a0058738d4d797435e8"} Jul 15 21:32:24.486516 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:24.486510 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg" event={"ID":"cb72104c-e8a8-4bf1-b48e-b5dbcb200d70","Type":"ContainerDied","Data":"ed1220bd0b8addf7b6732a9a7e1c30484f91c068a8a4e7f561a7653acb754927"} Jul 15 21:32:24.486723 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:24.486533 2579 scope.go:117] "RemoveContainer" containerID="f910a96538628ccb38026f950b604b2a66f0e17736fb0a0058738d4d797435e8" Jul 15 21:32:24.486723 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:24.486468 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg" Jul 15 21:32:24.496239 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:24.496222 2579 scope.go:117] "RemoveContainer" containerID="e0e126c42850886502e31ed680509bb05220d01fb9af093d12d9a0934e922c10" Jul 15 21:32:24.504644 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:24.504626 2579 scope.go:117] "RemoveContainer" containerID="f910a96538628ccb38026f950b604b2a66f0e17736fb0a0058738d4d797435e8" Jul 15 21:32:24.504895 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:32:24.504867 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f910a96538628ccb38026f950b604b2a66f0e17736fb0a0058738d4d797435e8\": container with ID starting with f910a96538628ccb38026f950b604b2a66f0e17736fb0a0058738d4d797435e8 not found: ID does not exist" containerID="f910a96538628ccb38026f950b604b2a66f0e17736fb0a0058738d4d797435e8" Jul 15 21:32:24.504950 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:24.504904 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f910a96538628ccb38026f950b604b2a66f0e17736fb0a0058738d4d797435e8"} err="failed to get container status \"f910a96538628ccb38026f950b604b2a66f0e17736fb0a0058738d4d797435e8\": rpc error: code = NotFound desc = could not find container \"f910a96538628ccb38026f950b604b2a66f0e17736fb0a0058738d4d797435e8\": container with ID starting with f910a96538628ccb38026f950b604b2a66f0e17736fb0a0058738d4d797435e8 not found: ID does not exist" Jul 15 21:32:24.504950 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:24.504921 2579 scope.go:117] "RemoveContainer" containerID="e0e126c42850886502e31ed680509bb05220d01fb9af093d12d9a0934e922c10" Jul 15 21:32:24.505184 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:32:24.505167 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0e126c42850886502e31ed680509bb05220d01fb9af093d12d9a0934e922c10\": container with ID starting with e0e126c42850886502e31ed680509bb05220d01fb9af093d12d9a0934e922c10 not found: ID does not exist" containerID="e0e126c42850886502e31ed680509bb05220d01fb9af093d12d9a0934e922c10" Jul 15 21:32:24.505251 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:24.505189 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0e126c42850886502e31ed680509bb05220d01fb9af093d12d9a0934e922c10"} err="failed to get container status \"e0e126c42850886502e31ed680509bb05220d01fb9af093d12d9a0934e922c10\": rpc error: code = NotFound desc = could not find container \"e0e126c42850886502e31ed680509bb05220d01fb9af093d12d9a0934e922c10\": container with ID starting with e0e126c42850886502e31ed680509bb05220d01fb9af093d12d9a0934e922c10 not found: ID does not exist" Jul 15 21:32:24.520810 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:24.520784 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg"] Jul 15 21:32:24.524788 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:24.524763 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-18bc3701-gw-openshift-default-744995f875-mnrdg"] Jul 15 21:32:25.106329 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:25.106290 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:32:25.591834 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:25.591789 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:32:25.997182 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:25.997144 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb72104c-e8a8-4bf1-b48e-b5dbcb200d70" path="/var/lib/kubelet/pods/cb72104c-e8a8-4bf1-b48e-b5dbcb200d70/volumes" Jul 15 21:32:26.087789 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:26.087749 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:32:32.260721 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:32.260683 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:32:32.413483 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:32.408444 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:32:32.616141 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:32.616106 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22"] Jul 15 21:32:32.616497 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:32.616484 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="794548fa-1781-456e-9d3e-39172e4cd4e4" containerName="istio-proxy" Jul 15 21:32:32.616539 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:32.616498 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="794548fa-1781-456e-9d3e-39172e4cd4e4" containerName="istio-proxy" Jul 15 21:32:32.616539 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:32.616508 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb72104c-e8a8-4bf1-b48e-b5dbcb200d70" containerName="istio-proxy" Jul 15 21:32:32.616539 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:32.616514 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb72104c-e8a8-4bf1-b48e-b5dbcb200d70" containerName="istio-proxy" Jul 15 21:32:32.616539 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:32.616523 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="794548fa-1781-456e-9d3e-39172e4cd4e4" containerName="istio-proxy" Jul 15 21:32:32.616539 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:32.616529 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="794548fa-1781-456e-9d3e-39172e4cd4e4" containerName="istio-proxy" Jul 15 21:32:32.616701 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:32.616597 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="794548fa-1781-456e-9d3e-39172e4cd4e4" containerName="istio-proxy" Jul 15 21:32:32.616701 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:32.616606 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb72104c-e8a8-4bf1-b48e-b5dbcb200d70" containerName="istio-proxy" Jul 15 21:32:32.616701 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:32.616613 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="794548fa-1781-456e-9d3e-39172e4cd4e4" containerName="istio-proxy" Jul 15 21:32:32.616701 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:32.616619 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb72104c-e8a8-4bf1-b48e-b5dbcb200d70" containerName="istio-proxy" Jul 15 21:32:32.621042 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:32.621025 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22" Jul 15 21:32:32.626345 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:32.626312 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-b0fcdc21-openshift-default-dockercfg-gsflz\"" Jul 15 21:32:32.656118 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:32.656082 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7423f6a4-c3dc-4188-881a-0587321f8662-istio-podinfo\") pod \"e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22\" (UID: \"7423f6a4-c3dc-4188-881a-0587321f8662\") " pod="openshift-ingress/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22" Jul 15 21:32:32.656255 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:32.656142 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7423f6a4-c3dc-4188-881a-0587321f8662-workload-certs\") pod \"e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22\" (UID: \"7423f6a4-c3dc-4188-881a-0587321f8662\") " pod="openshift-ingress/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22" Jul 15 21:32:32.656255 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:32.656184 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7423f6a4-c3dc-4188-881a-0587321f8662-workload-socket\") pod \"e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22\" (UID: \"7423f6a4-c3dc-4188-881a-0587321f8662\") " pod="openshift-ingress/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22" Jul 15 21:32:32.656346 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:32.656271 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7423f6a4-c3dc-4188-881a-0587321f8662-credential-socket\") pod \"e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22\" (UID: \"7423f6a4-c3dc-4188-881a-0587321f8662\") " pod="openshift-ingress/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22" Jul 15 21:32:32.656390 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:32.656372 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7423f6a4-c3dc-4188-881a-0587321f8662-istio-token\") pod \"e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22\" (UID: \"7423f6a4-c3dc-4188-881a-0587321f8662\") " pod="openshift-ingress/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22" Jul 15 21:32:32.656426 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:32.656405 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7423f6a4-c3dc-4188-881a-0587321f8662-istiod-ca-cert\") pod \"e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22\" (UID: \"7423f6a4-c3dc-4188-881a-0587321f8662\") " pod="openshift-ingress/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22" Jul 15 21:32:32.656469 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:32.656443 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7423f6a4-c3dc-4188-881a-0587321f8662-istio-envoy\") pod \"e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22\" (UID: \"7423f6a4-c3dc-4188-881a-0587321f8662\") " pod="openshift-ingress/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22" Jul 15 21:32:32.656509 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:32.656472 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j85q7\" (UniqueName: \"kubernetes.io/projected/7423f6a4-c3dc-4188-881a-0587321f8662-kube-api-access-j85q7\") pod \"e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22\" (UID: \"7423f6a4-c3dc-4188-881a-0587321f8662\") " pod="openshift-ingress/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22" Jul 15 21:32:32.656562 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:32.656515 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7423f6a4-c3dc-4188-881a-0587321f8662-istio-data\") pod \"e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22\" (UID: \"7423f6a4-c3dc-4188-881a-0587321f8662\") " pod="openshift-ingress/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22" Jul 15 21:32:32.675003 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:32.674970 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22"] Jul 15 21:32:32.714576 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:32.714542 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:32:32.757599 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:32.757561 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7423f6a4-c3dc-4188-881a-0587321f8662-istio-podinfo\") pod \"e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22\" (UID: \"7423f6a4-c3dc-4188-881a-0587321f8662\") " pod="openshift-ingress/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22" Jul 15 21:32:32.757797 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:32.757615 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7423f6a4-c3dc-4188-881a-0587321f8662-workload-certs\") pod \"e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22\" (UID: \"7423f6a4-c3dc-4188-881a-0587321f8662\") " pod="openshift-ingress/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22" Jul 15 21:32:32.757797 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:32.757649 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7423f6a4-c3dc-4188-881a-0587321f8662-workload-socket\") pod \"e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22\" (UID: \"7423f6a4-c3dc-4188-881a-0587321f8662\") " pod="openshift-ingress/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22" Jul 15 21:32:32.757797 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:32.757681 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7423f6a4-c3dc-4188-881a-0587321f8662-credential-socket\") pod \"e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22\" (UID: \"7423f6a4-c3dc-4188-881a-0587321f8662\") " pod="openshift-ingress/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22" Jul 15 21:32:32.757797 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:32.757725 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7423f6a4-c3dc-4188-881a-0587321f8662-istio-token\") pod \"e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22\" (UID: \"7423f6a4-c3dc-4188-881a-0587321f8662\") " pod="openshift-ingress/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22" Jul 15 21:32:32.757797 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:32.757746 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7423f6a4-c3dc-4188-881a-0587321f8662-istiod-ca-cert\") pod \"e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22\" (UID: \"7423f6a4-c3dc-4188-881a-0587321f8662\") " pod="openshift-ingress/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22" Jul 15 21:32:32.757797 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:32.757781 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7423f6a4-c3dc-4188-881a-0587321f8662-istio-envoy\") pod \"e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22\" (UID: \"7423f6a4-c3dc-4188-881a-0587321f8662\") " pod="openshift-ingress/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22" Jul 15 21:32:32.758157 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:32.757807 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j85q7\" (UniqueName: \"kubernetes.io/projected/7423f6a4-c3dc-4188-881a-0587321f8662-kube-api-access-j85q7\") pod \"e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22\" (UID: \"7423f6a4-c3dc-4188-881a-0587321f8662\") " pod="openshift-ingress/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22" Jul 15 21:32:32.758157 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:32.757851 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7423f6a4-c3dc-4188-881a-0587321f8662-istio-data\") pod \"e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22\" (UID: \"7423f6a4-c3dc-4188-881a-0587321f8662\") " pod="openshift-ingress/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22" Jul 15 21:32:32.758273 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:32.758165 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7423f6a4-c3dc-4188-881a-0587321f8662-workload-socket\") pod \"e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22\" (UID: \"7423f6a4-c3dc-4188-881a-0587321f8662\") " pod="openshift-ingress/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22" Jul 15 21:32:32.758273 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:32.758189 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7423f6a4-c3dc-4188-881a-0587321f8662-credential-socket\") pod \"e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22\" (UID: \"7423f6a4-c3dc-4188-881a-0587321f8662\") " pod="openshift-ingress/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22" Jul 15 21:32:32.758273 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:32.758247 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7423f6a4-c3dc-4188-881a-0587321f8662-workload-certs\") pod \"e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22\" (UID: \"7423f6a4-c3dc-4188-881a-0587321f8662\") " pod="openshift-ingress/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22" Jul 15 21:32:32.758411 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:32.758296 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7423f6a4-c3dc-4188-881a-0587321f8662-istio-data\") pod \"e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22\" (UID: \"7423f6a4-c3dc-4188-881a-0587321f8662\") " pod="openshift-ingress/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22" Jul 15 21:32:32.758635 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:32.758618 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7423f6a4-c3dc-4188-881a-0587321f8662-istiod-ca-cert\") pod \"e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22\" (UID: \"7423f6a4-c3dc-4188-881a-0587321f8662\") " pod="openshift-ingress/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22" Jul 15 21:32:32.760099 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:32.760080 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7423f6a4-c3dc-4188-881a-0587321f8662-istio-podinfo\") pod \"e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22\" (UID: \"7423f6a4-c3dc-4188-881a-0587321f8662\") " pod="openshift-ingress/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22" Jul 15 21:32:32.760231 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:32.760215 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7423f6a4-c3dc-4188-881a-0587321f8662-istio-envoy\") pod \"e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22\" (UID: \"7423f6a4-c3dc-4188-881a-0587321f8662\") " pod="openshift-ingress/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22" Jul 15 21:32:32.776240 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:32.776217 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j85q7\" (UniqueName: \"kubernetes.io/projected/7423f6a4-c3dc-4188-881a-0587321f8662-kube-api-access-j85q7\") pod \"e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22\" (UID: \"7423f6a4-c3dc-4188-881a-0587321f8662\") " pod="openshift-ingress/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22" Jul 15 21:32:32.777737 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:32.777720 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7423f6a4-c3dc-4188-881a-0587321f8662-istio-token\") pod \"e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22\" (UID: \"7423f6a4-c3dc-4188-881a-0587321f8662\") " pod="openshift-ingress/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22" Jul 15 21:32:32.931767 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:32.931735 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22" Jul 15 21:32:32.998252 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:32.998136 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:32:33.113662 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:32:33.113632 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7423f6a4_c3dc_4188_881a_0587321f8662.slice/crio-cebe8c0dd750fa76228db8a66817e287c78324a01ec6608bef27fa0c2f14631a WatchSource:0}: Error finding container cebe8c0dd750fa76228db8a66817e287c78324a01ec6608bef27fa0c2f14631a: Status 404 returned error can't find the container with id cebe8c0dd750fa76228db8a66817e287c78324a01ec6608bef27fa0c2f14631a Jul 15 21:32:33.115798 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:33.115767 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:32:33.115871 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:33.115831 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:32:33.115871 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:33.115860 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:32:33.116166 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:33.116140 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22"] Jul 15 21:32:33.539780 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:33.539736 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22" event={"ID":"7423f6a4-c3dc-4188-881a-0587321f8662","Type":"ContainerStarted","Data":"78a954741cbfa508ec7db58749e838de41e5c836f1a33b93c71529d4275cb7ef"} Jul 15 21:32:33.540296 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:33.539805 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22" event={"ID":"7423f6a4-c3dc-4188-881a-0587321f8662","Type":"ContainerStarted","Data":"cebe8c0dd750fa76228db8a66817e287c78324a01ec6608bef27fa0c2f14631a"} Jul 15 21:32:33.566640 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:33.566588 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22" podStartSLOduration=1.566571857 podStartE2EDuration="1.566571857s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:32:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:32:33.565144642 +0000 UTC m=+1858.106457193" watchObservedRunningTime="2026-07-15 21:32:33.566571857 +0000 UTC m=+1858.107884457" Jul 15 21:32:33.774771 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:33.774733 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:32:33.932300 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:33.932259 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22" Jul 15 21:32:34.244962 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:34.244875 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:32:34.363676 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:34.363640 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:32:34.468233 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:34.468194 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:32:34.600716 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:34.600628 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:32:34.762318 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:34.762274 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:32:34.936904 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:34.936876 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22" Jul 15 21:32:35.550498 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:35.550468 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22" Jul 15 21:32:35.551541 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:35.551524 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22" Jul 15 21:32:36.303742 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:36.303700 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:32:36.431732 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:36.431637 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:32:44.600456 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:44.600355 2579 generic.go:356] "Generic (PLEG): container finished" podID="7423f6a4-c3dc-4188-881a-0587321f8662" containerID="78a954741cbfa508ec7db58749e838de41e5c836f1a33b93c71529d4275cb7ef" exitCode=0 Jul 15 21:32:44.600456 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:44.600424 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22" event={"ID":"7423f6a4-c3dc-4188-881a-0587321f8662","Type":"ContainerDied","Data":"78a954741cbfa508ec7db58749e838de41e5c836f1a33b93c71529d4275cb7ef"} Jul 15 21:32:44.600891 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:44.600785 2579 scope.go:117] "RemoveContainer" containerID="78a954741cbfa508ec7db58749e838de41e5c836f1a33b93c71529d4275cb7ef" Jul 15 21:32:44.601823 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:44.601797 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:32:44.601895 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:44.601879 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:32:44.601938 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:44.601926 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:32:45.608850 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:45.608807 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22" event={"ID":"7423f6a4-c3dc-4188-881a-0587321f8662","Type":"ContainerStarted","Data":"c3eb2b4df55d37bbc969971002eb76bd705741499cdbb4185676495d698c650d"} Jul 15 21:32:45.934205 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:45.934098 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22" Jul 15 21:32:45.934396 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:45.934268 2579 patch_prober.go:28] interesting pod/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 15 21:32:45.934396 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:45.934327 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22" podUID="7423f6a4-c3dc-4188-881a-0587321f8662" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 15 21:32:46.933167 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:46.933126 2579 patch_prober.go:28] interesting pod/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 15 21:32:46.933683 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:46.933207 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22" podUID="7423f6a4-c3dc-4188-881a-0587321f8662" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 15 21:32:47.932496 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:47.932460 2579 patch_prober.go:28] interesting pod/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 15 21:32:47.933178 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:47.932777 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22" podUID="7423f6a4-c3dc-4188-881a-0587321f8662" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 15 21:32:48.933280 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:48.933234 2579 patch_prober.go:28] interesting pod/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 15 21:32:48.933801 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:48.933315 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22" podUID="7423f6a4-c3dc-4188-881a-0587321f8662" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 15 21:32:49.937215 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:49.937185 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22" Jul 15 21:32:49.937859 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:49.937445 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22" Jul 15 21:32:49.938366 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:32:49.938345 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22" Jul 15 21:33:39.665027 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:39.664988 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:33:42.693091 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:42.693027 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:33:43.939258 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:43.939220 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:33:50.083617 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:50.083577 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:33:50.192955 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:50.192917 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:33:54.979461 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:54.979424 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22"] Jul 15 21:33:54.979889 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:54.979658 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22" podUID="7423f6a4-c3dc-4188-881a-0587321f8662" containerName="istio-proxy" containerID="cri-o://c3eb2b4df55d37bbc969971002eb76bd705741499cdbb4185676495d698c650d" gracePeriod=30 Jul 15 21:33:55.002310 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:55.002280 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:33:55.293679 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:55.293592 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n"] Jul 15 21:33:55.294113 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:55.294095 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb72104c-e8a8-4bf1-b48e-b5dbcb200d70" containerName="istio-proxy" Jul 15 21:33:55.294113 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:55.294115 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb72104c-e8a8-4bf1-b48e-b5dbcb200d70" containerName="istio-proxy" Jul 15 21:33:55.297125 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:55.297100 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n" Jul 15 21:33:55.301598 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:55.301571 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-994ab52c-openshift-default-dockercfg-2rpdz\"" Jul 15 21:33:55.309590 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:55.309564 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n"] Jul 15 21:33:55.335507 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:55.335471 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:33:55.418636 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:55.418600 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4401cae4-e406-43e9-9ab0-735104ee0cad-workload-socket\") pod \"e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n\" (UID: \"4401cae4-e406-43e9-9ab0-735104ee0cad\") " pod="openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n" Jul 15 21:33:55.418801 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:55.418640 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4401cae4-e406-43e9-9ab0-735104ee0cad-istio-data\") pod \"e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n\" (UID: \"4401cae4-e406-43e9-9ab0-735104ee0cad\") " pod="openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n" Jul 15 21:33:55.418801 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:55.418663 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4401cae4-e406-43e9-9ab0-735104ee0cad-istio-podinfo\") pod \"e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n\" (UID: \"4401cae4-e406-43e9-9ab0-735104ee0cad\") " pod="openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n" Jul 15 21:33:55.418801 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:55.418688 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4401cae4-e406-43e9-9ab0-735104ee0cad-credential-socket\") pod \"e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n\" (UID: \"4401cae4-e406-43e9-9ab0-735104ee0cad\") " pod="openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n" Jul 15 21:33:55.418801 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:55.418718 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4401cae4-e406-43e9-9ab0-735104ee0cad-istio-token\") pod \"e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n\" (UID: \"4401cae4-e406-43e9-9ab0-735104ee0cad\") " pod="openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n" Jul 15 21:33:55.418801 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:55.418742 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brmrn\" (UniqueName: \"kubernetes.io/projected/4401cae4-e406-43e9-9ab0-735104ee0cad-kube-api-access-brmrn\") pod \"e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n\" (UID: \"4401cae4-e406-43e9-9ab0-735104ee0cad\") " pod="openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n" Jul 15 21:33:55.418973 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:55.418836 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4401cae4-e406-43e9-9ab0-735104ee0cad-istiod-ca-cert\") pod \"e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n\" (UID: \"4401cae4-e406-43e9-9ab0-735104ee0cad\") " pod="openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n" Jul 15 21:33:55.418973 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:55.418863 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4401cae4-e406-43e9-9ab0-735104ee0cad-workload-certs\") pod \"e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n\" (UID: \"4401cae4-e406-43e9-9ab0-735104ee0cad\") " pod="openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n" Jul 15 21:33:55.418973 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:55.418879 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4401cae4-e406-43e9-9ab0-735104ee0cad-istio-envoy\") pod \"e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n\" (UID: \"4401cae4-e406-43e9-9ab0-735104ee0cad\") " pod="openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n" Jul 15 21:33:55.519992 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:55.519945 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brmrn\" (UniqueName: \"kubernetes.io/projected/4401cae4-e406-43e9-9ab0-735104ee0cad-kube-api-access-brmrn\") pod \"e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n\" (UID: \"4401cae4-e406-43e9-9ab0-735104ee0cad\") " pod="openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n" Jul 15 21:33:55.520235 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:55.520051 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4401cae4-e406-43e9-9ab0-735104ee0cad-istiod-ca-cert\") pod \"e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n\" (UID: \"4401cae4-e406-43e9-9ab0-735104ee0cad\") " pod="openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n" Jul 15 21:33:55.520235 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:55.520113 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4401cae4-e406-43e9-9ab0-735104ee0cad-workload-certs\") pod \"e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n\" (UID: \"4401cae4-e406-43e9-9ab0-735104ee0cad\") " pod="openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n" Jul 15 21:33:55.520235 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:55.520136 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4401cae4-e406-43e9-9ab0-735104ee0cad-istio-envoy\") pod \"e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n\" (UID: \"4401cae4-e406-43e9-9ab0-735104ee0cad\") " pod="openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n" Jul 15 21:33:55.520235 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:55.520192 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4401cae4-e406-43e9-9ab0-735104ee0cad-workload-socket\") pod \"e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n\" (UID: \"4401cae4-e406-43e9-9ab0-735104ee0cad\") " pod="openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n" Jul 15 21:33:55.520235 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:55.520219 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4401cae4-e406-43e9-9ab0-735104ee0cad-istio-data\") pod \"e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n\" (UID: \"4401cae4-e406-43e9-9ab0-735104ee0cad\") " pod="openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n" Jul 15 21:33:55.520517 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:55.520363 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4401cae4-e406-43e9-9ab0-735104ee0cad-istio-podinfo\") pod \"e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n\" (UID: \"4401cae4-e406-43e9-9ab0-735104ee0cad\") " pod="openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n" Jul 15 21:33:55.520517 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:55.520404 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4401cae4-e406-43e9-9ab0-735104ee0cad-credential-socket\") pod \"e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n\" (UID: \"4401cae4-e406-43e9-9ab0-735104ee0cad\") " pod="openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n" Jul 15 21:33:55.520517 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:55.520440 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4401cae4-e406-43e9-9ab0-735104ee0cad-istio-token\") pod \"e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n\" (UID: \"4401cae4-e406-43e9-9ab0-735104ee0cad\") " pod="openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n" Jul 15 21:33:55.520656 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:55.520599 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4401cae4-e406-43e9-9ab0-735104ee0cad-workload-socket\") pod \"e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n\" (UID: \"4401cae4-e406-43e9-9ab0-735104ee0cad\") " pod="openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n" Jul 15 21:33:55.520789 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:55.520761 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4401cae4-e406-43e9-9ab0-735104ee0cad-istio-data\") pod \"e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n\" (UID: \"4401cae4-e406-43e9-9ab0-735104ee0cad\") " pod="openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n" Jul 15 21:33:55.520927 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:55.520897 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4401cae4-e406-43e9-9ab0-735104ee0cad-credential-socket\") pod \"e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n\" (UID: \"4401cae4-e406-43e9-9ab0-735104ee0cad\") " pod="openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n" Jul 15 21:33:55.520927 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:55.520862 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4401cae4-e406-43e9-9ab0-735104ee0cad-istiod-ca-cert\") pod \"e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n\" (UID: \"4401cae4-e406-43e9-9ab0-735104ee0cad\") " pod="openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n" Jul 15 21:33:55.521122 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:55.520961 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4401cae4-e406-43e9-9ab0-735104ee0cad-workload-certs\") pod \"e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n\" (UID: \"4401cae4-e406-43e9-9ab0-735104ee0cad\") " pod="openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n" Jul 15 21:33:55.522888 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:55.522867 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4401cae4-e406-43e9-9ab0-735104ee0cad-istio-envoy\") pod \"e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n\" (UID: \"4401cae4-e406-43e9-9ab0-735104ee0cad\") " pod="openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n" Jul 15 21:33:55.523039 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:55.523023 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4401cae4-e406-43e9-9ab0-735104ee0cad-istio-podinfo\") pod \"e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n\" (UID: \"4401cae4-e406-43e9-9ab0-735104ee0cad\") " pod="openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n" Jul 15 21:33:55.529474 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:55.529447 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4401cae4-e406-43e9-9ab0-735104ee0cad-istio-token\") pod \"e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n\" (UID: \"4401cae4-e406-43e9-9ab0-735104ee0cad\") " pod="openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n" Jul 15 21:33:55.529576 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:55.529563 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brmrn\" (UniqueName: \"kubernetes.io/projected/4401cae4-e406-43e9-9ab0-735104ee0cad-kube-api-access-brmrn\") pod \"e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n\" (UID: \"4401cae4-e406-43e9-9ab0-735104ee0cad\") " pod="openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n" Jul 15 21:33:55.609659 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:55.609621 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n" Jul 15 21:33:55.751191 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:55.751156 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n"] Jul 15 21:33:55.752370 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:33:55.752348 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4401cae4_e406_43e9_9ab0_735104ee0cad.slice/crio-cc98ec7706519e214375b2360e50e00ce70c50aa7dd1f97671e3f96af0b157ed WatchSource:0}: Error finding container cc98ec7706519e214375b2360e50e00ce70c50aa7dd1f97671e3f96af0b157ed: Status 404 returned error can't find the container with id cc98ec7706519e214375b2360e50e00ce70c50aa7dd1f97671e3f96af0b157ed Jul 15 21:33:55.757776 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:55.757703 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:33:55.757866 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:55.757830 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:33:55.757905 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:55.757869 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:33:55.984544 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:55.984445 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n" event={"ID":"4401cae4-e406-43e9-9ab0-735104ee0cad","Type":"ContainerStarted","Data":"0d055276aac13e39cc06b43b823e1e3649c0d625e9ab7027767b7716a9b3642a"} Jul 15 21:33:55.984544 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:55.984500 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n" event={"ID":"4401cae4-e406-43e9-9ab0-735104ee0cad","Type":"ContainerStarted","Data":"cc98ec7706519e214375b2360e50e00ce70c50aa7dd1f97671e3f96af0b157ed"} Jul 15 21:33:56.007116 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:56.007041 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n" podStartSLOduration=1.007020051 podStartE2EDuration="1.007020051s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:33:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:33:56.006426585 +0000 UTC m=+1940.547739136" watchObservedRunningTime="2026-07-15 21:33:56.007020051 +0000 UTC m=+1940.548332596" Jul 15 21:33:56.579095 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:56.579030 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:33:56.610040 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:56.610008 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n" Jul 15 21:33:56.615488 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:56.615381 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n" Jul 15 21:33:56.699919 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:56.699879 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:33:56.800563 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:56.800522 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:33:56.937862 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:56.937822 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:33:56.990648 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:56.990617 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n" Jul 15 21:33:56.991529 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:56.991512 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n" Jul 15 21:33:57.576045 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:57.576000 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:33:57.677988 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:33:57.677939 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:34:00.238996 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:00.238922 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22" Jul 15 21:34:00.368592 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:00.368560 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7423f6a4-c3dc-4188-881a-0587321f8662-workload-socket\") pod \"7423f6a4-c3dc-4188-881a-0587321f8662\" (UID: \"7423f6a4-c3dc-4188-881a-0587321f8662\") " Jul 15 21:34:00.368592 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:00.368603 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7423f6a4-c3dc-4188-881a-0587321f8662-credential-socket\") pod \"7423f6a4-c3dc-4188-881a-0587321f8662\" (UID: \"7423f6a4-c3dc-4188-881a-0587321f8662\") " Jul 15 21:34:00.368865 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:00.368640 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7423f6a4-c3dc-4188-881a-0587321f8662-istio-token\") pod \"7423f6a4-c3dc-4188-881a-0587321f8662\" (UID: \"7423f6a4-c3dc-4188-881a-0587321f8662\") " Jul 15 21:34:00.368865 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:00.368661 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7423f6a4-c3dc-4188-881a-0587321f8662-istio-podinfo\") pod \"7423f6a4-c3dc-4188-881a-0587321f8662\" (UID: \"7423f6a4-c3dc-4188-881a-0587321f8662\") " Jul 15 21:34:00.368865 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:00.368701 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j85q7\" (UniqueName: \"kubernetes.io/projected/7423f6a4-c3dc-4188-881a-0587321f8662-kube-api-access-j85q7\") pod \"7423f6a4-c3dc-4188-881a-0587321f8662\" (UID: \"7423f6a4-c3dc-4188-881a-0587321f8662\") " Jul 15 21:34:00.368865 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:00.368728 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7423f6a4-c3dc-4188-881a-0587321f8662-istio-data\") pod \"7423f6a4-c3dc-4188-881a-0587321f8662\" (UID: \"7423f6a4-c3dc-4188-881a-0587321f8662\") " Jul 15 21:34:00.368865 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:00.368757 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7423f6a4-c3dc-4188-881a-0587321f8662-istiod-ca-cert\") pod \"7423f6a4-c3dc-4188-881a-0587321f8662\" (UID: \"7423f6a4-c3dc-4188-881a-0587321f8662\") " Jul 15 21:34:00.368865 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:00.368783 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7423f6a4-c3dc-4188-881a-0587321f8662-istio-envoy\") pod \"7423f6a4-c3dc-4188-881a-0587321f8662\" (UID: \"7423f6a4-c3dc-4188-881a-0587321f8662\") " Jul 15 21:34:00.368865 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:00.368807 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7423f6a4-c3dc-4188-881a-0587321f8662-workload-certs\") pod \"7423f6a4-c3dc-4188-881a-0587321f8662\" (UID: \"7423f6a4-c3dc-4188-881a-0587321f8662\") " Jul 15 21:34:00.369397 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:00.368887 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7423f6a4-c3dc-4188-881a-0587321f8662-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7423f6a4-c3dc-4188-881a-0587321f8662" (UID: "7423f6a4-c3dc-4188-881a-0587321f8662"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:34:00.369397 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:00.369173 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7423f6a4-c3dc-4188-881a-0587321f8662-workload-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:34:00.369397 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:00.369311 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7423f6a4-c3dc-4188-881a-0587321f8662-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7423f6a4-c3dc-4188-881a-0587321f8662" (UID: "7423f6a4-c3dc-4188-881a-0587321f8662"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 21:34:00.369603 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:00.369454 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7423f6a4-c3dc-4188-881a-0587321f8662-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7423f6a4-c3dc-4188-881a-0587321f8662" (UID: "7423f6a4-c3dc-4188-881a-0587321f8662"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:34:00.369874 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:00.369844 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7423f6a4-c3dc-4188-881a-0587321f8662-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7423f6a4-c3dc-4188-881a-0587321f8662" (UID: "7423f6a4-c3dc-4188-881a-0587321f8662"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:34:00.370324 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:00.370297 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7423f6a4-c3dc-4188-881a-0587321f8662-istio-data" (OuterVolumeSpecName: "istio-data") pod "7423f6a4-c3dc-4188-881a-0587321f8662" (UID: "7423f6a4-c3dc-4188-881a-0587321f8662"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:34:00.371273 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:00.371249 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7423f6a4-c3dc-4188-881a-0587321f8662-kube-api-access-j85q7" (OuterVolumeSpecName: "kube-api-access-j85q7") pod "7423f6a4-c3dc-4188-881a-0587321f8662" (UID: "7423f6a4-c3dc-4188-881a-0587321f8662"). InnerVolumeSpecName "kube-api-access-j85q7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:34:00.371516 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:00.371491 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7423f6a4-c3dc-4188-881a-0587321f8662-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7423f6a4-c3dc-4188-881a-0587321f8662" (UID: "7423f6a4-c3dc-4188-881a-0587321f8662"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:34:00.371567 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:00.371552 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7423f6a4-c3dc-4188-881a-0587321f8662-istio-token" (OuterVolumeSpecName: "istio-token") pod "7423f6a4-c3dc-4188-881a-0587321f8662" (UID: "7423f6a4-c3dc-4188-881a-0587321f8662"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:34:00.371603 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:00.371570 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7423f6a4-c3dc-4188-881a-0587321f8662-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7423f6a4-c3dc-4188-881a-0587321f8662" (UID: "7423f6a4-c3dc-4188-881a-0587321f8662"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 21:34:00.469903 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:00.469859 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j85q7\" (UniqueName: \"kubernetes.io/projected/7423f6a4-c3dc-4188-881a-0587321f8662-kube-api-access-j85q7\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:34:00.469903 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:00.469896 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7423f6a4-c3dc-4188-881a-0587321f8662-istio-data\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:34:00.469903 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:00.469906 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7423f6a4-c3dc-4188-881a-0587321f8662-istiod-ca-cert\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:34:00.469903 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:00.469914 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7423f6a4-c3dc-4188-881a-0587321f8662-istio-envoy\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:34:00.470233 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:00.469922 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7423f6a4-c3dc-4188-881a-0587321f8662-workload-certs\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:34:00.470233 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:00.469930 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7423f6a4-c3dc-4188-881a-0587321f8662-credential-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:34:00.470233 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:00.469938 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7423f6a4-c3dc-4188-881a-0587321f8662-istio-token\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:34:00.470233 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:00.469946 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7423f6a4-c3dc-4188-881a-0587321f8662-istio-podinfo\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:34:01.015213 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:01.015173 2579 generic.go:356] "Generic (PLEG): container finished" podID="7423f6a4-c3dc-4188-881a-0587321f8662" containerID="c3eb2b4df55d37bbc969971002eb76bd705741499cdbb4185676495d698c650d" exitCode=0 Jul 15 21:34:01.015404 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:01.015233 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22" Jul 15 21:34:01.015404 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:01.015233 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22" event={"ID":"7423f6a4-c3dc-4188-881a-0587321f8662","Type":"ContainerDied","Data":"c3eb2b4df55d37bbc969971002eb76bd705741499cdbb4185676495d698c650d"} Jul 15 21:34:01.015404 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:01.015289 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22" event={"ID":"7423f6a4-c3dc-4188-881a-0587321f8662","Type":"ContainerDied","Data":"cebe8c0dd750fa76228db8a66817e287c78324a01ec6608bef27fa0c2f14631a"} Jul 15 21:34:01.015404 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:01.015321 2579 scope.go:117] "RemoveContainer" containerID="c3eb2b4df55d37bbc969971002eb76bd705741499cdbb4185676495d698c650d" Jul 15 21:34:01.025381 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:01.025356 2579 scope.go:117] "RemoveContainer" containerID="78a954741cbfa508ec7db58749e838de41e5c836f1a33b93c71529d4275cb7ef" Jul 15 21:34:01.035481 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:01.035457 2579 scope.go:117] "RemoveContainer" containerID="c3eb2b4df55d37bbc969971002eb76bd705741499cdbb4185676495d698c650d" Jul 15 21:34:01.035823 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:34:01.035793 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3eb2b4df55d37bbc969971002eb76bd705741499cdbb4185676495d698c650d\": container with ID starting with c3eb2b4df55d37bbc969971002eb76bd705741499cdbb4185676495d698c650d not found: ID does not exist" containerID="c3eb2b4df55d37bbc969971002eb76bd705741499cdbb4185676495d698c650d" Jul 15 21:34:01.035953 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:01.035831 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3eb2b4df55d37bbc969971002eb76bd705741499cdbb4185676495d698c650d"} err="failed to get container status \"c3eb2b4df55d37bbc969971002eb76bd705741499cdbb4185676495d698c650d\": rpc error: code = NotFound desc = could not find container \"c3eb2b4df55d37bbc969971002eb76bd705741499cdbb4185676495d698c650d\": container with ID starting with c3eb2b4df55d37bbc969971002eb76bd705741499cdbb4185676495d698c650d not found: ID does not exist" Jul 15 21:34:01.035953 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:01.035855 2579 scope.go:117] "RemoveContainer" containerID="78a954741cbfa508ec7db58749e838de41e5c836f1a33b93c71529d4275cb7ef" Jul 15 21:34:01.036142 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:34:01.036124 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78a954741cbfa508ec7db58749e838de41e5c836f1a33b93c71529d4275cb7ef\": container with ID starting with 78a954741cbfa508ec7db58749e838de41e5c836f1a33b93c71529d4275cb7ef not found: ID does not exist" containerID="78a954741cbfa508ec7db58749e838de41e5c836f1a33b93c71529d4275cb7ef" Jul 15 21:34:01.036201 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:01.036148 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78a954741cbfa508ec7db58749e838de41e5c836f1a33b93c71529d4275cb7ef"} err="failed to get container status \"78a954741cbfa508ec7db58749e838de41e5c836f1a33b93c71529d4275cb7ef\": rpc error: code = NotFound desc = could not find container \"78a954741cbfa508ec7db58749e838de41e5c836f1a33b93c71529d4275cb7ef\": container with ID starting with 78a954741cbfa508ec7db58749e838de41e5c836f1a33b93c71529d4275cb7ef not found: ID does not exist" Jul 15 21:34:01.039690 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:01.039662 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22"] Jul 15 21:34:01.043374 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:01.043348 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-b0fcdc21-openshift-default-5d859ff4cc-8ql22"] Jul 15 21:34:01.996856 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:01.996812 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7423f6a4-c3dc-4188-881a-0587321f8662" path="/var/lib/kubelet/pods/7423f6a4-c3dc-4188-881a-0587321f8662/volumes" Jul 15 21:34:04.987003 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:04.986963 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:34:11.409350 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:11.409309 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:34:11.511792 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:11.511746 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:34:12.935727 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:12.935410 2579 patch_prober.go:28] interesting pod/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": read tcp 10.132.0.2:33078->10.132.0.49:15021: read: connection reset by peer" start-of-body= Jul 15 21:34:12.935727 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:12.935509 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n" podUID="4401cae4-e406-43e9-9ab0-735104ee0cad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": read tcp 10.132.0.2:33078->10.132.0.49:15021: read: connection reset by peer" Jul 15 21:34:13.090341 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:13.090257 2579 generic.go:356] "Generic (PLEG): container finished" podID="4401cae4-e406-43e9-9ab0-735104ee0cad" containerID="0d055276aac13e39cc06b43b823e1e3649c0d625e9ab7027767b7716a9b3642a" exitCode=0 Jul 15 21:34:13.090341 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:13.090332 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n" event={"ID":"4401cae4-e406-43e9-9ab0-735104ee0cad","Type":"ContainerDied","Data":"0d055276aac13e39cc06b43b823e1e3649c0d625e9ab7027767b7716a9b3642a"} Jul 15 21:34:13.090782 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:13.090762 2579 scope.go:117] "RemoveContainer" containerID="0d055276aac13e39cc06b43b823e1e3649c0d625e9ab7027767b7716a9b3642a" Jul 15 21:34:13.091891 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:13.091861 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:34:13.091957 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:13.091941 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:34:13.092000 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:13.091972 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:34:14.098288 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:14.098246 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n" event={"ID":"4401cae4-e406-43e9-9ab0-735104ee0cad","Type":"ContainerStarted","Data":"9c03c838add3acfd560da6c59f17db3a66037734aa3a5e53c86d9970eaa39ca0"} Jul 15 21:34:14.610308 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:14.610267 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n" Jul 15 21:34:14.610560 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:14.610535 2579 patch_prober.go:28] interesting pod/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 15 21:34:14.610652 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:14.610595 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n" podUID="4401cae4-e406-43e9-9ab0-735104ee0cad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 15 21:34:15.032023 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:15.031892 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:34:15.332883 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:15.332783 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:34:15.401600 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:15.401560 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:34:15.611085 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:15.611027 2579 patch_prober.go:28] interesting pod/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 15 21:34:15.611256 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:15.611125 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n" podUID="4401cae4-e406-43e9-9ab0-735104ee0cad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 15 21:34:16.610040 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:16.610003 2579 patch_prober.go:28] interesting pod/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 15 21:34:16.610529 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:16.610055 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n" podUID="4401cae4-e406-43e9-9ab0-735104ee0cad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 15 21:34:17.610338 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:17.610305 2579 patch_prober.go:28] interesting pod/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 15 21:34:17.610735 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:17.610368 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n" podUID="4401cae4-e406-43e9-9ab0-735104ee0cad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 15 21:34:18.613893 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:18.613856 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n" Jul 15 21:34:18.614299 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:18.614096 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n" Jul 15 21:34:18.614810 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:34:18.614789 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n" Jul 15 21:35:01.417998 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:01.417956 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:35:03.988053 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:03.988017 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:35:06.011952 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:06.011918 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:35:12.200753 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:12.200716 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:35:12.300755 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:12.300701 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:35:16.198689 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:16.198650 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n"] Jul 15 21:35:16.199206 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:16.198934 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n" podUID="4401cae4-e406-43e9-9ab0-735104ee0cad" containerName="istio-proxy" containerID="cri-o://9c03c838add3acfd560da6c59f17db3a66037734aa3a5e53c86d9970eaa39ca0" gracePeriod=30 Jul 15 21:35:16.230747 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:16.230713 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:35:16.819670 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:16.819634 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:35:16.921357 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:16.921308 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:35:17.022989 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:17.022953 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:35:18.550795 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:18.550762 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:35:18.616905 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:18.616872 2579 patch_prober.go:28] interesting pod/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 15 21:35:18.617086 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:18.616926 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n" podUID="4401cae4-e406-43e9-9ab0-735104ee0cad" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 21:35:21.460730 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:21.460658 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n" Jul 15 21:35:21.464702 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:21.464662 2579 generic.go:356] "Generic (PLEG): container finished" podID="4401cae4-e406-43e9-9ab0-735104ee0cad" containerID="9c03c838add3acfd560da6c59f17db3a66037734aa3a5e53c86d9970eaa39ca0" exitCode=0 Jul 15 21:35:21.464847 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:21.464712 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n" Jul 15 21:35:21.464847 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:21.464748 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n" event={"ID":"4401cae4-e406-43e9-9ab0-735104ee0cad","Type":"ContainerDied","Data":"9c03c838add3acfd560da6c59f17db3a66037734aa3a5e53c86d9970eaa39ca0"} Jul 15 21:35:21.464847 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:21.464804 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n" event={"ID":"4401cae4-e406-43e9-9ab0-735104ee0cad","Type":"ContainerDied","Data":"cc98ec7706519e214375b2360e50e00ce70c50aa7dd1f97671e3f96af0b157ed"} Jul 15 21:35:21.464847 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:21.464837 2579 scope.go:117] "RemoveContainer" containerID="9c03c838add3acfd560da6c59f17db3a66037734aa3a5e53c86d9970eaa39ca0" Jul 15 21:35:21.473743 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:21.473724 2579 scope.go:117] "RemoveContainer" containerID="0d055276aac13e39cc06b43b823e1e3649c0d625e9ab7027767b7716a9b3642a" Jul 15 21:35:21.483260 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:21.483228 2579 scope.go:117] "RemoveContainer" containerID="9c03c838add3acfd560da6c59f17db3a66037734aa3a5e53c86d9970eaa39ca0" Jul 15 21:35:21.484047 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:35:21.484008 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c03c838add3acfd560da6c59f17db3a66037734aa3a5e53c86d9970eaa39ca0\": container with ID starting with 9c03c838add3acfd560da6c59f17db3a66037734aa3a5e53c86d9970eaa39ca0 not found: ID does not exist" containerID="9c03c838add3acfd560da6c59f17db3a66037734aa3a5e53c86d9970eaa39ca0" Jul 15 21:35:21.484186 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:21.484050 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c03c838add3acfd560da6c59f17db3a66037734aa3a5e53c86d9970eaa39ca0"} err="failed to get container status \"9c03c838add3acfd560da6c59f17db3a66037734aa3a5e53c86d9970eaa39ca0\": rpc error: code = NotFound desc = could not find container \"9c03c838add3acfd560da6c59f17db3a66037734aa3a5e53c86d9970eaa39ca0\": container with ID starting with 9c03c838add3acfd560da6c59f17db3a66037734aa3a5e53c86d9970eaa39ca0 not found: ID does not exist" Jul 15 21:35:21.484186 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:21.484098 2579 scope.go:117] "RemoveContainer" containerID="0d055276aac13e39cc06b43b823e1e3649c0d625e9ab7027767b7716a9b3642a" Jul 15 21:35:21.484519 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:35:21.484494 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d055276aac13e39cc06b43b823e1e3649c0d625e9ab7027767b7716a9b3642a\": container with ID starting with 0d055276aac13e39cc06b43b823e1e3649c0d625e9ab7027767b7716a9b3642a not found: ID does not exist" containerID="0d055276aac13e39cc06b43b823e1e3649c0d625e9ab7027767b7716a9b3642a" Jul 15 21:35:21.484622 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:21.484528 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d055276aac13e39cc06b43b823e1e3649c0d625e9ab7027767b7716a9b3642a"} err="failed to get container status \"0d055276aac13e39cc06b43b823e1e3649c0d625e9ab7027767b7716a9b3642a\": rpc error: code = NotFound desc = could not find container \"0d055276aac13e39cc06b43b823e1e3649c0d625e9ab7027767b7716a9b3642a\": container with ID starting with 0d055276aac13e39cc06b43b823e1e3649c0d625e9ab7027767b7716a9b3642a not found: ID does not exist" Jul 15 21:35:21.569122 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:21.569090 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4401cae4-e406-43e9-9ab0-735104ee0cad-istio-data\") pod \"4401cae4-e406-43e9-9ab0-735104ee0cad\" (UID: \"4401cae4-e406-43e9-9ab0-735104ee0cad\") " Jul 15 21:35:21.569325 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:21.569157 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4401cae4-e406-43e9-9ab0-735104ee0cad-workload-socket\") pod \"4401cae4-e406-43e9-9ab0-735104ee0cad\" (UID: \"4401cae4-e406-43e9-9ab0-735104ee0cad\") " Jul 15 21:35:21.569325 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:21.569191 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-brmrn\" (UniqueName: \"kubernetes.io/projected/4401cae4-e406-43e9-9ab0-735104ee0cad-kube-api-access-brmrn\") pod \"4401cae4-e406-43e9-9ab0-735104ee0cad\" (UID: \"4401cae4-e406-43e9-9ab0-735104ee0cad\") " Jul 15 21:35:21.569325 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:21.569236 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4401cae4-e406-43e9-9ab0-735104ee0cad-workload-certs\") pod \"4401cae4-e406-43e9-9ab0-735104ee0cad\" (UID: \"4401cae4-e406-43e9-9ab0-735104ee0cad\") " Jul 15 21:35:21.569528 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:21.569375 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4401cae4-e406-43e9-9ab0-735104ee0cad-credential-socket\") pod \"4401cae4-e406-43e9-9ab0-735104ee0cad\" (UID: \"4401cae4-e406-43e9-9ab0-735104ee0cad\") " Jul 15 21:35:21.569528 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:21.569417 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4401cae4-e406-43e9-9ab0-735104ee0cad-istio-token\") pod \"4401cae4-e406-43e9-9ab0-735104ee0cad\" (UID: \"4401cae4-e406-43e9-9ab0-735104ee0cad\") " Jul 15 21:35:21.569528 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:21.569441 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4401cae4-e406-43e9-9ab0-735104ee0cad-istiod-ca-cert\") pod \"4401cae4-e406-43e9-9ab0-735104ee0cad\" (UID: \"4401cae4-e406-43e9-9ab0-735104ee0cad\") " Jul 15 21:35:21.569528 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:21.569450 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4401cae4-e406-43e9-9ab0-735104ee0cad-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4401cae4-e406-43e9-9ab0-735104ee0cad" (UID: "4401cae4-e406-43e9-9ab0-735104ee0cad"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:35:21.569528 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:21.569499 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4401cae4-e406-43e9-9ab0-735104ee0cad-istio-podinfo\") pod \"4401cae4-e406-43e9-9ab0-735104ee0cad\" (UID: \"4401cae4-e406-43e9-9ab0-735104ee0cad\") " Jul 15 21:35:21.569528 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:21.569524 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4401cae4-e406-43e9-9ab0-735104ee0cad-istio-envoy\") pod \"4401cae4-e406-43e9-9ab0-735104ee0cad\" (UID: \"4401cae4-e406-43e9-9ab0-735104ee0cad\") " Jul 15 21:35:21.569851 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:21.569533 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4401cae4-e406-43e9-9ab0-735104ee0cad-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4401cae4-e406-43e9-9ab0-735104ee0cad" (UID: "4401cae4-e406-43e9-9ab0-735104ee0cad"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:35:21.569851 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:21.569616 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4401cae4-e406-43e9-9ab0-735104ee0cad-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4401cae4-e406-43e9-9ab0-735104ee0cad" (UID: "4401cae4-e406-43e9-9ab0-735104ee0cad"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:35:21.569851 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:21.569838 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4401cae4-e406-43e9-9ab0-735104ee0cad-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4401cae4-e406-43e9-9ab0-735104ee0cad" (UID: "4401cae4-e406-43e9-9ab0-735104ee0cad"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 21:35:21.570076 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:21.569856 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4401cae4-e406-43e9-9ab0-735104ee0cad-workload-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:35:21.570076 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:21.569875 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4401cae4-e406-43e9-9ab0-735104ee0cad-workload-certs\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:35:21.570076 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:21.569890 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4401cae4-e406-43e9-9ab0-735104ee0cad-credential-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:35:21.570076 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:21.570029 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4401cae4-e406-43e9-9ab0-735104ee0cad-istio-data" (OuterVolumeSpecName: "istio-data") pod "4401cae4-e406-43e9-9ab0-735104ee0cad" (UID: "4401cae4-e406-43e9-9ab0-735104ee0cad"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:35:21.571774 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:21.571743 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4401cae4-e406-43e9-9ab0-735104ee0cad-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4401cae4-e406-43e9-9ab0-735104ee0cad" (UID: "4401cae4-e406-43e9-9ab0-735104ee0cad"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:35:21.571898 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:21.571801 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4401cae4-e406-43e9-9ab0-735104ee0cad-kube-api-access-brmrn" (OuterVolumeSpecName: "kube-api-access-brmrn") pod "4401cae4-e406-43e9-9ab0-735104ee0cad" (UID: "4401cae4-e406-43e9-9ab0-735104ee0cad"). InnerVolumeSpecName "kube-api-access-brmrn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:35:21.572024 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:21.572004 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4401cae4-e406-43e9-9ab0-735104ee0cad-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4401cae4-e406-43e9-9ab0-735104ee0cad" (UID: "4401cae4-e406-43e9-9ab0-735104ee0cad"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 21:35:21.572024 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:21.572007 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4401cae4-e406-43e9-9ab0-735104ee0cad-istio-token" (OuterVolumeSpecName: "istio-token") pod "4401cae4-e406-43e9-9ab0-735104ee0cad" (UID: "4401cae4-e406-43e9-9ab0-735104ee0cad"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:35:21.670796 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:21.670741 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-brmrn\" (UniqueName: \"kubernetes.io/projected/4401cae4-e406-43e9-9ab0-735104ee0cad-kube-api-access-brmrn\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:35:21.670796 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:21.670793 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4401cae4-e406-43e9-9ab0-735104ee0cad-istio-token\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:35:21.670796 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:21.670803 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4401cae4-e406-43e9-9ab0-735104ee0cad-istiod-ca-cert\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:35:21.670796 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:21.670812 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4401cae4-e406-43e9-9ab0-735104ee0cad-istio-podinfo\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:35:21.671109 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:21.670820 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4401cae4-e406-43e9-9ab0-735104ee0cad-istio-envoy\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:35:21.671109 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:21.670828 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4401cae4-e406-43e9-9ab0-735104ee0cad-istio-data\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:35:21.790090 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:21.790042 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n"] Jul 15 21:35:21.793645 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:21.793619 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-994ab52c-openshift-default-7587b69775-vsz9n"] Jul 15 21:35:21.998174 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:21.998138 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4401cae4-e406-43e9-9ab0-735104ee0cad" path="/var/lib/kubelet/pods/4401cae4-e406-43e9-9ab0-735104ee0cad/volumes" Jul 15 21:35:23.847337 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:23.847297 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:35:24.832482 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:24.832439 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:35:24.936398 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:24.936354 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:35:25.030594 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:25.030560 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:35:25.131788 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:25.131728 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:35:25.494703 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:25.494604 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:35:54.518022 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:35:54.517988 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:36:00.370623 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:00.370537 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:36:00.439839 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:00.439799 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:36:00.556915 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:00.556884 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:36:16.447610 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:16.447573 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:36:18.729928 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:18.729897 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:36:18.825921 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:18.825886 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:36:22.615989 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:22.615958 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:36:22.748979 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:22.748945 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:36:22.969126 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:22.969002 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:36:33.403271 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:33.403232 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:36:34.208080 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:34.208029 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:36:36.128678 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:36.128652 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m5r2m_0db09c1e-d24d-4519-93c8-c185018c3d24/ovn-acl-logging/0.log" Jul 15 21:36:36.130666 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:36.130644 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m5r2m_0db09c1e-d24d-4519-93c8-c185018c3d24/ovn-acl-logging/0.log" Jul 15 21:36:40.748490 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:40.748457 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:36:40.837001 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:40.836966 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:36:40.978811 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:40.978768 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:36:41.076014 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:41.075919 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:36:41.887131 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:41.887084 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:36:41.977955 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:41.977921 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:36:42.082051 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:42.082017 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:36:43.758052 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:43.758016 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:36:44.199155 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:44.199120 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:36:49.934830 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:49.934791 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:36:50.001863 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:50.001830 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:36:50.614581 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:50.614543 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:36:53.782303 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:53.782265 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:36:56.963261 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:56.963218 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:36:57.176885 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:57.176851 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:36:57.307373 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:57.307284 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:36:57.583486 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:57.583391 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs"] Jul 15 21:36:57.583833 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:57.583814 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7423f6a4-c3dc-4188-881a-0587321f8662" containerName="istio-proxy" Jul 15 21:36:57.583930 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:57.583837 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7423f6a4-c3dc-4188-881a-0587321f8662" containerName="istio-proxy" Jul 15 21:36:57.583930 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:57.583855 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7423f6a4-c3dc-4188-881a-0587321f8662" containerName="istio-proxy" Jul 15 21:36:57.583930 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:57.583864 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7423f6a4-c3dc-4188-881a-0587321f8662" containerName="istio-proxy" Jul 15 21:36:57.583930 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:57.583875 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4401cae4-e406-43e9-9ab0-735104ee0cad" containerName="istio-proxy" Jul 15 21:36:57.583930 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:57.583885 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="4401cae4-e406-43e9-9ab0-735104ee0cad" containerName="istio-proxy" Jul 15 21:36:57.583930 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:57.583909 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4401cae4-e406-43e9-9ab0-735104ee0cad" containerName="istio-proxy" Jul 15 21:36:57.583930 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:57.583918 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="4401cae4-e406-43e9-9ab0-735104ee0cad" containerName="istio-proxy" Jul 15 21:36:57.584325 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:57.584018 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7423f6a4-c3dc-4188-881a-0587321f8662" containerName="istio-proxy" Jul 15 21:36:57.584325 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:57.584030 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="4401cae4-e406-43e9-9ab0-735104ee0cad" containerName="istio-proxy" Jul 15 21:36:57.584325 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:57.584041 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7423f6a4-c3dc-4188-881a-0587321f8662" containerName="istio-proxy" Jul 15 21:36:57.584325 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:57.584054 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="4401cae4-e406-43e9-9ab0-735104ee0cad" containerName="istio-proxy" Jul 15 21:36:57.587356 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:57.587334 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs" Jul 15 21:36:57.591365 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:57.591340 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-0af6f2e9-openshift-default-dockercfg-hnfgd\"" Jul 15 21:36:57.606316 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:57.606289 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:36:57.612872 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:57.612847 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs"] Jul 15 21:36:57.678971 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:57.678939 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d45071be-ff25-4f66-98f0-117316d826b2-istio-data\") pod \"e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs\" (UID: \"d45071be-ff25-4f66-98f0-117316d826b2\") " pod="openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs" Jul 15 21:36:57.678971 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:57.678979 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d45071be-ff25-4f66-98f0-117316d826b2-istio-podinfo\") pod \"e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs\" (UID: \"d45071be-ff25-4f66-98f0-117316d826b2\") " pod="openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs" Jul 15 21:36:57.679268 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:57.679024 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d45071be-ff25-4f66-98f0-117316d826b2-istio-token\") pod \"e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs\" (UID: \"d45071be-ff25-4f66-98f0-117316d826b2\") " pod="openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs" Jul 15 21:36:57.679268 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:57.679096 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d45071be-ff25-4f66-98f0-117316d826b2-workload-socket\") pod \"e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs\" (UID: \"d45071be-ff25-4f66-98f0-117316d826b2\") " pod="openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs" Jul 15 21:36:57.679268 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:57.679130 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d45071be-ff25-4f66-98f0-117316d826b2-workload-certs\") pod \"e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs\" (UID: \"d45071be-ff25-4f66-98f0-117316d826b2\") " pod="openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs" Jul 15 21:36:57.679268 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:57.679173 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d45071be-ff25-4f66-98f0-117316d826b2-credential-socket\") pod \"e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs\" (UID: \"d45071be-ff25-4f66-98f0-117316d826b2\") " pod="openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs" Jul 15 21:36:57.679268 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:57.679244 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w89md\" (UniqueName: \"kubernetes.io/projected/d45071be-ff25-4f66-98f0-117316d826b2-kube-api-access-w89md\") pod \"e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs\" (UID: \"d45071be-ff25-4f66-98f0-117316d826b2\") " pod="openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs" Jul 15 21:36:57.679458 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:57.679293 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d45071be-ff25-4f66-98f0-117316d826b2-istio-envoy\") pod \"e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs\" (UID: \"d45071be-ff25-4f66-98f0-117316d826b2\") " pod="openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs" Jul 15 21:36:57.679458 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:57.679322 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d45071be-ff25-4f66-98f0-117316d826b2-istiod-ca-cert\") pod \"e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs\" (UID: \"d45071be-ff25-4f66-98f0-117316d826b2\") " pod="openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs" Jul 15 21:36:57.780452 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:57.780408 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w89md\" (UniqueName: \"kubernetes.io/projected/d45071be-ff25-4f66-98f0-117316d826b2-kube-api-access-w89md\") pod \"e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs\" (UID: \"d45071be-ff25-4f66-98f0-117316d826b2\") " pod="openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs" Jul 15 21:36:57.780452 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:57.780453 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d45071be-ff25-4f66-98f0-117316d826b2-istio-envoy\") pod \"e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs\" (UID: \"d45071be-ff25-4f66-98f0-117316d826b2\") " pod="openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs" Jul 15 21:36:57.780716 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:57.780567 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d45071be-ff25-4f66-98f0-117316d826b2-istiod-ca-cert\") pod \"e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs\" (UID: \"d45071be-ff25-4f66-98f0-117316d826b2\") " pod="openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs" Jul 15 21:36:57.780716 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:57.780626 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d45071be-ff25-4f66-98f0-117316d826b2-istio-data\") pod \"e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs\" (UID: \"d45071be-ff25-4f66-98f0-117316d826b2\") " pod="openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs" Jul 15 21:36:57.780716 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:57.780657 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d45071be-ff25-4f66-98f0-117316d826b2-istio-podinfo\") pod \"e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs\" (UID: \"d45071be-ff25-4f66-98f0-117316d826b2\") " pod="openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs" Jul 15 21:36:57.780716 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:57.780707 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d45071be-ff25-4f66-98f0-117316d826b2-istio-token\") pod \"e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs\" (UID: \"d45071be-ff25-4f66-98f0-117316d826b2\") " pod="openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs" Jul 15 21:36:57.780936 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:57.780757 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d45071be-ff25-4f66-98f0-117316d826b2-workload-socket\") pod \"e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs\" (UID: \"d45071be-ff25-4f66-98f0-117316d826b2\") " pod="openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs" Jul 15 21:36:57.780936 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:57.780788 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d45071be-ff25-4f66-98f0-117316d826b2-workload-certs\") pod \"e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs\" (UID: \"d45071be-ff25-4f66-98f0-117316d826b2\") " pod="openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs" Jul 15 21:36:57.780936 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:57.780839 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d45071be-ff25-4f66-98f0-117316d826b2-credential-socket\") pod \"e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs\" (UID: \"d45071be-ff25-4f66-98f0-117316d826b2\") " pod="openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs" Jul 15 21:36:57.781112 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:57.781024 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d45071be-ff25-4f66-98f0-117316d826b2-istio-data\") pod \"e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs\" (UID: \"d45071be-ff25-4f66-98f0-117316d826b2\") " pod="openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs" Jul 15 21:36:57.781183 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:57.781149 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d45071be-ff25-4f66-98f0-117316d826b2-workload-socket\") pod \"e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs\" (UID: \"d45071be-ff25-4f66-98f0-117316d826b2\") " pod="openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs" Jul 15 21:36:57.781244 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:57.781196 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d45071be-ff25-4f66-98f0-117316d826b2-credential-socket\") pod \"e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs\" (UID: \"d45071be-ff25-4f66-98f0-117316d826b2\") " pod="openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs" Jul 15 21:36:57.781327 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:57.781304 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d45071be-ff25-4f66-98f0-117316d826b2-workload-certs\") pod \"e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs\" (UID: \"d45071be-ff25-4f66-98f0-117316d826b2\") " pod="openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs" Jul 15 21:36:57.781438 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:57.781415 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d45071be-ff25-4f66-98f0-117316d826b2-istiod-ca-cert\") pod \"e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs\" (UID: \"d45071be-ff25-4f66-98f0-117316d826b2\") " pod="openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs" Jul 15 21:36:57.782880 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:57.782859 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d45071be-ff25-4f66-98f0-117316d826b2-istio-envoy\") pod \"e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs\" (UID: \"d45071be-ff25-4f66-98f0-117316d826b2\") " pod="openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs" Jul 15 21:36:57.783249 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:57.783229 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d45071be-ff25-4f66-98f0-117316d826b2-istio-podinfo\") pod \"e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs\" (UID: \"d45071be-ff25-4f66-98f0-117316d826b2\") " pod="openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs" Jul 15 21:36:57.790652 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:57.790565 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:36:57.790652 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:57.790608 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d45071be-ff25-4f66-98f0-117316d826b2-istio-token\") pod \"e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs\" (UID: \"d45071be-ff25-4f66-98f0-117316d826b2\") " pod="openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs" Jul 15 21:36:57.790833 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:57.790744 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w89md\" (UniqueName: \"kubernetes.io/projected/d45071be-ff25-4f66-98f0-117316d826b2-kube-api-access-w89md\") pod \"e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs\" (UID: \"d45071be-ff25-4f66-98f0-117316d826b2\") " pod="openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs" Jul 15 21:36:57.897678 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:57.897648 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs" Jul 15 21:36:58.056853 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:58.056829 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs"] Jul 15 21:36:58.058391 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:36:58.058359 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd45071be_ff25_4f66_98f0_117316d826b2.slice/crio-5e6760f9e29e339907795656978758e8d6f82c33740b960e296d083255b03445 WatchSource:0}: Error finding container 5e6760f9e29e339907795656978758e8d6f82c33740b960e296d083255b03445: Status 404 returned error can't find the container with id 5e6760f9e29e339907795656978758e8d6f82c33740b960e296d083255b03445 Jul 15 21:36:58.060655 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:58.060625 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:36:58.060736 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:58.060691 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:36:58.060736 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:58.060725 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:36:58.992499 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:58.992459 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs" event={"ID":"d45071be-ff25-4f66-98f0-117316d826b2","Type":"ContainerStarted","Data":"15a3f6cb19d5209fcb56f2ed77123e816cc4cef3d8e675d1d9b935d9371dc85b"} Jul 15 21:36:58.992689 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:58.992536 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs" event={"ID":"d45071be-ff25-4f66-98f0-117316d826b2","Type":"ContainerStarted","Data":"5e6760f9e29e339907795656978758e8d6f82c33740b960e296d083255b03445"} Jul 15 21:36:59.015392 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:59.015345 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs" podStartSLOduration=2.01532944 podStartE2EDuration="2.01532944s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:36:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:36:59.014039176 +0000 UTC m=+2123.555351727" watchObservedRunningTime="2026-07-15 21:36:59.01532944 +0000 UTC m=+2123.556641992" Jul 15 21:36:59.898785 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:59.898746 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs" Jul 15 21:36:59.903638 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:59.903612 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs" Jul 15 21:36:59.998674 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:59.998641 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs" Jul 15 21:36:59.999664 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:36:59.999646 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs" Jul 15 21:37:01.033659 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:01.033623 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:37:01.164496 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:01.164416 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:37:01.288159 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:01.288053 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:37:01.447332 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:01.447292 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:37:01.972732 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:01.972698 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:37:02.073876 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:02.073832 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:37:05.470111 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:05.470045 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:37:05.688369 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:05.688327 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:37:06.147605 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:06.147572 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:37:06.248842 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:06.248807 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:37:06.564055 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:06.563942 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:37:10.764562 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:10.764524 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:37:12.494305 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:12.494258 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:37:12.560622 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:12.560585 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:37:14.084237 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:14.084202 2579 generic.go:356] "Generic (PLEG): container finished" podID="d45071be-ff25-4f66-98f0-117316d826b2" containerID="15a3f6cb19d5209fcb56f2ed77123e816cc4cef3d8e675d1d9b935d9371dc85b" exitCode=0 Jul 15 21:37:14.084657 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:14.084272 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs" event={"ID":"d45071be-ff25-4f66-98f0-117316d826b2","Type":"ContainerDied","Data":"15a3f6cb19d5209fcb56f2ed77123e816cc4cef3d8e675d1d9b935d9371dc85b"} Jul 15 21:37:14.084717 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:14.084666 2579 scope.go:117] "RemoveContainer" containerID="15a3f6cb19d5209fcb56f2ed77123e816cc4cef3d8e675d1d9b935d9371dc85b" Jul 15 21:37:14.085674 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:14.085648 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:37:14.085798 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:14.085730 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:37:14.085798 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:14.085778 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:37:14.999006 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:14.998972 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs" Jul 15 21:37:15.092234 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:15.092198 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs" event={"ID":"d45071be-ff25-4f66-98f0-117316d826b2","Type":"ContainerStarted","Data":"3d28db0e4c1d24a15a72bace3711856e98672677f0ea5dc4d7f92c32291a3cc5"} Jul 15 21:37:15.898376 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:15.898341 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs" Jul 15 21:37:15.899476 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:15.899450 2579 patch_prober.go:28] interesting pod/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 15 21:37:15.899590 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:15.899500 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs" podUID="d45071be-ff25-4f66-98f0-117316d826b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 15 21:37:16.898555 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:16.898516 2579 patch_prober.go:28] interesting pod/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 15 21:37:16.899117 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:16.898578 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs" podUID="d45071be-ff25-4f66-98f0-117316d826b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 15 21:37:17.899024 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:17.898983 2579 patch_prober.go:28] interesting pod/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 15 21:37:17.899458 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:17.899050 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs" podUID="d45071be-ff25-4f66-98f0-117316d826b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 15 21:37:18.898378 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:18.898344 2579 patch_prober.go:28] interesting pod/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 15 21:37:18.898572 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:18.898403 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs" podUID="d45071be-ff25-4f66-98f0-117316d826b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 15 21:37:19.902243 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:19.902205 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs" Jul 15 21:37:19.902625 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:19.902477 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs" Jul 15 21:37:19.903229 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:19.903210 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs" Jul 15 21:37:23.651015 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:23.650978 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:37:26.684440 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:26.684406 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:37:27.939561 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:27.939511 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:37:33.965055 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:33.965019 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:37:34.031706 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:34.031664 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:37:38.599446 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:38.599408 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs"] Jul 15 21:37:38.599980 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:38.599682 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs" podUID="d45071be-ff25-4f66-98f0-117316d826b2" containerName="istio-proxy" containerID="cri-o://3d28db0e4c1d24a15a72bace3711856e98672677f0ea5dc4d7f92c32291a3cc5" gracePeriod=30 Jul 15 21:37:38.629324 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:38.629284 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:37:38.848336 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:38.848297 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:37:39.332269 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:39.332222 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:37:39.401368 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:39.401335 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:37:43.855330 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:43.855302 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs" Jul 15 21:37:43.897906 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:43.897819 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d45071be-ff25-4f66-98f0-117316d826b2-istiod-ca-cert\") pod \"d45071be-ff25-4f66-98f0-117316d826b2\" (UID: \"d45071be-ff25-4f66-98f0-117316d826b2\") " Jul 15 21:37:43.898059 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:43.897901 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d45071be-ff25-4f66-98f0-117316d826b2-istio-data\") pod \"d45071be-ff25-4f66-98f0-117316d826b2\" (UID: \"d45071be-ff25-4f66-98f0-117316d826b2\") " Jul 15 21:37:43.898059 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:43.897947 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d45071be-ff25-4f66-98f0-117316d826b2-istio-envoy\") pod \"d45071be-ff25-4f66-98f0-117316d826b2\" (UID: \"d45071be-ff25-4f66-98f0-117316d826b2\") " Jul 15 21:37:43.898059 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:43.897979 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d45071be-ff25-4f66-98f0-117316d826b2-istio-podinfo\") pod \"d45071be-ff25-4f66-98f0-117316d826b2\" (UID: \"d45071be-ff25-4f66-98f0-117316d826b2\") " Jul 15 21:37:43.898059 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:43.897999 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d45071be-ff25-4f66-98f0-117316d826b2-credential-socket\") pod \"d45071be-ff25-4f66-98f0-117316d826b2\" (UID: \"d45071be-ff25-4f66-98f0-117316d826b2\") " Jul 15 21:37:43.898059 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:43.898057 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d45071be-ff25-4f66-98f0-117316d826b2-workload-socket\") pod \"d45071be-ff25-4f66-98f0-117316d826b2\" (UID: \"d45071be-ff25-4f66-98f0-117316d826b2\") " Jul 15 21:37:43.898463 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:43.898105 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d45071be-ff25-4f66-98f0-117316d826b2-istio-token\") pod \"d45071be-ff25-4f66-98f0-117316d826b2\" (UID: \"d45071be-ff25-4f66-98f0-117316d826b2\") " Jul 15 21:37:43.898463 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:43.898137 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w89md\" (UniqueName: \"kubernetes.io/projected/d45071be-ff25-4f66-98f0-117316d826b2-kube-api-access-w89md\") pod \"d45071be-ff25-4f66-98f0-117316d826b2\" (UID: \"d45071be-ff25-4f66-98f0-117316d826b2\") " Jul 15 21:37:43.898463 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:43.898172 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d45071be-ff25-4f66-98f0-117316d826b2-workload-certs\") pod \"d45071be-ff25-4f66-98f0-117316d826b2\" (UID: \"d45071be-ff25-4f66-98f0-117316d826b2\") " Jul 15 21:37:43.898463 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:43.898258 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d45071be-ff25-4f66-98f0-117316d826b2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d45071be-ff25-4f66-98f0-117316d826b2" (UID: "d45071be-ff25-4f66-98f0-117316d826b2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 21:37:43.898463 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:43.898359 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d45071be-ff25-4f66-98f0-117316d826b2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d45071be-ff25-4f66-98f0-117316d826b2" (UID: "d45071be-ff25-4f66-98f0-117316d826b2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:37:43.899049 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:43.898823 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d45071be-ff25-4f66-98f0-117316d826b2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d45071be-ff25-4f66-98f0-117316d826b2" (UID: "d45071be-ff25-4f66-98f0-117316d826b2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:37:43.899049 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:43.898912 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d45071be-ff25-4f66-98f0-117316d826b2-istio-data" (OuterVolumeSpecName: "istio-data") pod "d45071be-ff25-4f66-98f0-117316d826b2" (UID: "d45071be-ff25-4f66-98f0-117316d826b2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:37:43.899049 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:43.898956 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d45071be-ff25-4f66-98f0-117316d826b2-credential-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:37:43.899049 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:43.898988 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d45071be-ff25-4f66-98f0-117316d826b2-workload-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:37:43.899049 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:43.899012 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d45071be-ff25-4f66-98f0-117316d826b2-istiod-ca-cert\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:37:43.900919 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:43.899420 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d45071be-ff25-4f66-98f0-117316d826b2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d45071be-ff25-4f66-98f0-117316d826b2" (UID: "d45071be-ff25-4f66-98f0-117316d826b2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:37:43.901384 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:43.901358 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d45071be-ff25-4f66-98f0-117316d826b2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d45071be-ff25-4f66-98f0-117316d826b2" (UID: "d45071be-ff25-4f66-98f0-117316d826b2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:37:43.908031 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:43.907459 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d45071be-ff25-4f66-98f0-117316d826b2-kube-api-access-w89md" (OuterVolumeSpecName: "kube-api-access-w89md") pod "d45071be-ff25-4f66-98f0-117316d826b2" (UID: "d45071be-ff25-4f66-98f0-117316d826b2"). InnerVolumeSpecName "kube-api-access-w89md". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:37:43.908031 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:43.907571 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d45071be-ff25-4f66-98f0-117316d826b2-istio-token" (OuterVolumeSpecName: "istio-token") pod "d45071be-ff25-4f66-98f0-117316d826b2" (UID: "d45071be-ff25-4f66-98f0-117316d826b2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:37:43.909641 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:43.909613 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d45071be-ff25-4f66-98f0-117316d826b2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d45071be-ff25-4f66-98f0-117316d826b2" (UID: "d45071be-ff25-4f66-98f0-117316d826b2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 21:37:43.999698 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:43.999667 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d45071be-ff25-4f66-98f0-117316d826b2-istio-token\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:37:43.999698 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:43.999692 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w89md\" (UniqueName: \"kubernetes.io/projected/d45071be-ff25-4f66-98f0-117316d826b2-kube-api-access-w89md\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:37:43.999698 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:43.999702 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d45071be-ff25-4f66-98f0-117316d826b2-workload-certs\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:37:43.999935 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:43.999711 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d45071be-ff25-4f66-98f0-117316d826b2-istio-data\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:37:43.999935 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:43.999719 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d45071be-ff25-4f66-98f0-117316d826b2-istio-envoy\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:37:43.999935 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:43.999728 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d45071be-ff25-4f66-98f0-117316d826b2-istio-podinfo\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:37:44.249749 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:44.249655 2579 generic.go:356] "Generic (PLEG): container finished" podID="d45071be-ff25-4f66-98f0-117316d826b2" containerID="3d28db0e4c1d24a15a72bace3711856e98672677f0ea5dc4d7f92c32291a3cc5" exitCode=0 Jul 15 21:37:44.249749 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:44.249706 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs" event={"ID":"d45071be-ff25-4f66-98f0-117316d826b2","Type":"ContainerDied","Data":"3d28db0e4c1d24a15a72bace3711856e98672677f0ea5dc4d7f92c32291a3cc5"} Jul 15 21:37:44.249749 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:44.249741 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs" Jul 15 21:37:44.250029 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:44.249772 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs" event={"ID":"d45071be-ff25-4f66-98f0-117316d826b2","Type":"ContainerDied","Data":"5e6760f9e29e339907795656978758e8d6f82c33740b960e296d083255b03445"} Jul 15 21:37:44.250029 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:44.249795 2579 scope.go:117] "RemoveContainer" containerID="3d28db0e4c1d24a15a72bace3711856e98672677f0ea5dc4d7f92c32291a3cc5" Jul 15 21:37:44.259368 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:44.259337 2579 scope.go:117] "RemoveContainer" containerID="15a3f6cb19d5209fcb56f2ed77123e816cc4cef3d8e675d1d9b935d9371dc85b" Jul 15 21:37:44.267812 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:44.267791 2579 scope.go:117] "RemoveContainer" containerID="3d28db0e4c1d24a15a72bace3711856e98672677f0ea5dc4d7f92c32291a3cc5" Jul 15 21:37:44.268110 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:37:44.268089 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d28db0e4c1d24a15a72bace3711856e98672677f0ea5dc4d7f92c32291a3cc5\": container with ID starting with 3d28db0e4c1d24a15a72bace3711856e98672677f0ea5dc4d7f92c32291a3cc5 not found: ID does not exist" containerID="3d28db0e4c1d24a15a72bace3711856e98672677f0ea5dc4d7f92c32291a3cc5" Jul 15 21:37:44.268193 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:44.268121 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d28db0e4c1d24a15a72bace3711856e98672677f0ea5dc4d7f92c32291a3cc5"} err="failed to get container status \"3d28db0e4c1d24a15a72bace3711856e98672677f0ea5dc4d7f92c32291a3cc5\": rpc error: code = NotFound desc = could not find container \"3d28db0e4c1d24a15a72bace3711856e98672677f0ea5dc4d7f92c32291a3cc5\": container with ID starting with 3d28db0e4c1d24a15a72bace3711856e98672677f0ea5dc4d7f92c32291a3cc5 not found: ID does not exist" Jul 15 21:37:44.268193 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:44.268139 2579 scope.go:117] "RemoveContainer" containerID="15a3f6cb19d5209fcb56f2ed77123e816cc4cef3d8e675d1d9b935d9371dc85b" Jul 15 21:37:44.268371 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:37:44.268354 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15a3f6cb19d5209fcb56f2ed77123e816cc4cef3d8e675d1d9b935d9371dc85b\": container with ID starting with 15a3f6cb19d5209fcb56f2ed77123e816cc4cef3d8e675d1d9b935d9371dc85b not found: ID does not exist" containerID="15a3f6cb19d5209fcb56f2ed77123e816cc4cef3d8e675d1d9b935d9371dc85b" Jul 15 21:37:44.268434 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:44.268373 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15a3f6cb19d5209fcb56f2ed77123e816cc4cef3d8e675d1d9b935d9371dc85b"} err="failed to get container status \"15a3f6cb19d5209fcb56f2ed77123e816cc4cef3d8e675d1d9b935d9371dc85b\": rpc error: code = NotFound desc = could not find container \"15a3f6cb19d5209fcb56f2ed77123e816cc4cef3d8e675d1d9b935d9371dc85b\": container with ID starting with 15a3f6cb19d5209fcb56f2ed77123e816cc4cef3d8e675d1d9b935d9371dc85b not found: ID does not exist" Jul 15 21:37:44.269714 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:44.269691 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs"] Jul 15 21:37:44.272985 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:44.272963 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-0af6f2e9-openshift-default-cdf479855-mnhxs"] Jul 15 21:37:45.276875 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:45.276837 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:37:45.387016 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:45.386980 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:37:45.578376 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:45.578283 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:37:45.687514 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:45.687479 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:37:45.997047 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:45.997015 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d45071be-ff25-4f66-98f0-117316d826b2" path="/var/lib/kubelet/pods/d45071be-ff25-4f66-98f0-117316d826b2/volumes" Jul 15 21:37:51.450405 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:51.450368 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:37:51.513379 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:51.513339 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:37:58.245299 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:58.245255 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:37:58.548920 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:58.548825 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb"] Jul 15 21:37:58.549422 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:58.549397 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d45071be-ff25-4f66-98f0-117316d826b2" containerName="istio-proxy" Jul 15 21:37:58.549558 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:58.549424 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d45071be-ff25-4f66-98f0-117316d826b2" containerName="istio-proxy" Jul 15 21:37:58.549558 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:58.549456 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d45071be-ff25-4f66-98f0-117316d826b2" containerName="istio-proxy" Jul 15 21:37:58.549558 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:58.549462 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d45071be-ff25-4f66-98f0-117316d826b2" containerName="istio-proxy" Jul 15 21:37:58.549558 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:58.549537 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d45071be-ff25-4f66-98f0-117316d826b2" containerName="istio-proxy" Jul 15 21:37:58.549558 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:58.549546 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d45071be-ff25-4f66-98f0-117316d826b2" containerName="istio-proxy" Jul 15 21:37:58.553552 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:58.553528 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb" Jul 15 21:37:58.557104 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:58.557052 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-42fd7c24-openshift-default-dockercfg-tbqnf\"" Jul 15 21:37:58.571626 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:58.571585 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:37:58.572385 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:58.572364 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb"] Jul 15 21:37:58.627046 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:58.626996 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvzfw\" (UniqueName: \"kubernetes.io/projected/d4de7bef-cc47-4c59-ba1a-e25352549fc0-kube-api-access-tvzfw\") pod \"e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb\" (UID: \"d4de7bef-cc47-4c59-ba1a-e25352549fc0\") " pod="openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb" Jul 15 21:37:58.627269 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:58.627094 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4de7bef-cc47-4c59-ba1a-e25352549fc0-istio-podinfo\") pod \"e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb\" (UID: \"d4de7bef-cc47-4c59-ba1a-e25352549fc0\") " pod="openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb" Jul 15 21:37:58.627269 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:58.627129 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4de7bef-cc47-4c59-ba1a-e25352549fc0-credential-socket\") pod \"e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb\" (UID: \"d4de7bef-cc47-4c59-ba1a-e25352549fc0\") " pod="openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb" Jul 15 21:37:58.627269 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:58.627198 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4de7bef-cc47-4c59-ba1a-e25352549fc0-workload-certs\") pod \"e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb\" (UID: \"d4de7bef-cc47-4c59-ba1a-e25352549fc0\") " pod="openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb" Jul 15 21:37:58.627269 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:58.627257 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4de7bef-cc47-4c59-ba1a-e25352549fc0-workload-socket\") pod \"e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb\" (UID: \"d4de7bef-cc47-4c59-ba1a-e25352549fc0\") " pod="openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb" Jul 15 21:37:58.627491 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:58.627285 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4de7bef-cc47-4c59-ba1a-e25352549fc0-istiod-ca-cert\") pod \"e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb\" (UID: \"d4de7bef-cc47-4c59-ba1a-e25352549fc0\") " pod="openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb" Jul 15 21:37:58.627491 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:58.627314 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4de7bef-cc47-4c59-ba1a-e25352549fc0-istio-data\") pod \"e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb\" (UID: \"d4de7bef-cc47-4c59-ba1a-e25352549fc0\") " pod="openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb" Jul 15 21:37:58.627491 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:58.627358 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4de7bef-cc47-4c59-ba1a-e25352549fc0-istio-envoy\") pod \"e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb\" (UID: \"d4de7bef-cc47-4c59-ba1a-e25352549fc0\") " pod="openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb" Jul 15 21:37:58.627491 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:58.627446 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4de7bef-cc47-4c59-ba1a-e25352549fc0-istio-token\") pod \"e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb\" (UID: \"d4de7bef-cc47-4c59-ba1a-e25352549fc0\") " pod="openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb" Jul 15 21:37:58.727967 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:58.727913 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4de7bef-cc47-4c59-ba1a-e25352549fc0-istio-podinfo\") pod \"e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb\" (UID: \"d4de7bef-cc47-4c59-ba1a-e25352549fc0\") " pod="openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb" Jul 15 21:37:58.727967 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:58.727973 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4de7bef-cc47-4c59-ba1a-e25352549fc0-credential-socket\") pod \"e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb\" (UID: \"d4de7bef-cc47-4c59-ba1a-e25352549fc0\") " pod="openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb" Jul 15 21:37:58.728240 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:58.728005 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4de7bef-cc47-4c59-ba1a-e25352549fc0-workload-certs\") pod \"e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb\" (UID: \"d4de7bef-cc47-4c59-ba1a-e25352549fc0\") " pod="openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb" Jul 15 21:37:58.728240 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:58.728078 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4de7bef-cc47-4c59-ba1a-e25352549fc0-workload-socket\") pod \"e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb\" (UID: \"d4de7bef-cc47-4c59-ba1a-e25352549fc0\") " pod="openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb" Jul 15 21:37:58.728240 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:58.728108 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4de7bef-cc47-4c59-ba1a-e25352549fc0-istiod-ca-cert\") pod \"e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb\" (UID: \"d4de7bef-cc47-4c59-ba1a-e25352549fc0\") " pod="openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb" Jul 15 21:37:58.728431 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:58.728404 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4de7bef-cc47-4c59-ba1a-e25352549fc0-credential-socket\") pod \"e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb\" (UID: \"d4de7bef-cc47-4c59-ba1a-e25352549fc0\") " pod="openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb" Jul 15 21:37:58.728492 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:58.728451 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4de7bef-cc47-4c59-ba1a-e25352549fc0-istio-data\") pod \"e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb\" (UID: \"d4de7bef-cc47-4c59-ba1a-e25352549fc0\") " pod="openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb" Jul 15 21:37:58.728492 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:58.728455 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4de7bef-cc47-4c59-ba1a-e25352549fc0-workload-socket\") pod \"e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb\" (UID: \"d4de7bef-cc47-4c59-ba1a-e25352549fc0\") " pod="openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb" Jul 15 21:37:58.728607 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:58.728489 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4de7bef-cc47-4c59-ba1a-e25352549fc0-workload-certs\") pod \"e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb\" (UID: \"d4de7bef-cc47-4c59-ba1a-e25352549fc0\") " pod="openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb" Jul 15 21:37:58.728607 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:58.728511 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4de7bef-cc47-4c59-ba1a-e25352549fc0-istio-envoy\") pod \"e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb\" (UID: \"d4de7bef-cc47-4c59-ba1a-e25352549fc0\") " pod="openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb" Jul 15 21:37:58.728607 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:58.728601 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4de7bef-cc47-4c59-ba1a-e25352549fc0-istio-token\") pod \"e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb\" (UID: \"d4de7bef-cc47-4c59-ba1a-e25352549fc0\") " pod="openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb" Jul 15 21:37:58.728760 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:58.728630 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvzfw\" (UniqueName: \"kubernetes.io/projected/d4de7bef-cc47-4c59-ba1a-e25352549fc0-kube-api-access-tvzfw\") pod \"e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb\" (UID: \"d4de7bef-cc47-4c59-ba1a-e25352549fc0\") " pod="openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb" Jul 15 21:37:58.729134 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:58.729109 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4de7bef-cc47-4c59-ba1a-e25352549fc0-istio-data\") pod \"e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb\" (UID: \"d4de7bef-cc47-4c59-ba1a-e25352549fc0\") " pod="openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb" Jul 15 21:37:58.729499 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:58.729476 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4de7bef-cc47-4c59-ba1a-e25352549fc0-istiod-ca-cert\") pod \"e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb\" (UID: \"d4de7bef-cc47-4c59-ba1a-e25352549fc0\") " pod="openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb" Jul 15 21:37:58.730714 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:58.730689 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4de7bef-cc47-4c59-ba1a-e25352549fc0-istio-envoy\") pod \"e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb\" (UID: \"d4de7bef-cc47-4c59-ba1a-e25352549fc0\") " pod="openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb" Jul 15 21:37:58.730821 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:58.730803 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4de7bef-cc47-4c59-ba1a-e25352549fc0-istio-podinfo\") pod \"e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb\" (UID: \"d4de7bef-cc47-4c59-ba1a-e25352549fc0\") " pod="openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb" Jul 15 21:37:58.737526 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:58.737501 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4de7bef-cc47-4c59-ba1a-e25352549fc0-istio-token\") pod \"e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb\" (UID: \"d4de7bef-cc47-4c59-ba1a-e25352549fc0\") " pod="openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb" Jul 15 21:37:58.737637 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:58.737525 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvzfw\" (UniqueName: \"kubernetes.io/projected/d4de7bef-cc47-4c59-ba1a-e25352549fc0-kube-api-access-tvzfw\") pod \"e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb\" (UID: \"d4de7bef-cc47-4c59-ba1a-e25352549fc0\") " pod="openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb" Jul 15 21:37:58.864264 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:58.864220 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb" Jul 15 21:37:58.879873 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:58.879813 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:37:59.024939 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:37:59.024541 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4de7bef_cc47_4c59_ba1a_e25352549fc0.slice/crio-0c4d86be9af01804f1d18ad1f8b6d48f089e2bde8d3f3b0cfec657bdb8e677f6 WatchSource:0}: Error finding container 0c4d86be9af01804f1d18ad1f8b6d48f089e2bde8d3f3b0cfec657bdb8e677f6: Status 404 returned error can't find the container with id 0c4d86be9af01804f1d18ad1f8b6d48f089e2bde8d3f3b0cfec657bdb8e677f6 Jul 15 21:37:59.024939 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:59.024799 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb"] Jul 15 21:37:59.027381 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:59.027340 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:37:59.027553 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:59.027425 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:37:59.027553 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:59.027463 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:37:59.331746 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:59.331634 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb" event={"ID":"d4de7bef-cc47-4c59-ba1a-e25352549fc0","Type":"ContainerStarted","Data":"09f89fc93320bd55433f76e5a029af8e62e8d1c41119a035ae653685ce7fa191"} Jul 15 21:37:59.331746 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:59.331721 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb" event={"ID":"d4de7bef-cc47-4c59-ba1a-e25352549fc0","Type":"ContainerStarted","Data":"0c4d86be9af01804f1d18ad1f8b6d48f089e2bde8d3f3b0cfec657bdb8e677f6"} Jul 15 21:37:59.356416 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:59.356351 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb" podStartSLOduration=1.356326266 podStartE2EDuration="1.356326266s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:37:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:37:59.353644677 +0000 UTC m=+2183.894957233" watchObservedRunningTime="2026-07-15 21:37:59.356326266 +0000 UTC m=+2183.897638817" Jul 15 21:37:59.865193 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:37:59.865146 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb" Jul 15 21:38:00.869883 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:38:00.869853 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb" Jul 15 21:38:01.277093 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:38:01.276985 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:38:01.342073 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:38:01.342023 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb" Jul 15 21:38:01.342983 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:38:01.342963 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb" Jul 15 21:38:01.964993 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:38:01.964957 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:38:02.065647 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:38:02.065599 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:38:05.849439 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:38:05.849400 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:38:06.305627 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:38:06.305589 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:38:14.814574 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:38:14.814532 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:38:15.419188 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:38:15.419150 2579 generic.go:356] "Generic (PLEG): container finished" podID="d4de7bef-cc47-4c59-ba1a-e25352549fc0" containerID="09f89fc93320bd55433f76e5a029af8e62e8d1c41119a035ae653685ce7fa191" exitCode=0 Jul 15 21:38:15.419414 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:38:15.419227 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb" event={"ID":"d4de7bef-cc47-4c59-ba1a-e25352549fc0","Type":"ContainerDied","Data":"09f89fc93320bd55433f76e5a029af8e62e8d1c41119a035ae653685ce7fa191"} Jul 15 21:38:15.419763 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:38:15.419746 2579 scope.go:117] "RemoveContainer" containerID="09f89fc93320bd55433f76e5a029af8e62e8d1c41119a035ae653685ce7fa191" Jul 15 21:38:15.420810 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:38:15.420782 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:38:15.420863 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:38:15.420850 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:38:15.420899 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:38:15.420882 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:38:16.342107 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:38:16.342058 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb" Jul 15 21:38:16.429022 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:38:16.428984 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb" event={"ID":"d4de7bef-cc47-4c59-ba1a-e25352549fc0","Type":"ContainerStarted","Data":"bd41c32d9416f4e404a443949a08da17103ea2c740a1429405f033b6d3483ce6"} Jul 15 21:38:16.864796 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:38:16.864754 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb" Jul 15 21:38:16.866021 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:38:16.865990 2579 patch_prober.go:28] interesting pod/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 15 21:38:16.866173 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:38:16.866044 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb" podUID="d4de7bef-cc47-4c59-ba1a-e25352549fc0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 15 21:38:17.865438 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:38:17.865399 2579 patch_prober.go:28] interesting pod/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 15 21:38:17.865834 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:38:17.865460 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb" podUID="d4de7bef-cc47-4c59-ba1a-e25352549fc0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 15 21:38:18.864859 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:38:18.864821 2579 patch_prober.go:28] interesting pod/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 15 21:38:18.865038 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:38:18.864875 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb" podUID="d4de7bef-cc47-4c59-ba1a-e25352549fc0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 15 21:38:19.865179 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:38:19.865138 2579 patch_prober.go:28] interesting pod/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 15 21:38:19.865575 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:38:19.865202 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb" podUID="d4de7bef-cc47-4c59-ba1a-e25352549fc0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 15 21:38:20.865436 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:38:20.865393 2579 patch_prober.go:28] interesting pod/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 15 21:38:20.865842 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:38:20.865459 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb" podUID="d4de7bef-cc47-4c59-ba1a-e25352549fc0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 15 21:38:21.429048 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:38:21.429016 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:38:21.869623 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:38:21.869593 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb" Jul 15 21:38:21.869986 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:38:21.869963 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb" Jul 15 21:38:21.870577 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:38:21.870561 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb" Jul 15 21:38:30.487705 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:38:30.487662 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:38:44.333994 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:38:44.333951 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:38:48.939553 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:38:48.939510 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:38:48.988359 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:38:48.988319 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:38:54.010529 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:38:54.010494 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:39:31.020017 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:31.019981 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:39:31.216465 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:31.216428 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:39:37.557868 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:37.557832 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:39:37.626676 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:37.626637 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:39:41.208954 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:41.208923 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb"] Jul 15 21:39:41.209421 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:41.209132 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb" podUID="d4de7bef-cc47-4c59-ba1a-e25352549fc0" containerName="istio-proxy" containerID="cri-o://bd41c32d9416f4e404a443949a08da17103ea2c740a1429405f033b6d3483ce6" gracePeriod=30 Jul 15 21:39:41.244324 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:41.244272 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:39:41.480242 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:41.480152 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:39:42.519589 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:42.519560 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:39:42.625156 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:42.625116 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:39:42.720585 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:42.720550 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:39:43.427806 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:43.427767 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:39:43.525817 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:43.525779 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:39:43.628350 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:43.628318 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:39:44.969417 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:44.969382 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:39:45.486883 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:45.486845 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:39:46.470988 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:46.470920 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb" Jul 15 21:39:46.578656 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:46.578617 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4de7bef-cc47-4c59-ba1a-e25352549fc0-istio-token\") pod \"d4de7bef-cc47-4c59-ba1a-e25352549fc0\" (UID: \"d4de7bef-cc47-4c59-ba1a-e25352549fc0\") " Jul 15 21:39:46.578656 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:46.578654 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4de7bef-cc47-4c59-ba1a-e25352549fc0-workload-socket\") pod \"d4de7bef-cc47-4c59-ba1a-e25352549fc0\" (UID: \"d4de7bef-cc47-4c59-ba1a-e25352549fc0\") " Jul 15 21:39:46.578908 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:46.578697 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4de7bef-cc47-4c59-ba1a-e25352549fc0-credential-socket\") pod \"d4de7bef-cc47-4c59-ba1a-e25352549fc0\" (UID: \"d4de7bef-cc47-4c59-ba1a-e25352549fc0\") " Jul 15 21:39:46.578908 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:46.578720 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tvzfw\" (UniqueName: \"kubernetes.io/projected/d4de7bef-cc47-4c59-ba1a-e25352549fc0-kube-api-access-tvzfw\") pod \"d4de7bef-cc47-4c59-ba1a-e25352549fc0\" (UID: \"d4de7bef-cc47-4c59-ba1a-e25352549fc0\") " Jul 15 21:39:46.578908 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:46.578734 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4de7bef-cc47-4c59-ba1a-e25352549fc0-istio-envoy\") pod \"d4de7bef-cc47-4c59-ba1a-e25352549fc0\" (UID: \"d4de7bef-cc47-4c59-ba1a-e25352549fc0\") " Jul 15 21:39:46.578908 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:46.578759 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4de7bef-cc47-4c59-ba1a-e25352549fc0-istiod-ca-cert\") pod \"d4de7bef-cc47-4c59-ba1a-e25352549fc0\" (UID: \"d4de7bef-cc47-4c59-ba1a-e25352549fc0\") " Jul 15 21:39:46.578908 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:46.578812 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4de7bef-cc47-4c59-ba1a-e25352549fc0-istio-data\") pod \"d4de7bef-cc47-4c59-ba1a-e25352549fc0\" (UID: \"d4de7bef-cc47-4c59-ba1a-e25352549fc0\") " Jul 15 21:39:46.578908 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:46.578846 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4de7bef-cc47-4c59-ba1a-e25352549fc0-workload-certs\") pod \"d4de7bef-cc47-4c59-ba1a-e25352549fc0\" (UID: \"d4de7bef-cc47-4c59-ba1a-e25352549fc0\") " Jul 15 21:39:46.578908 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:46.578884 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4de7bef-cc47-4c59-ba1a-e25352549fc0-istio-podinfo\") pod \"d4de7bef-cc47-4c59-ba1a-e25352549fc0\" (UID: \"d4de7bef-cc47-4c59-ba1a-e25352549fc0\") " Jul 15 21:39:46.579322 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:46.578911 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4de7bef-cc47-4c59-ba1a-e25352549fc0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d4de7bef-cc47-4c59-ba1a-e25352549fc0" (UID: "d4de7bef-cc47-4c59-ba1a-e25352549fc0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:39:46.579322 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:46.579098 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4de7bef-cc47-4c59-ba1a-e25352549fc0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d4de7bef-cc47-4c59-ba1a-e25352549fc0" (UID: "d4de7bef-cc47-4c59-ba1a-e25352549fc0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:39:46.579322 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:46.579177 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4de7bef-cc47-4c59-ba1a-e25352549fc0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d4de7bef-cc47-4c59-ba1a-e25352549fc0" (UID: "d4de7bef-cc47-4c59-ba1a-e25352549fc0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 21:39:46.579322 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:46.579197 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4de7bef-cc47-4c59-ba1a-e25352549fc0-workload-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:39:46.579322 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:46.579214 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4de7bef-cc47-4c59-ba1a-e25352549fc0-credential-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:39:46.579603 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:46.579313 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4de7bef-cc47-4c59-ba1a-e25352549fc0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d4de7bef-cc47-4c59-ba1a-e25352549fc0" (UID: "d4de7bef-cc47-4c59-ba1a-e25352549fc0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:39:46.579967 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:46.579942 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4de7bef-cc47-4c59-ba1a-e25352549fc0-istio-data" (OuterVolumeSpecName: "istio-data") pod "d4de7bef-cc47-4c59-ba1a-e25352549fc0" (UID: "d4de7bef-cc47-4c59-ba1a-e25352549fc0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:39:46.581401 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:46.581373 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4de7bef-cc47-4c59-ba1a-e25352549fc0-kube-api-access-tvzfw" (OuterVolumeSpecName: "kube-api-access-tvzfw") pod "d4de7bef-cc47-4c59-ba1a-e25352549fc0" (UID: "d4de7bef-cc47-4c59-ba1a-e25352549fc0"). InnerVolumeSpecName "kube-api-access-tvzfw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:39:46.581401 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:46.581397 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4de7bef-cc47-4c59-ba1a-e25352549fc0-istio-token" (OuterVolumeSpecName: "istio-token") pod "d4de7bef-cc47-4c59-ba1a-e25352549fc0" (UID: "d4de7bef-cc47-4c59-ba1a-e25352549fc0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:39:46.581608 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:46.581591 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4de7bef-cc47-4c59-ba1a-e25352549fc0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d4de7bef-cc47-4c59-ba1a-e25352549fc0" (UID: "d4de7bef-cc47-4c59-ba1a-e25352549fc0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:39:46.581783 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:46.581766 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d4de7bef-cc47-4c59-ba1a-e25352549fc0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d4de7bef-cc47-4c59-ba1a-e25352549fc0" (UID: "d4de7bef-cc47-4c59-ba1a-e25352549fc0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 21:39:46.680504 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:46.680467 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tvzfw\" (UniqueName: \"kubernetes.io/projected/d4de7bef-cc47-4c59-ba1a-e25352549fc0-kube-api-access-tvzfw\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:39:46.680504 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:46.680491 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4de7bef-cc47-4c59-ba1a-e25352549fc0-istio-envoy\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:39:46.680504 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:46.680500 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4de7bef-cc47-4c59-ba1a-e25352549fc0-istiod-ca-cert\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:39:46.680504 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:46.680508 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4de7bef-cc47-4c59-ba1a-e25352549fc0-istio-data\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:39:46.680504 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:46.680516 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4de7bef-cc47-4c59-ba1a-e25352549fc0-workload-certs\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:39:46.680890 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:46.680524 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4de7bef-cc47-4c59-ba1a-e25352549fc0-istio-podinfo\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:39:46.680890 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:46.680531 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4de7bef-cc47-4c59-ba1a-e25352549fc0-istio-token\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:39:46.929420 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:46.929387 2579 generic.go:356] "Generic (PLEG): container finished" podID="d4de7bef-cc47-4c59-ba1a-e25352549fc0" containerID="bd41c32d9416f4e404a443949a08da17103ea2c740a1429405f033b6d3483ce6" exitCode=0 Jul 15 21:39:46.929603 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:46.929434 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb" Jul 15 21:39:46.929603 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:46.929477 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb" event={"ID":"d4de7bef-cc47-4c59-ba1a-e25352549fc0","Type":"ContainerDied","Data":"bd41c32d9416f4e404a443949a08da17103ea2c740a1429405f033b6d3483ce6"} Jul 15 21:39:46.929603 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:46.929525 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb" event={"ID":"d4de7bef-cc47-4c59-ba1a-e25352549fc0","Type":"ContainerDied","Data":"0c4d86be9af01804f1d18ad1f8b6d48f089e2bde8d3f3b0cfec657bdb8e677f6"} Jul 15 21:39:46.929603 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:46.929549 2579 scope.go:117] "RemoveContainer" containerID="bd41c32d9416f4e404a443949a08da17103ea2c740a1429405f033b6d3483ce6" Jul 15 21:39:46.938991 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:46.938967 2579 scope.go:117] "RemoveContainer" containerID="09f89fc93320bd55433f76e5a029af8e62e8d1c41119a035ae653685ce7fa191" Jul 15 21:39:46.948148 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:46.948125 2579 scope.go:117] "RemoveContainer" containerID="bd41c32d9416f4e404a443949a08da17103ea2c740a1429405f033b6d3483ce6" Jul 15 21:39:46.948432 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:39:46.948411 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd41c32d9416f4e404a443949a08da17103ea2c740a1429405f033b6d3483ce6\": container with ID starting with bd41c32d9416f4e404a443949a08da17103ea2c740a1429405f033b6d3483ce6 not found: ID does not exist" containerID="bd41c32d9416f4e404a443949a08da17103ea2c740a1429405f033b6d3483ce6" Jul 15 21:39:46.948508 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:46.948441 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd41c32d9416f4e404a443949a08da17103ea2c740a1429405f033b6d3483ce6"} err="failed to get container status \"bd41c32d9416f4e404a443949a08da17103ea2c740a1429405f033b6d3483ce6\": rpc error: code = NotFound desc = could not find container \"bd41c32d9416f4e404a443949a08da17103ea2c740a1429405f033b6d3483ce6\": container with ID starting with bd41c32d9416f4e404a443949a08da17103ea2c740a1429405f033b6d3483ce6 not found: ID does not exist" Jul 15 21:39:46.948508 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:46.948459 2579 scope.go:117] "RemoveContainer" containerID="09f89fc93320bd55433f76e5a029af8e62e8d1c41119a035ae653685ce7fa191" Jul 15 21:39:46.948698 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:39:46.948680 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09f89fc93320bd55433f76e5a029af8e62e8d1c41119a035ae653685ce7fa191\": container with ID starting with 09f89fc93320bd55433f76e5a029af8e62e8d1c41119a035ae653685ce7fa191 not found: ID does not exist" containerID="09f89fc93320bd55433f76e5a029af8e62e8d1c41119a035ae653685ce7fa191" Jul 15 21:39:46.948739 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:46.948706 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09f89fc93320bd55433f76e5a029af8e62e8d1c41119a035ae653685ce7fa191"} err="failed to get container status \"09f89fc93320bd55433f76e5a029af8e62e8d1c41119a035ae653685ce7fa191\": rpc error: code = NotFound desc = could not find container \"09f89fc93320bd55433f76e5a029af8e62e8d1c41119a035ae653685ce7fa191\": container with ID starting with 09f89fc93320bd55433f76e5a029af8e62e8d1c41119a035ae653685ce7fa191 not found: ID does not exist" Jul 15 21:39:46.955909 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:46.955883 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb"] Jul 15 21:39:46.962925 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:46.962903 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-42fd7c24-openshift-default-755f6d6f75-grqnb"] Jul 15 21:39:47.997660 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:47.997618 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4de7bef-cc47-4c59-ba1a-e25352549fc0" path="/var/lib/kubelet/pods/d4de7bef-cc47-4c59-ba1a-e25352549fc0/volumes" Jul 15 21:39:51.268383 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:51.268333 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:39:51.347041 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:51.346999 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:39:51.397755 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:51.397720 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:39:51.964179 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:51.964148 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:39:57.118555 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:57.118520 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:39:58.919991 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:58.919950 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:39:59.039444 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:59.039409 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:39:59.158387 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:39:59.158343 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:40:09.661528 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:09.661489 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:40:09.991391 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:09.991266 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:40:30.997174 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:30.997085 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:40:37.086479 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:37.086439 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:40:37.162324 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:37.162283 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:40:37.414897 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:37.414859 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:40:37.712015 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:37.711928 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g"] Jul 15 21:40:37.712476 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:37.712455 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4de7bef-cc47-4c59-ba1a-e25352549fc0" containerName="istio-proxy" Jul 15 21:40:37.712582 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:37.712478 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4de7bef-cc47-4c59-ba1a-e25352549fc0" containerName="istio-proxy" Jul 15 21:40:37.712582 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:37.712515 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4de7bef-cc47-4c59-ba1a-e25352549fc0" containerName="istio-proxy" Jul 15 21:40:37.712582 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:37.712524 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4de7bef-cc47-4c59-ba1a-e25352549fc0" containerName="istio-proxy" Jul 15 21:40:37.712738 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:37.712611 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4de7bef-cc47-4c59-ba1a-e25352549fc0" containerName="istio-proxy" Jul 15 21:40:37.712738 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:37.712624 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4de7bef-cc47-4c59-ba1a-e25352549fc0" containerName="istio-proxy" Jul 15 21:40:37.715275 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:37.715252 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g" Jul 15 21:40:37.719089 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:37.719042 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-7c4663-openshift-default-dockercfg-wtmjw\"" Jul 15 21:40:37.731320 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:37.731293 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g"] Jul 15 21:40:37.740458 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:37.740421 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:40:37.793906 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:37.793867 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a00277f-80e6-416a-89cc-7d197c2f6c4a-workload-socket\") pod \"e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g\" (UID: \"3a00277f-80e6-416a-89cc-7d197c2f6c4a\") " pod="openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g" Jul 15 21:40:37.794125 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:37.793912 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a00277f-80e6-416a-89cc-7d197c2f6c4a-istio-token\") pod \"e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g\" (UID: \"3a00277f-80e6-416a-89cc-7d197c2f6c4a\") " pod="openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g" Jul 15 21:40:37.794125 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:37.793945 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a00277f-80e6-416a-89cc-7d197c2f6c4a-istio-podinfo\") pod \"e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g\" (UID: \"3a00277f-80e6-416a-89cc-7d197c2f6c4a\") " pod="openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g" Jul 15 21:40:37.794125 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:37.793967 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a00277f-80e6-416a-89cc-7d197c2f6c4a-istio-data\") pod \"e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g\" (UID: \"3a00277f-80e6-416a-89cc-7d197c2f6c4a\") " pod="openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g" Jul 15 21:40:37.794125 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:37.794009 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pq7jw\" (UniqueName: \"kubernetes.io/projected/3a00277f-80e6-416a-89cc-7d197c2f6c4a-kube-api-access-pq7jw\") pod \"e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g\" (UID: \"3a00277f-80e6-416a-89cc-7d197c2f6c4a\") " pod="openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g" Jul 15 21:40:37.794125 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:37.794039 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a00277f-80e6-416a-89cc-7d197c2f6c4a-credential-socket\") pod \"e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g\" (UID: \"3a00277f-80e6-416a-89cc-7d197c2f6c4a\") " pod="openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g" Jul 15 21:40:37.794125 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:37.794082 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a00277f-80e6-416a-89cc-7d197c2f6c4a-istio-envoy\") pod \"e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g\" (UID: \"3a00277f-80e6-416a-89cc-7d197c2f6c4a\") " pod="openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g" Jul 15 21:40:37.794125 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:37.794108 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a00277f-80e6-416a-89cc-7d197c2f6c4a-workload-certs\") pod \"e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g\" (UID: \"3a00277f-80e6-416a-89cc-7d197c2f6c4a\") " pod="openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g" Jul 15 21:40:37.794421 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:37.794149 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a00277f-80e6-416a-89cc-7d197c2f6c4a-istiod-ca-cert\") pod \"e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g\" (UID: \"3a00277f-80e6-416a-89cc-7d197c2f6c4a\") " pod="openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g" Jul 15 21:40:37.894643 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:37.894605 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a00277f-80e6-416a-89cc-7d197c2f6c4a-workload-socket\") pod \"e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g\" (UID: \"3a00277f-80e6-416a-89cc-7d197c2f6c4a\") " pod="openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g" Jul 15 21:40:37.894843 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:37.894657 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a00277f-80e6-416a-89cc-7d197c2f6c4a-istio-token\") pod \"e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g\" (UID: \"3a00277f-80e6-416a-89cc-7d197c2f6c4a\") " pod="openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g" Jul 15 21:40:37.894843 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:37.894691 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a00277f-80e6-416a-89cc-7d197c2f6c4a-istio-podinfo\") pod \"e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g\" (UID: \"3a00277f-80e6-416a-89cc-7d197c2f6c4a\") " pod="openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g" Jul 15 21:40:37.894843 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:37.894717 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a00277f-80e6-416a-89cc-7d197c2f6c4a-istio-data\") pod \"e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g\" (UID: \"3a00277f-80e6-416a-89cc-7d197c2f6c4a\") " pod="openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g" Jul 15 21:40:37.894843 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:37.894772 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pq7jw\" (UniqueName: \"kubernetes.io/projected/3a00277f-80e6-416a-89cc-7d197c2f6c4a-kube-api-access-pq7jw\") pod \"e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g\" (UID: \"3a00277f-80e6-416a-89cc-7d197c2f6c4a\") " pod="openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g" Jul 15 21:40:37.894843 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:37.894805 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a00277f-80e6-416a-89cc-7d197c2f6c4a-credential-socket\") pod \"e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g\" (UID: \"3a00277f-80e6-416a-89cc-7d197c2f6c4a\") " pod="openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g" Jul 15 21:40:37.895151 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:37.894851 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a00277f-80e6-416a-89cc-7d197c2f6c4a-istio-envoy\") pod \"e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g\" (UID: \"3a00277f-80e6-416a-89cc-7d197c2f6c4a\") " pod="openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g" Jul 15 21:40:37.895151 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:37.894880 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a00277f-80e6-416a-89cc-7d197c2f6c4a-workload-certs\") pod \"e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g\" (UID: \"3a00277f-80e6-416a-89cc-7d197c2f6c4a\") " pod="openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g" Jul 15 21:40:37.895151 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:37.894929 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a00277f-80e6-416a-89cc-7d197c2f6c4a-istiod-ca-cert\") pod \"e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g\" (UID: \"3a00277f-80e6-416a-89cc-7d197c2f6c4a\") " pod="openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g" Jul 15 21:40:37.895151 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:37.895095 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a00277f-80e6-416a-89cc-7d197c2f6c4a-workload-socket\") pod \"e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g\" (UID: \"3a00277f-80e6-416a-89cc-7d197c2f6c4a\") " pod="openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g" Jul 15 21:40:37.895151 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:37.895101 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a00277f-80e6-416a-89cc-7d197c2f6c4a-istio-data\") pod \"e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g\" (UID: \"3a00277f-80e6-416a-89cc-7d197c2f6c4a\") " pod="openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g" Jul 15 21:40:37.895417 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:37.895206 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a00277f-80e6-416a-89cc-7d197c2f6c4a-credential-socket\") pod \"e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g\" (UID: \"3a00277f-80e6-416a-89cc-7d197c2f6c4a\") " pod="openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g" Jul 15 21:40:37.895417 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:37.895370 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a00277f-80e6-416a-89cc-7d197c2f6c4a-workload-certs\") pod \"e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g\" (UID: \"3a00277f-80e6-416a-89cc-7d197c2f6c4a\") " pod="openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g" Jul 15 21:40:37.895652 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:37.895631 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a00277f-80e6-416a-89cc-7d197c2f6c4a-istiod-ca-cert\") pod \"e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g\" (UID: \"3a00277f-80e6-416a-89cc-7d197c2f6c4a\") " pod="openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g" Jul 15 21:40:37.897128 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:37.897106 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a00277f-80e6-416a-89cc-7d197c2f6c4a-istio-envoy\") pod \"e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g\" (UID: \"3a00277f-80e6-416a-89cc-7d197c2f6c4a\") " pod="openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g" Jul 15 21:40:37.897407 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:37.897390 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a00277f-80e6-416a-89cc-7d197c2f6c4a-istio-podinfo\") pod \"e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g\" (UID: \"3a00277f-80e6-416a-89cc-7d197c2f6c4a\") " pod="openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g" Jul 15 21:40:37.904115 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:37.904048 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a00277f-80e6-416a-89cc-7d197c2f6c4a-istio-token\") pod \"e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g\" (UID: \"3a00277f-80e6-416a-89cc-7d197c2f6c4a\") " pod="openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g" Jul 15 21:40:37.904242 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:37.904216 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pq7jw\" (UniqueName: \"kubernetes.io/projected/3a00277f-80e6-416a-89cc-7d197c2f6c4a-kube-api-access-pq7jw\") pod \"e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g\" (UID: \"3a00277f-80e6-416a-89cc-7d197c2f6c4a\") " pod="openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g" Jul 15 21:40:38.025837 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:38.025727 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g" Jul 15 21:40:38.094729 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:38.094603 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:40:38.187719 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:38.187695 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g"] Jul 15 21:40:38.189957 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:40:38.189931 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a00277f_80e6_416a_89cc_7d197c2f6c4a.slice/crio-ec2271f4d2de34e0aa8f736dd8e3e31c4ad80d1652fd3d6290d1ee4807332bfd WatchSource:0}: Error finding container ec2271f4d2de34e0aa8f736dd8e3e31c4ad80d1652fd3d6290d1ee4807332bfd: Status 404 returned error can't find the container with id ec2271f4d2de34e0aa8f736dd8e3e31c4ad80d1652fd3d6290d1ee4807332bfd Jul 15 21:40:38.192676 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:38.192636 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:40:38.192769 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:38.192721 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:40:38.192808 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:38.192763 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:40:39.203394 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:39.203354 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g" event={"ID":"3a00277f-80e6-416a-89cc-7d197c2f6c4a","Type":"ContainerStarted","Data":"62dfa1a7cfcf9c9f6fa21778a5464dda3e655c43919c7d4c36ea78ac88f101e5"} Jul 15 21:40:39.203833 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:39.203455 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g" event={"ID":"3a00277f-80e6-416a-89cc-7d197c2f6c4a","Type":"ContainerStarted","Data":"ec2271f4d2de34e0aa8f736dd8e3e31c4ad80d1652fd3d6290d1ee4807332bfd"} Jul 15 21:40:39.235241 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:39.235190 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g" podStartSLOduration=2.235174297 podStartE2EDuration="2.235174297s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:40:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:40:39.231268787 +0000 UTC m=+2343.772581339" watchObservedRunningTime="2026-07-15 21:40:39.235174297 +0000 UTC m=+2343.776486846" Jul 15 21:40:40.026307 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:40.026261 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g" Jul 15 21:40:40.031700 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:40.031675 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g" Jul 15 21:40:40.208305 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:40.208273 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g" Jul 15 21:40:40.209220 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:40.209197 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g" Jul 15 21:40:41.283037 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:41.283002 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:40:41.438358 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:41.438317 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:40:41.606389 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:41.606348 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:40:41.695811 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:41.695773 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:40:42.221641 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:42.221601 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:40:42.318663 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:42.318607 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:40:45.721944 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:45.721889 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:40:46.060807 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:46.060715 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:40:46.352377 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:46.352299 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l"] Jul 15 21:40:46.355807 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:46.355782 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l" Jul 15 21:40:46.359242 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:46.359218 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-c419c9-openshift-default-dockercfg-z86hz\"" Jul 15 21:40:46.378657 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:46.378627 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l"] Jul 15 21:40:46.384258 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:46.384227 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:40:46.471179 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:46.471132 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2218f8f7-03a3-4158-a386-229948f34e94-workload-certs\") pod \"e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l\" (UID: \"2218f8f7-03a3-4158-a386-229948f34e94\") " pod="openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l" Jul 15 21:40:46.471179 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:46.471176 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2218f8f7-03a3-4158-a386-229948f34e94-istio-envoy\") pod \"e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l\" (UID: \"2218f8f7-03a3-4158-a386-229948f34e94\") " pod="openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l" Jul 15 21:40:46.471451 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:46.471213 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6bsg\" (UniqueName: \"kubernetes.io/projected/2218f8f7-03a3-4158-a386-229948f34e94-kube-api-access-q6bsg\") pod \"e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l\" (UID: \"2218f8f7-03a3-4158-a386-229948f34e94\") " pod="openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l" Jul 15 21:40:46.471451 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:46.471251 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2218f8f7-03a3-4158-a386-229948f34e94-workload-socket\") pod \"e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l\" (UID: \"2218f8f7-03a3-4158-a386-229948f34e94\") " pod="openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l" Jul 15 21:40:46.471451 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:46.471319 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2218f8f7-03a3-4158-a386-229948f34e94-istiod-ca-cert\") pod \"e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l\" (UID: \"2218f8f7-03a3-4158-a386-229948f34e94\") " pod="openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l" Jul 15 21:40:46.471451 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:46.471363 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2218f8f7-03a3-4158-a386-229948f34e94-istio-podinfo\") pod \"e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l\" (UID: \"2218f8f7-03a3-4158-a386-229948f34e94\") " pod="openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l" Jul 15 21:40:46.471451 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:46.471396 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2218f8f7-03a3-4158-a386-229948f34e94-credential-socket\") pod \"e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l\" (UID: \"2218f8f7-03a3-4158-a386-229948f34e94\") " pod="openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l" Jul 15 21:40:46.471451 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:46.471417 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2218f8f7-03a3-4158-a386-229948f34e94-istio-data\") pod \"e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l\" (UID: \"2218f8f7-03a3-4158-a386-229948f34e94\") " pod="openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l" Jul 15 21:40:46.471451 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:46.471434 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2218f8f7-03a3-4158-a386-229948f34e94-istio-token\") pod \"e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l\" (UID: \"2218f8f7-03a3-4158-a386-229948f34e94\") " pod="openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l" Jul 15 21:40:46.572784 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:46.572742 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2218f8f7-03a3-4158-a386-229948f34e94-workload-certs\") pod \"e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l\" (UID: \"2218f8f7-03a3-4158-a386-229948f34e94\") " pod="openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l" Jul 15 21:40:46.572986 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:46.572794 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2218f8f7-03a3-4158-a386-229948f34e94-istio-envoy\") pod \"e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l\" (UID: \"2218f8f7-03a3-4158-a386-229948f34e94\") " pod="openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l" Jul 15 21:40:46.572986 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:46.572834 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6bsg\" (UniqueName: \"kubernetes.io/projected/2218f8f7-03a3-4158-a386-229948f34e94-kube-api-access-q6bsg\") pod \"e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l\" (UID: \"2218f8f7-03a3-4158-a386-229948f34e94\") " pod="openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l" Jul 15 21:40:46.572986 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:46.572873 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2218f8f7-03a3-4158-a386-229948f34e94-workload-socket\") pod \"e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l\" (UID: \"2218f8f7-03a3-4158-a386-229948f34e94\") " pod="openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l" Jul 15 21:40:46.572986 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:46.572911 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2218f8f7-03a3-4158-a386-229948f34e94-istiod-ca-cert\") pod \"e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l\" (UID: \"2218f8f7-03a3-4158-a386-229948f34e94\") " pod="openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l" Jul 15 21:40:46.572986 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:46.572952 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2218f8f7-03a3-4158-a386-229948f34e94-istio-podinfo\") pod \"e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l\" (UID: \"2218f8f7-03a3-4158-a386-229948f34e94\") " pod="openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l" Jul 15 21:40:46.572986 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:46.572981 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2218f8f7-03a3-4158-a386-229948f34e94-credential-socket\") pod \"e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l\" (UID: \"2218f8f7-03a3-4158-a386-229948f34e94\") " pod="openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l" Jul 15 21:40:46.573356 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:46.573008 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2218f8f7-03a3-4158-a386-229948f34e94-istio-data\") pod \"e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l\" (UID: \"2218f8f7-03a3-4158-a386-229948f34e94\") " pod="openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l" Jul 15 21:40:46.573356 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:46.573033 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2218f8f7-03a3-4158-a386-229948f34e94-istio-token\") pod \"e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l\" (UID: \"2218f8f7-03a3-4158-a386-229948f34e94\") " pod="openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l" Jul 15 21:40:46.573356 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:46.573238 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2218f8f7-03a3-4158-a386-229948f34e94-workload-certs\") pod \"e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l\" (UID: \"2218f8f7-03a3-4158-a386-229948f34e94\") " pod="openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l" Jul 15 21:40:46.573356 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:46.573258 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2218f8f7-03a3-4158-a386-229948f34e94-workload-socket\") pod \"e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l\" (UID: \"2218f8f7-03a3-4158-a386-229948f34e94\") " pod="openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l" Jul 15 21:40:46.573566 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:46.573508 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2218f8f7-03a3-4158-a386-229948f34e94-istio-data\") pod \"e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l\" (UID: \"2218f8f7-03a3-4158-a386-229948f34e94\") " pod="openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l" Jul 15 21:40:46.573605 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:46.573569 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2218f8f7-03a3-4158-a386-229948f34e94-credential-socket\") pod \"e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l\" (UID: \"2218f8f7-03a3-4158-a386-229948f34e94\") " pod="openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l" Jul 15 21:40:46.573699 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:46.573674 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2218f8f7-03a3-4158-a386-229948f34e94-istiod-ca-cert\") pod \"e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l\" (UID: \"2218f8f7-03a3-4158-a386-229948f34e94\") " pod="openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l" Jul 15 21:40:46.575275 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:46.575251 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2218f8f7-03a3-4158-a386-229948f34e94-istio-envoy\") pod \"e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l\" (UID: \"2218f8f7-03a3-4158-a386-229948f34e94\") " pod="openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l" Jul 15 21:40:46.575681 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:46.575662 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2218f8f7-03a3-4158-a386-229948f34e94-istio-podinfo\") pod \"e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l\" (UID: \"2218f8f7-03a3-4158-a386-229948f34e94\") " pod="openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l" Jul 15 21:40:46.581687 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:46.581660 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2218f8f7-03a3-4158-a386-229948f34e94-istio-token\") pod \"e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l\" (UID: \"2218f8f7-03a3-4158-a386-229948f34e94\") " pod="openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l" Jul 15 21:40:46.581810 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:46.581788 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6bsg\" (UniqueName: \"kubernetes.io/projected/2218f8f7-03a3-4158-a386-229948f34e94-kube-api-access-q6bsg\") pod \"e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l\" (UID: \"2218f8f7-03a3-4158-a386-229948f34e94\") " pod="openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l" Jul 15 21:40:46.666543 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:46.666505 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l" Jul 15 21:40:46.668254 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:46.668216 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:40:46.810943 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:46.810911 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l"] Jul 15 21:40:46.812161 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:40:46.812134 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2218f8f7_03a3_4158_a386_229948f34e94.slice/crio-68b51f2890b258ec66989d8b26016d1e9461bffc221587e85561538cc600562a WatchSource:0}: Error finding container 68b51f2890b258ec66989d8b26016d1e9461bffc221587e85561538cc600562a: Status 404 returned error can't find the container with id 68b51f2890b258ec66989d8b26016d1e9461bffc221587e85561538cc600562a Jul 15 21:40:46.814315 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:46.814282 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:40:46.814419 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:46.814347 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:40:46.814419 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:46.814393 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:40:47.244681 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:47.244646 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l" event={"ID":"2218f8f7-03a3-4158-a386-229948f34e94","Type":"ContainerStarted","Data":"fd2135ff498e9b6016947b51729dc1ab380cef4767fe3243edf3c13c9c1c1fc7"} Jul 15 21:40:47.244903 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:47.244701 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l" event={"ID":"2218f8f7-03a3-4158-a386-229948f34e94","Type":"ContainerStarted","Data":"68b51f2890b258ec66989d8b26016d1e9461bffc221587e85561538cc600562a"} Jul 15 21:40:47.268935 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:47.268879 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l" podStartSLOduration=1.268862415 podStartE2EDuration="1.268862415s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:40:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:40:47.267239092 +0000 UTC m=+2351.808551656" watchObservedRunningTime="2026-07-15 21:40:47.268862415 +0000 UTC m=+2351.810174966" Jul 15 21:40:47.667415 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:47.667378 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l" Jul 15 21:40:47.672371 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:47.672343 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l" Jul 15 21:40:48.250858 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:48.250820 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l" Jul 15 21:40:48.251728 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:48.251709 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l" Jul 15 21:40:48.798089 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:48.794942 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:40:49.299843 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:49.299789 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:40:49.386419 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:49.386369 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:40:49.577493 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:49.577390 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:40:49.785488 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:49.785451 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:40:50.109049 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:50.109011 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:40:50.211160 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:50.211125 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:40:53.741867 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:53.741825 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:40:56.250337 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:56.250292 2579 patch_prober.go:28] interesting pod/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 15 21:40:56.250892 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:56.250375 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g" podUID="3a00277f-80e6-416a-89cc-7d197c2f6c4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 15 21:40:57.303853 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:57.303822 2579 generic.go:356] "Generic (PLEG): container finished" podID="3a00277f-80e6-416a-89cc-7d197c2f6c4a" containerID="62dfa1a7cfcf9c9f6fa21778a5464dda3e655c43919c7d4c36ea78ac88f101e5" exitCode=0 Jul 15 21:40:57.304311 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:57.303871 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g" event={"ID":"3a00277f-80e6-416a-89cc-7d197c2f6c4a","Type":"ContainerDied","Data":"62dfa1a7cfcf9c9f6fa21778a5464dda3e655c43919c7d4c36ea78ac88f101e5"} Jul 15 21:40:57.304311 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:57.304244 2579 scope.go:117] "RemoveContainer" containerID="62dfa1a7cfcf9c9f6fa21778a5464dda3e655c43919c7d4c36ea78ac88f101e5" Jul 15 21:40:57.305225 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:57.305192 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:40:57.305340 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:57.305279 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:40:57.305340 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:57.305326 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:40:58.311074 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:58.311022 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g" event={"ID":"3a00277f-80e6-416a-89cc-7d197c2f6c4a","Type":"ContainerStarted","Data":"f98d426057ff1dc000d518d8556ad3c74e1f882fbd8a8f7c14724a933a25e39e"} Jul 15 21:40:59.026220 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:59.026183 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g" Jul 15 21:40:59.026449 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:59.026314 2579 patch_prober.go:28] interesting pod/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 15 21:40:59.026449 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:40:59.026358 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g" podUID="3a00277f-80e6-416a-89cc-7d197c2f6c4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 15 21:41:00.027035 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:41:00.026996 2579 patch_prober.go:28] interesting pod/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 15 21:41:00.027646 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:41:00.027610 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g" podUID="3a00277f-80e6-416a-89cc-7d197c2f6c4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 15 21:41:01.026764 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:41:01.026716 2579 patch_prober.go:28] interesting pod/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 15 21:41:01.026990 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:41:01.026784 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g" podUID="3a00277f-80e6-416a-89cc-7d197c2f6c4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 15 21:41:02.032091 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:41:02.029784 2579 patch_prober.go:28] interesting pod/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 15 21:41:02.032091 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:41:02.029871 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g" podUID="3a00277f-80e6-416a-89cc-7d197c2f6c4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 15 21:41:03.030622 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:41:03.030589 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g" Jul 15 21:41:03.030981 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:41:03.030958 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g" Jul 15 21:41:03.032020 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:41:03.031991 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g" Jul 15 21:41:03.496173 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:41:03.496132 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:41:04.011424 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:41:04.011386 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:41:04.293278 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:41:04.293193 2579 patch_prober.go:28] interesting pod/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 15 21:41:04.293278 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:41:04.293265 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l" podUID="2218f8f7-03a3-4158-a386-229948f34e94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 15 21:41:05.358619 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:41:05.358587 2579 generic.go:356] "Generic (PLEG): container finished" podID="2218f8f7-03a3-4158-a386-229948f34e94" containerID="fd2135ff498e9b6016947b51729dc1ab380cef4767fe3243edf3c13c9c1c1fc7" exitCode=0 Jul 15 21:41:05.359030 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:41:05.358635 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l" event={"ID":"2218f8f7-03a3-4158-a386-229948f34e94","Type":"ContainerDied","Data":"fd2135ff498e9b6016947b51729dc1ab380cef4767fe3243edf3c13c9c1c1fc7"} Jul 15 21:41:05.359132 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:41:05.359115 2579 scope.go:117] "RemoveContainer" containerID="fd2135ff498e9b6016947b51729dc1ab380cef4767fe3243edf3c13c9c1c1fc7" Jul 15 21:41:05.360303 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:41:05.360268 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:41:05.360434 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:41:05.360359 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:41:05.360434 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:41:05.360404 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:41:05.892126 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:41:05.892083 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:41:06.366524 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:41:06.366486 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l" event={"ID":"2218f8f7-03a3-4158-a386-229948f34e94","Type":"ContainerStarted","Data":"0e480a5bad36b854682220546b0e447861af7cd10aad7afcd6ceb5a4cf4c0938"} Jul 15 21:41:06.666855 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:41:06.666753 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l" Jul 15 21:41:06.667040 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:41:06.667014 2579 patch_prober.go:28] interesting pod/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 15 21:41:06.667116 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:41:06.667095 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l" podUID="2218f8f7-03a3-4158-a386-229948f34e94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 15 21:41:07.667505 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:41:07.667465 2579 patch_prober.go:28] interesting pod/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 15 21:41:07.667893 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:41:07.667518 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l" podUID="2218f8f7-03a3-4158-a386-229948f34e94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 15 21:41:08.667256 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:41:08.667218 2579 patch_prober.go:28] interesting pod/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 15 21:41:08.667453 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:41:08.667278 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l" podUID="2218f8f7-03a3-4158-a386-229948f34e94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 15 21:41:09.667724 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:41:09.667688 2579 patch_prober.go:28] interesting pod/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 15 21:41:09.668257 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:41:09.667745 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l" podUID="2218f8f7-03a3-4158-a386-229948f34e94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 15 21:41:10.670685 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:41:10.670651 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l" Jul 15 21:41:10.671189 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:41:10.671049 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l" Jul 15 21:41:10.671660 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:41:10.671641 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l" Jul 15 21:41:11.281573 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:41:11.281536 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:41:11.401298 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:41:11.401264 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:41:16.292759 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:41:16.292716 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:41:19.323570 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:41:19.323529 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:41:19.522519 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:41:19.522481 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:41:23.900022 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:41:23.899981 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:41:23.982217 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:41:23.982169 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:41:25.930297 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:41:25.930263 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:41:31.300306 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:41:31.300260 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:41:36.168276 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:41:36.168251 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m5r2m_0db09c1e-d24d-4519-93c8-c185018c3d24/ovn-acl-logging/0.log" Jul 15 21:41:36.170976 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:41:36.170953 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m5r2m_0db09c1e-d24d-4519-93c8-c185018c3d24/ovn-acl-logging/0.log" Jul 15 21:41:42.128548 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:41:42.128507 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:41:43.014272 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:41:43.014230 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:41:43.843931 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:41:43.843897 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:41:44.028761 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:41:44.028720 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:41:47.195494 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:41:47.195456 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:41:53.660369 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:41:53.660329 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:41:58.236023 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:41:58.235922 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:42:03.987580 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:03.987543 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:42:04.114656 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:04.114615 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:42:04.186368 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:04.186332 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:42:08.976163 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:08.976120 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g"] Jul 15 21:42:08.976659 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:08.976396 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g" podUID="3a00277f-80e6-416a-89cc-7d197c2f6c4a" containerName="istio-proxy" containerID="cri-o://f98d426057ff1dc000d518d8556ad3c74e1f882fbd8a8f7c14724a933a25e39e" gracePeriod=30 Jul 15 21:42:08.988701 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:08.988659 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:42:09.284120 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:09.284000 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:42:10.077915 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:10.077882 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:42:10.178396 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:10.178358 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:42:14.227242 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:14.227211 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g" Jul 15 21:42:14.288049 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:14.287959 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a00277f-80e6-416a-89cc-7d197c2f6c4a-istio-envoy\") pod \"3a00277f-80e6-416a-89cc-7d197c2f6c4a\" (UID: \"3a00277f-80e6-416a-89cc-7d197c2f6c4a\") " Jul 15 21:42:14.288049 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:14.288003 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a00277f-80e6-416a-89cc-7d197c2f6c4a-istiod-ca-cert\") pod \"3a00277f-80e6-416a-89cc-7d197c2f6c4a\" (UID: \"3a00277f-80e6-416a-89cc-7d197c2f6c4a\") " Jul 15 21:42:14.288049 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:14.288027 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a00277f-80e6-416a-89cc-7d197c2f6c4a-workload-socket\") pod \"3a00277f-80e6-416a-89cc-7d197c2f6c4a\" (UID: \"3a00277f-80e6-416a-89cc-7d197c2f6c4a\") " Jul 15 21:42:14.288349 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:14.288056 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a00277f-80e6-416a-89cc-7d197c2f6c4a-istio-data\") pod \"3a00277f-80e6-416a-89cc-7d197c2f6c4a\" (UID: \"3a00277f-80e6-416a-89cc-7d197c2f6c4a\") " Jul 15 21:42:14.288349 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:14.288101 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a00277f-80e6-416a-89cc-7d197c2f6c4a-istio-podinfo\") pod \"3a00277f-80e6-416a-89cc-7d197c2f6c4a\" (UID: \"3a00277f-80e6-416a-89cc-7d197c2f6c4a\") " Jul 15 21:42:14.288349 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:14.288137 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pq7jw\" (UniqueName: \"kubernetes.io/projected/3a00277f-80e6-416a-89cc-7d197c2f6c4a-kube-api-access-pq7jw\") pod \"3a00277f-80e6-416a-89cc-7d197c2f6c4a\" (UID: \"3a00277f-80e6-416a-89cc-7d197c2f6c4a\") " Jul 15 21:42:14.288349 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:14.288230 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a00277f-80e6-416a-89cc-7d197c2f6c4a-credential-socket\") pod \"3a00277f-80e6-416a-89cc-7d197c2f6c4a\" (UID: \"3a00277f-80e6-416a-89cc-7d197c2f6c4a\") " Jul 15 21:42:14.288349 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:14.288259 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a00277f-80e6-416a-89cc-7d197c2f6c4a-workload-certs\") pod \"3a00277f-80e6-416a-89cc-7d197c2f6c4a\" (UID: \"3a00277f-80e6-416a-89cc-7d197c2f6c4a\") " Jul 15 21:42:14.288349 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:14.288284 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a00277f-80e6-416a-89cc-7d197c2f6c4a-istio-token\") pod \"3a00277f-80e6-416a-89cc-7d197c2f6c4a\" (UID: \"3a00277f-80e6-416a-89cc-7d197c2f6c4a\") " Jul 15 21:42:14.288662 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:14.288366 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a00277f-80e6-416a-89cc-7d197c2f6c4a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3a00277f-80e6-416a-89cc-7d197c2f6c4a" (UID: "3a00277f-80e6-416a-89cc-7d197c2f6c4a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:42:14.288662 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:14.288513 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a00277f-80e6-416a-89cc-7d197c2f6c4a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3a00277f-80e6-416a-89cc-7d197c2f6c4a" (UID: "3a00277f-80e6-416a-89cc-7d197c2f6c4a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 21:42:14.288662 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:14.288622 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a00277f-80e6-416a-89cc-7d197c2f6c4a-istiod-ca-cert\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:42:14.288662 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:14.288643 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a00277f-80e6-416a-89cc-7d197c2f6c4a-workload-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:42:14.288873 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:14.288850 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a00277f-80e6-416a-89cc-7d197c2f6c4a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3a00277f-80e6-416a-89cc-7d197c2f6c4a" (UID: "3a00277f-80e6-416a-89cc-7d197c2f6c4a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:42:14.288937 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:14.288878 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a00277f-80e6-416a-89cc-7d197c2f6c4a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3a00277f-80e6-416a-89cc-7d197c2f6c4a" (UID: "3a00277f-80e6-416a-89cc-7d197c2f6c4a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:42:14.289129 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:14.289109 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a00277f-80e6-416a-89cc-7d197c2f6c4a-istio-data" (OuterVolumeSpecName: "istio-data") pod "3a00277f-80e6-416a-89cc-7d197c2f6c4a" (UID: "3a00277f-80e6-416a-89cc-7d197c2f6c4a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:42:14.290615 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:14.290594 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a00277f-80e6-416a-89cc-7d197c2f6c4a-istio-token" (OuterVolumeSpecName: "istio-token") pod "3a00277f-80e6-416a-89cc-7d197c2f6c4a" (UID: "3a00277f-80e6-416a-89cc-7d197c2f6c4a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:42:14.290734 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:14.290612 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a00277f-80e6-416a-89cc-7d197c2f6c4a-kube-api-access-pq7jw" (OuterVolumeSpecName: "kube-api-access-pq7jw") pod "3a00277f-80e6-416a-89cc-7d197c2f6c4a" (UID: "3a00277f-80e6-416a-89cc-7d197c2f6c4a"). InnerVolumeSpecName "kube-api-access-pq7jw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:42:14.290734 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:14.290691 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3a00277f-80e6-416a-89cc-7d197c2f6c4a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3a00277f-80e6-416a-89cc-7d197c2f6c4a" (UID: "3a00277f-80e6-416a-89cc-7d197c2f6c4a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 21:42:14.290820 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:14.290723 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a00277f-80e6-416a-89cc-7d197c2f6c4a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3a00277f-80e6-416a-89cc-7d197c2f6c4a" (UID: "3a00277f-80e6-416a-89cc-7d197c2f6c4a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:42:14.389719 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:14.389678 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a00277f-80e6-416a-89cc-7d197c2f6c4a-workload-certs\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:42:14.389719 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:14.389710 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a00277f-80e6-416a-89cc-7d197c2f6c4a-istio-token\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:42:14.389719 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:14.389722 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a00277f-80e6-416a-89cc-7d197c2f6c4a-istio-envoy\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:42:14.389972 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:14.389733 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a00277f-80e6-416a-89cc-7d197c2f6c4a-istio-data\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:42:14.389972 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:14.389744 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a00277f-80e6-416a-89cc-7d197c2f6c4a-istio-podinfo\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:42:14.389972 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:14.389755 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pq7jw\" (UniqueName: \"kubernetes.io/projected/3a00277f-80e6-416a-89cc-7d197c2f6c4a-kube-api-access-pq7jw\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:42:14.389972 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:14.389769 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a00277f-80e6-416a-89cc-7d197c2f6c4a-credential-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:42:14.766286 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:14.766244 2579 generic.go:356] "Generic (PLEG): container finished" podID="3a00277f-80e6-416a-89cc-7d197c2f6c4a" containerID="f98d426057ff1dc000d518d8556ad3c74e1f882fbd8a8f7c14724a933a25e39e" exitCode=0 Jul 15 21:42:14.766471 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:14.766309 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g" Jul 15 21:42:14.766471 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:14.766312 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g" event={"ID":"3a00277f-80e6-416a-89cc-7d197c2f6c4a","Type":"ContainerDied","Data":"f98d426057ff1dc000d518d8556ad3c74e1f882fbd8a8f7c14724a933a25e39e"} Jul 15 21:42:14.766471 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:14.766456 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g" event={"ID":"3a00277f-80e6-416a-89cc-7d197c2f6c4a","Type":"ContainerDied","Data":"ec2271f4d2de34e0aa8f736dd8e3e31c4ad80d1652fd3d6290d1ee4807332bfd"} Jul 15 21:42:14.766579 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:14.766479 2579 scope.go:117] "RemoveContainer" containerID="f98d426057ff1dc000d518d8556ad3c74e1f882fbd8a8f7c14724a933a25e39e" Jul 15 21:42:14.776444 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:14.776426 2579 scope.go:117] "RemoveContainer" containerID="62dfa1a7cfcf9c9f6fa21778a5464dda3e655c43919c7d4c36ea78ac88f101e5" Jul 15 21:42:14.784766 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:14.784749 2579 scope.go:117] "RemoveContainer" containerID="f98d426057ff1dc000d518d8556ad3c74e1f882fbd8a8f7c14724a933a25e39e" Jul 15 21:42:14.785096 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:42:14.785045 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f98d426057ff1dc000d518d8556ad3c74e1f882fbd8a8f7c14724a933a25e39e\": container with ID starting with f98d426057ff1dc000d518d8556ad3c74e1f882fbd8a8f7c14724a933a25e39e not found: ID does not exist" containerID="f98d426057ff1dc000d518d8556ad3c74e1f882fbd8a8f7c14724a933a25e39e" Jul 15 21:42:14.785197 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:14.785103 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f98d426057ff1dc000d518d8556ad3c74e1f882fbd8a8f7c14724a933a25e39e"} err="failed to get container status \"f98d426057ff1dc000d518d8556ad3c74e1f882fbd8a8f7c14724a933a25e39e\": rpc error: code = NotFound desc = could not find container \"f98d426057ff1dc000d518d8556ad3c74e1f882fbd8a8f7c14724a933a25e39e\": container with ID starting with f98d426057ff1dc000d518d8556ad3c74e1f882fbd8a8f7c14724a933a25e39e not found: ID does not exist" Jul 15 21:42:14.785197 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:14.785121 2579 scope.go:117] "RemoveContainer" containerID="62dfa1a7cfcf9c9f6fa21778a5464dda3e655c43919c7d4c36ea78ac88f101e5" Jul 15 21:42:14.785373 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:42:14.785355 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62dfa1a7cfcf9c9f6fa21778a5464dda3e655c43919c7d4c36ea78ac88f101e5\": container with ID starting with 62dfa1a7cfcf9c9f6fa21778a5464dda3e655c43919c7d4c36ea78ac88f101e5 not found: ID does not exist" containerID="62dfa1a7cfcf9c9f6fa21778a5464dda3e655c43919c7d4c36ea78ac88f101e5" Jul 15 21:42:14.785421 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:14.785378 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62dfa1a7cfcf9c9f6fa21778a5464dda3e655c43919c7d4c36ea78ac88f101e5"} err="failed to get container status \"62dfa1a7cfcf9c9f6fa21778a5464dda3e655c43919c7d4c36ea78ac88f101e5\": rpc error: code = NotFound desc = could not find container \"62dfa1a7cfcf9c9f6fa21778a5464dda3e655c43919c7d4c36ea78ac88f101e5\": container with ID starting with 62dfa1a7cfcf9c9f6fa21778a5464dda3e655c43919c7d4c36ea78ac88f101e5 not found: ID does not exist" Jul 15 21:42:14.791361 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:14.791337 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g"] Jul 15 21:42:14.795190 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:14.795163 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-7c4663-openshift-default-568bbbcb67-tjs8g"] Jul 15 21:42:15.996896 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:15.996860 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a00277f-80e6-416a-89cc-7d197c2f6c4a" path="/var/lib/kubelet/pods/3a00277f-80e6-416a-89cc-7d197c2f6c4a/volumes" Jul 15 21:42:19.868390 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:19.868356 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:42:23.004543 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:23.004501 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:42:24.471739 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:24.471701 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:42:30.733903 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:30.733868 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:42:30.834498 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:30.834461 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:42:35.241740 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:35.241710 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l"] Jul 15 21:42:35.242141 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:35.241957 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l" podUID="2218f8f7-03a3-4158-a386-229948f34e94" containerName="istio-proxy" containerID="cri-o://0e480a5bad36b854682220546b0e447861af7cd10aad7afcd6ceb5a4cf4c0938" gracePeriod=30 Jul 15 21:42:35.283816 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:35.283780 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:42:35.573030 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:35.572943 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:42:35.976761 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:35.976718 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:42:36.304803 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:36.304772 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl"] Jul 15 21:42:36.305158 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:36.305139 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a00277f-80e6-416a-89cc-7d197c2f6c4a" containerName="istio-proxy" Jul 15 21:42:36.305158 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:36.305152 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a00277f-80e6-416a-89cc-7d197c2f6c4a" containerName="istio-proxy" Jul 15 21:42:36.305231 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:36.305170 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a00277f-80e6-416a-89cc-7d197c2f6c4a" containerName="istio-proxy" Jul 15 21:42:36.305231 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:36.305176 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a00277f-80e6-416a-89cc-7d197c2f6c4a" containerName="istio-proxy" Jul 15 21:42:36.305305 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:36.305232 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a00277f-80e6-416a-89cc-7d197c2f6c4a" containerName="istio-proxy" Jul 15 21:42:36.308004 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:36.307982 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl" Jul 15 21:42:36.311872 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:36.311847 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-652862-openshift-default-dockercfg-999n7\"" Jul 15 21:42:36.326943 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:36.326906 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl"] Jul 15 21:42:36.343254 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:36.343223 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:42:36.382158 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:36.382114 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e852dd0a-92d3-45de-8922-5e801112b481-credential-socket\") pod \"e2e-api-a-652862-openshift-default-c9d95f897-mzsgl\" (UID: \"e852dd0a-92d3-45de-8922-5e801112b481\") " pod="openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl" Jul 15 21:42:36.382451 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:36.382201 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e852dd0a-92d3-45de-8922-5e801112b481-istiod-ca-cert\") pod \"e2e-api-a-652862-openshift-default-c9d95f897-mzsgl\" (UID: \"e852dd0a-92d3-45de-8922-5e801112b481\") " pod="openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl" Jul 15 21:42:36.382451 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:36.382227 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9dxw\" (UniqueName: \"kubernetes.io/projected/e852dd0a-92d3-45de-8922-5e801112b481-kube-api-access-q9dxw\") pod \"e2e-api-a-652862-openshift-default-c9d95f897-mzsgl\" (UID: \"e852dd0a-92d3-45de-8922-5e801112b481\") " pod="openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl" Jul 15 21:42:36.382451 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:36.382310 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e852dd0a-92d3-45de-8922-5e801112b481-istio-data\") pod \"e2e-api-a-652862-openshift-default-c9d95f897-mzsgl\" (UID: \"e852dd0a-92d3-45de-8922-5e801112b481\") " pod="openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl" Jul 15 21:42:36.382451 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:36.382391 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e852dd0a-92d3-45de-8922-5e801112b481-workload-socket\") pod \"e2e-api-a-652862-openshift-default-c9d95f897-mzsgl\" (UID: \"e852dd0a-92d3-45de-8922-5e801112b481\") " pod="openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl" Jul 15 21:42:36.382451 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:36.382412 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e852dd0a-92d3-45de-8922-5e801112b481-workload-certs\") pod \"e2e-api-a-652862-openshift-default-c9d95f897-mzsgl\" (UID: \"e852dd0a-92d3-45de-8922-5e801112b481\") " pod="openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl" Jul 15 21:42:36.382451 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:36.382432 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e852dd0a-92d3-45de-8922-5e801112b481-istio-token\") pod \"e2e-api-a-652862-openshift-default-c9d95f897-mzsgl\" (UID: \"e852dd0a-92d3-45de-8922-5e801112b481\") " pod="openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl" Jul 15 21:42:36.382792 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:36.382472 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e852dd0a-92d3-45de-8922-5e801112b481-istio-podinfo\") pod \"e2e-api-a-652862-openshift-default-c9d95f897-mzsgl\" (UID: \"e852dd0a-92d3-45de-8922-5e801112b481\") " pod="openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl" Jul 15 21:42:36.382792 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:36.382537 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e852dd0a-92d3-45de-8922-5e801112b481-istio-envoy\") pod \"e2e-api-a-652862-openshift-default-c9d95f897-mzsgl\" (UID: \"e852dd0a-92d3-45de-8922-5e801112b481\") " pod="openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl" Jul 15 21:42:36.483830 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:36.483793 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e852dd0a-92d3-45de-8922-5e801112b481-istiod-ca-cert\") pod \"e2e-api-a-652862-openshift-default-c9d95f897-mzsgl\" (UID: \"e852dd0a-92d3-45de-8922-5e801112b481\") " pod="openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl" Jul 15 21:42:36.483830 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:36.483827 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9dxw\" (UniqueName: \"kubernetes.io/projected/e852dd0a-92d3-45de-8922-5e801112b481-kube-api-access-q9dxw\") pod \"e2e-api-a-652862-openshift-default-c9d95f897-mzsgl\" (UID: \"e852dd0a-92d3-45de-8922-5e801112b481\") " pod="openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl" Jul 15 21:42:36.484108 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:36.483882 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e852dd0a-92d3-45de-8922-5e801112b481-istio-data\") pod \"e2e-api-a-652862-openshift-default-c9d95f897-mzsgl\" (UID: \"e852dd0a-92d3-45de-8922-5e801112b481\") " pod="openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl" Jul 15 21:42:36.484108 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:36.483909 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e852dd0a-92d3-45de-8922-5e801112b481-workload-socket\") pod \"e2e-api-a-652862-openshift-default-c9d95f897-mzsgl\" (UID: \"e852dd0a-92d3-45de-8922-5e801112b481\") " pod="openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl" Jul 15 21:42:36.484108 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:36.483928 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e852dd0a-92d3-45de-8922-5e801112b481-workload-certs\") pod \"e2e-api-a-652862-openshift-default-c9d95f897-mzsgl\" (UID: \"e852dd0a-92d3-45de-8922-5e801112b481\") " pod="openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl" Jul 15 21:42:36.484108 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:36.484029 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e852dd0a-92d3-45de-8922-5e801112b481-istio-token\") pod \"e2e-api-a-652862-openshift-default-c9d95f897-mzsgl\" (UID: \"e852dd0a-92d3-45de-8922-5e801112b481\") " pod="openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl" Jul 15 21:42:36.484336 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:36.484121 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e852dd0a-92d3-45de-8922-5e801112b481-istio-podinfo\") pod \"e2e-api-a-652862-openshift-default-c9d95f897-mzsgl\" (UID: \"e852dd0a-92d3-45de-8922-5e801112b481\") " pod="openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl" Jul 15 21:42:36.484336 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:36.484160 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e852dd0a-92d3-45de-8922-5e801112b481-istio-envoy\") pod \"e2e-api-a-652862-openshift-default-c9d95f897-mzsgl\" (UID: \"e852dd0a-92d3-45de-8922-5e801112b481\") " pod="openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl" Jul 15 21:42:36.484336 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:36.484193 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e852dd0a-92d3-45de-8922-5e801112b481-credential-socket\") pod \"e2e-api-a-652862-openshift-default-c9d95f897-mzsgl\" (UID: \"e852dd0a-92d3-45de-8922-5e801112b481\") " pod="openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl" Jul 15 21:42:36.484509 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:36.484352 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e852dd0a-92d3-45de-8922-5e801112b481-workload-socket\") pod \"e2e-api-a-652862-openshift-default-c9d95f897-mzsgl\" (UID: \"e852dd0a-92d3-45de-8922-5e801112b481\") " pod="openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl" Jul 15 21:42:36.484509 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:36.484492 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e852dd0a-92d3-45de-8922-5e801112b481-istio-data\") pod \"e2e-api-a-652862-openshift-default-c9d95f897-mzsgl\" (UID: \"e852dd0a-92d3-45de-8922-5e801112b481\") " pod="openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl" Jul 15 21:42:36.484618 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:36.484515 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e852dd0a-92d3-45de-8922-5e801112b481-workload-certs\") pod \"e2e-api-a-652862-openshift-default-c9d95f897-mzsgl\" (UID: \"e852dd0a-92d3-45de-8922-5e801112b481\") " pod="openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl" Jul 15 21:42:36.484675 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:36.484638 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e852dd0a-92d3-45de-8922-5e801112b481-istiod-ca-cert\") pod \"e2e-api-a-652862-openshift-default-c9d95f897-mzsgl\" (UID: \"e852dd0a-92d3-45de-8922-5e801112b481\") " pod="openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl" Jul 15 21:42:36.484675 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:36.484659 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e852dd0a-92d3-45de-8922-5e801112b481-credential-socket\") pod \"e2e-api-a-652862-openshift-default-c9d95f897-mzsgl\" (UID: \"e852dd0a-92d3-45de-8922-5e801112b481\") " pod="openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl" Jul 15 21:42:36.486584 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:36.486558 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e852dd0a-92d3-45de-8922-5e801112b481-istio-envoy\") pod \"e2e-api-a-652862-openshift-default-c9d95f897-mzsgl\" (UID: \"e852dd0a-92d3-45de-8922-5e801112b481\") " pod="openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl" Jul 15 21:42:36.486808 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:36.486788 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e852dd0a-92d3-45de-8922-5e801112b481-istio-podinfo\") pod \"e2e-api-a-652862-openshift-default-c9d95f897-mzsgl\" (UID: \"e852dd0a-92d3-45de-8922-5e801112b481\") " pod="openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl" Jul 15 21:42:36.492471 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:36.492449 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e852dd0a-92d3-45de-8922-5e801112b481-istio-token\") pod \"e2e-api-a-652862-openshift-default-c9d95f897-mzsgl\" (UID: \"e852dd0a-92d3-45de-8922-5e801112b481\") " pod="openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl" Jul 15 21:42:36.492840 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:36.492815 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9dxw\" (UniqueName: \"kubernetes.io/projected/e852dd0a-92d3-45de-8922-5e801112b481-kube-api-access-q9dxw\") pod \"e2e-api-a-652862-openshift-default-c9d95f897-mzsgl\" (UID: \"e852dd0a-92d3-45de-8922-5e801112b481\") " pod="openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl" Jul 15 21:42:36.601147 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:36.600995 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:42:36.618945 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:36.618912 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl" Jul 15 21:42:36.770404 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:36.770368 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl"] Jul 15 21:42:36.772428 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:42:36.772400 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode852dd0a_92d3_45de_8922_5e801112b481.slice/crio-71002235fe5f6dc99cfb1d7829ef54786114d0be455c158d592a7571bf031923 WatchSource:0}: Error finding container 71002235fe5f6dc99cfb1d7829ef54786114d0be455c158d592a7571bf031923: Status 404 returned error can't find the container with id 71002235fe5f6dc99cfb1d7829ef54786114d0be455c158d592a7571bf031923 Jul 15 21:42:36.774573 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:36.774541 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:42:36.774657 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:36.774605 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:42:36.774657 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:36.774633 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:42:36.887573 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:36.887533 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl" event={"ID":"e852dd0a-92d3-45de-8922-5e801112b481","Type":"ContainerStarted","Data":"448dc604e87decf5e0cf3d194bdc3f5b7ac7ace4d2636b066e6fb2b444461ae9"} Jul 15 21:42:36.887780 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:36.887605 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl" event={"ID":"e852dd0a-92d3-45de-8922-5e801112b481","Type":"ContainerStarted","Data":"71002235fe5f6dc99cfb1d7829ef54786114d0be455c158d592a7571bf031923"} Jul 15 21:42:36.916925 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:36.916861 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl" podStartSLOduration=0.916842901 podStartE2EDuration="916.842901ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:42:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:42:36.91475124 +0000 UTC m=+2461.456063794" watchObservedRunningTime="2026-07-15 21:42:36.916842901 +0000 UTC m=+2461.458155450" Jul 15 21:42:37.619737 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:37.619702 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl" Jul 15 21:42:37.625033 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:37.625005 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl" Jul 15 21:42:37.893909 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:37.893815 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl" Jul 15 21:42:37.894916 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:37.894894 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl" Jul 15 21:42:38.967468 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:38.967427 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:42:39.077522 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:39.077483 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:42:39.172987 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:39.172956 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:42:39.266106 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:39.265983 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:42:39.790028 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:39.790000 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:42:39.890306 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:39.890267 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:42:40.503799 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:40.503729 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l" Jul 15 21:42:40.620127 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:40.620088 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2218f8f7-03a3-4158-a386-229948f34e94-istio-envoy\") pod \"2218f8f7-03a3-4158-a386-229948f34e94\" (UID: \"2218f8f7-03a3-4158-a386-229948f34e94\") " Jul 15 21:42:40.620127 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:40.620126 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2218f8f7-03a3-4158-a386-229948f34e94-credential-socket\") pod \"2218f8f7-03a3-4158-a386-229948f34e94\" (UID: \"2218f8f7-03a3-4158-a386-229948f34e94\") " Jul 15 21:42:40.620363 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:40.620169 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2218f8f7-03a3-4158-a386-229948f34e94-istiod-ca-cert\") pod \"2218f8f7-03a3-4158-a386-229948f34e94\" (UID: \"2218f8f7-03a3-4158-a386-229948f34e94\") " Jul 15 21:42:40.620363 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:40.620196 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2218f8f7-03a3-4158-a386-229948f34e94-istio-token\") pod \"2218f8f7-03a3-4158-a386-229948f34e94\" (UID: \"2218f8f7-03a3-4158-a386-229948f34e94\") " Jul 15 21:42:40.620363 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:40.620229 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q6bsg\" (UniqueName: \"kubernetes.io/projected/2218f8f7-03a3-4158-a386-229948f34e94-kube-api-access-q6bsg\") pod \"2218f8f7-03a3-4158-a386-229948f34e94\" (UID: \"2218f8f7-03a3-4158-a386-229948f34e94\") " Jul 15 21:42:40.620363 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:40.620252 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2218f8f7-03a3-4158-a386-229948f34e94-istio-podinfo\") pod \"2218f8f7-03a3-4158-a386-229948f34e94\" (UID: \"2218f8f7-03a3-4158-a386-229948f34e94\") " Jul 15 21:42:40.620529 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:40.620467 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2218f8f7-03a3-4158-a386-229948f34e94-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2218f8f7-03a3-4158-a386-229948f34e94" (UID: "2218f8f7-03a3-4158-a386-229948f34e94"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:42:40.620529 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:40.620521 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2218f8f7-03a3-4158-a386-229948f34e94-istio-data\") pod \"2218f8f7-03a3-4158-a386-229948f34e94\" (UID: \"2218f8f7-03a3-4158-a386-229948f34e94\") " Jul 15 21:42:40.620620 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:40.620569 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2218f8f7-03a3-4158-a386-229948f34e94-workload-certs\") pod \"2218f8f7-03a3-4158-a386-229948f34e94\" (UID: \"2218f8f7-03a3-4158-a386-229948f34e94\") " Jul 15 21:42:40.620671 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:40.620640 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2218f8f7-03a3-4158-a386-229948f34e94-workload-socket\") pod \"2218f8f7-03a3-4158-a386-229948f34e94\" (UID: \"2218f8f7-03a3-4158-a386-229948f34e94\") " Jul 15 21:42:40.620671 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:40.620656 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2218f8f7-03a3-4158-a386-229948f34e94-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2218f8f7-03a3-4158-a386-229948f34e94" (UID: "2218f8f7-03a3-4158-a386-229948f34e94"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 21:42:40.621037 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:40.620919 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2218f8f7-03a3-4158-a386-229948f34e94-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2218f8f7-03a3-4158-a386-229948f34e94" (UID: "2218f8f7-03a3-4158-a386-229948f34e94"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:42:40.621037 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:40.620969 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2218f8f7-03a3-4158-a386-229948f34e94-credential-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:42:40.621037 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:40.620990 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2218f8f7-03a3-4158-a386-229948f34e94-istiod-ca-cert\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:42:40.621037 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:40.621003 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2218f8f7-03a3-4158-a386-229948f34e94-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2218f8f7-03a3-4158-a386-229948f34e94" (UID: "2218f8f7-03a3-4158-a386-229948f34e94"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:42:40.622092 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:40.622046 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2218f8f7-03a3-4158-a386-229948f34e94-istio-data" (OuterVolumeSpecName: "istio-data") pod "2218f8f7-03a3-4158-a386-229948f34e94" (UID: "2218f8f7-03a3-4158-a386-229948f34e94"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:42:40.622614 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:40.622593 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2218f8f7-03a3-4158-a386-229948f34e94-kube-api-access-q6bsg" (OuterVolumeSpecName: "kube-api-access-q6bsg") pod "2218f8f7-03a3-4158-a386-229948f34e94" (UID: "2218f8f7-03a3-4158-a386-229948f34e94"). InnerVolumeSpecName "kube-api-access-q6bsg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:42:40.622614 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:40.622605 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2218f8f7-03a3-4158-a386-229948f34e94-istio-token" (OuterVolumeSpecName: "istio-token") pod "2218f8f7-03a3-4158-a386-229948f34e94" (UID: "2218f8f7-03a3-4158-a386-229948f34e94"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:42:40.622802 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:40.622785 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2218f8f7-03a3-4158-a386-229948f34e94-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2218f8f7-03a3-4158-a386-229948f34e94" (UID: "2218f8f7-03a3-4158-a386-229948f34e94"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:42:40.622862 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:40.622787 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2218f8f7-03a3-4158-a386-229948f34e94-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2218f8f7-03a3-4158-a386-229948f34e94" (UID: "2218f8f7-03a3-4158-a386-229948f34e94"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 21:42:40.721872 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:40.721831 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2218f8f7-03a3-4158-a386-229948f34e94-istio-envoy\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:42:40.721872 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:40.721862 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2218f8f7-03a3-4158-a386-229948f34e94-istio-token\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:42:40.721872 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:40.721876 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q6bsg\" (UniqueName: \"kubernetes.io/projected/2218f8f7-03a3-4158-a386-229948f34e94-kube-api-access-q6bsg\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:42:40.722159 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:40.721889 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2218f8f7-03a3-4158-a386-229948f34e94-istio-podinfo\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:42:40.722159 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:40.721901 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2218f8f7-03a3-4158-a386-229948f34e94-istio-data\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:42:40.722159 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:40.721912 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2218f8f7-03a3-4158-a386-229948f34e94-workload-certs\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:42:40.722159 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:40.721922 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2218f8f7-03a3-4158-a386-229948f34e94-workload-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:42:40.912622 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:40.912587 2579 generic.go:356] "Generic (PLEG): container finished" podID="2218f8f7-03a3-4158-a386-229948f34e94" containerID="0e480a5bad36b854682220546b0e447861af7cd10aad7afcd6ceb5a4cf4c0938" exitCode=0 Jul 15 21:42:40.912824 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:40.912634 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l" Jul 15 21:42:40.912824 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:40.912668 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l" event={"ID":"2218f8f7-03a3-4158-a386-229948f34e94","Type":"ContainerDied","Data":"0e480a5bad36b854682220546b0e447861af7cd10aad7afcd6ceb5a4cf4c0938"} Jul 15 21:42:40.912824 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:40.912762 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l" event={"ID":"2218f8f7-03a3-4158-a386-229948f34e94","Type":"ContainerDied","Data":"68b51f2890b258ec66989d8b26016d1e9461bffc221587e85561538cc600562a"} Jul 15 21:42:40.912824 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:40.912793 2579 scope.go:117] "RemoveContainer" containerID="0e480a5bad36b854682220546b0e447861af7cd10aad7afcd6ceb5a4cf4c0938" Jul 15 21:42:40.922832 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:40.922813 2579 scope.go:117] "RemoveContainer" containerID="fd2135ff498e9b6016947b51729dc1ab380cef4767fe3243edf3c13c9c1c1fc7" Jul 15 21:42:40.934086 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:40.934043 2579 scope.go:117] "RemoveContainer" containerID="0e480a5bad36b854682220546b0e447861af7cd10aad7afcd6ceb5a4cf4c0938" Jul 15 21:42:40.934419 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:42:40.934382 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e480a5bad36b854682220546b0e447861af7cd10aad7afcd6ceb5a4cf4c0938\": container with ID starting with 0e480a5bad36b854682220546b0e447861af7cd10aad7afcd6ceb5a4cf4c0938 not found: ID does not exist" containerID="0e480a5bad36b854682220546b0e447861af7cd10aad7afcd6ceb5a4cf4c0938" Jul 15 21:42:40.934507 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:40.934421 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e480a5bad36b854682220546b0e447861af7cd10aad7afcd6ceb5a4cf4c0938"} err="failed to get container status \"0e480a5bad36b854682220546b0e447861af7cd10aad7afcd6ceb5a4cf4c0938\": rpc error: code = NotFound desc = could not find container \"0e480a5bad36b854682220546b0e447861af7cd10aad7afcd6ceb5a4cf4c0938\": container with ID starting with 0e480a5bad36b854682220546b0e447861af7cd10aad7afcd6ceb5a4cf4c0938 not found: ID does not exist" Jul 15 21:42:40.934507 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:40.934480 2579 scope.go:117] "RemoveContainer" containerID="fd2135ff498e9b6016947b51729dc1ab380cef4767fe3243edf3c13c9c1c1fc7" Jul 15 21:42:40.934836 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:42:40.934805 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd2135ff498e9b6016947b51729dc1ab380cef4767fe3243edf3c13c9c1c1fc7\": container with ID starting with fd2135ff498e9b6016947b51729dc1ab380cef4767fe3243edf3c13c9c1c1fc7 not found: ID does not exist" containerID="fd2135ff498e9b6016947b51729dc1ab380cef4767fe3243edf3c13c9c1c1fc7" Jul 15 21:42:40.934959 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:40.934839 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd2135ff498e9b6016947b51729dc1ab380cef4767fe3243edf3c13c9c1c1fc7"} err="failed to get container status \"fd2135ff498e9b6016947b51729dc1ab380cef4767fe3243edf3c13c9c1c1fc7\": rpc error: code = NotFound desc = could not find container \"fd2135ff498e9b6016947b51729dc1ab380cef4767fe3243edf3c13c9c1c1fc7\": container with ID starting with fd2135ff498e9b6016947b51729dc1ab380cef4767fe3243edf3c13c9c1c1fc7 not found: ID does not exist" Jul 15 21:42:40.937749 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:40.937728 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l"] Jul 15 21:42:40.942447 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:40.942423 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-c419c9-openshift-default-59f55b94cc-ghh6l"] Jul 15 21:42:41.997347 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:41.997305 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2218f8f7-03a3-4158-a386-229948f34e94" path="/var/lib/kubelet/pods/2218f8f7-03a3-4158-a386-229948f34e94/volumes" Jul 15 21:42:43.369226 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:43.369192 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:42:43.842009 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:43.841908 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:42:44.126863 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:44.126824 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h"] Jul 15 21:42:44.127297 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:44.127279 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2218f8f7-03a3-4158-a386-229948f34e94" containerName="istio-proxy" Jul 15 21:42:44.127361 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:44.127298 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2218f8f7-03a3-4158-a386-229948f34e94" containerName="istio-proxy" Jul 15 21:42:44.127361 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:44.127312 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2218f8f7-03a3-4158-a386-229948f34e94" containerName="istio-proxy" Jul 15 21:42:44.127361 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:44.127318 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2218f8f7-03a3-4158-a386-229948f34e94" containerName="istio-proxy" Jul 15 21:42:44.127524 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:44.127376 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2218f8f7-03a3-4158-a386-229948f34e94" containerName="istio-proxy" Jul 15 21:42:44.127524 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:44.127385 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a00277f-80e6-416a-89cc-7d197c2f6c4a" containerName="istio-proxy" Jul 15 21:42:44.127524 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:44.127392 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2218f8f7-03a3-4158-a386-229948f34e94" containerName="istio-proxy" Jul 15 21:42:44.129350 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:44.129331 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h" Jul 15 21:42:44.132371 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:44.132351 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-da009d-openshift-default-dockercfg-rdlwj\"" Jul 15 21:42:44.142494 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:44.142472 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h"] Jul 15 21:42:44.152512 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:44.152482 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:42:44.253741 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:44.253700 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7073af4-ef2d-482f-b1a4-647d266aba4a-workload-socket\") pod \"e2e-api-b-da009d-openshift-default-756f669c5c-9b27h\" (UID: \"f7073af4-ef2d-482f-b1a4-647d266aba4a\") " pod="openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h" Jul 15 21:42:44.253890 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:44.253752 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxqtw\" (UniqueName: \"kubernetes.io/projected/f7073af4-ef2d-482f-b1a4-647d266aba4a-kube-api-access-fxqtw\") pod \"e2e-api-b-da009d-openshift-default-756f669c5c-9b27h\" (UID: \"f7073af4-ef2d-482f-b1a4-647d266aba4a\") " pod="openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h" Jul 15 21:42:44.254023 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:44.254002 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7073af4-ef2d-482f-b1a4-647d266aba4a-istio-envoy\") pod \"e2e-api-b-da009d-openshift-default-756f669c5c-9b27h\" (UID: \"f7073af4-ef2d-482f-b1a4-647d266aba4a\") " pod="openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h" Jul 15 21:42:44.254129 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:44.254083 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7073af4-ef2d-482f-b1a4-647d266aba4a-credential-socket\") pod \"e2e-api-b-da009d-openshift-default-756f669c5c-9b27h\" (UID: \"f7073af4-ef2d-482f-b1a4-647d266aba4a\") " pod="openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h" Jul 15 21:42:44.254129 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:44.254122 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7073af4-ef2d-482f-b1a4-647d266aba4a-istio-token\") pod \"e2e-api-b-da009d-openshift-default-756f669c5c-9b27h\" (UID: \"f7073af4-ef2d-482f-b1a4-647d266aba4a\") " pod="openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h" Jul 15 21:42:44.254230 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:44.254208 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7073af4-ef2d-482f-b1a4-647d266aba4a-istio-data\") pod \"e2e-api-b-da009d-openshift-default-756f669c5c-9b27h\" (UID: \"f7073af4-ef2d-482f-b1a4-647d266aba4a\") " pod="openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h" Jul 15 21:42:44.254335 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:44.254316 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7073af4-ef2d-482f-b1a4-647d266aba4a-istio-podinfo\") pod \"e2e-api-b-da009d-openshift-default-756f669c5c-9b27h\" (UID: \"f7073af4-ef2d-482f-b1a4-647d266aba4a\") " pod="openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h" Jul 15 21:42:44.254393 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:44.254361 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7073af4-ef2d-482f-b1a4-647d266aba4a-istiod-ca-cert\") pod \"e2e-api-b-da009d-openshift-default-756f669c5c-9b27h\" (UID: \"f7073af4-ef2d-482f-b1a4-647d266aba4a\") " pod="openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h" Jul 15 21:42:44.254444 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:44.254390 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7073af4-ef2d-482f-b1a4-647d266aba4a-workload-certs\") pod \"e2e-api-b-da009d-openshift-default-756f669c5c-9b27h\" (UID: \"f7073af4-ef2d-482f-b1a4-647d266aba4a\") " pod="openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h" Jul 15 21:42:44.354897 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:44.354851 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7073af4-ef2d-482f-b1a4-647d266aba4a-workload-certs\") pod \"e2e-api-b-da009d-openshift-default-756f669c5c-9b27h\" (UID: \"f7073af4-ef2d-482f-b1a4-647d266aba4a\") " pod="openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h" Jul 15 21:42:44.355121 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:44.354911 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7073af4-ef2d-482f-b1a4-647d266aba4a-workload-socket\") pod \"e2e-api-b-da009d-openshift-default-756f669c5c-9b27h\" (UID: \"f7073af4-ef2d-482f-b1a4-647d266aba4a\") " pod="openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h" Jul 15 21:42:44.355121 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:44.354937 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxqtw\" (UniqueName: \"kubernetes.io/projected/f7073af4-ef2d-482f-b1a4-647d266aba4a-kube-api-access-fxqtw\") pod \"e2e-api-b-da009d-openshift-default-756f669c5c-9b27h\" (UID: \"f7073af4-ef2d-482f-b1a4-647d266aba4a\") " pod="openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h" Jul 15 21:42:44.355121 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:44.354963 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7073af4-ef2d-482f-b1a4-647d266aba4a-istio-envoy\") pod \"e2e-api-b-da009d-openshift-default-756f669c5c-9b27h\" (UID: \"f7073af4-ef2d-482f-b1a4-647d266aba4a\") " pod="openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h" Jul 15 21:42:44.355121 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:44.355002 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7073af4-ef2d-482f-b1a4-647d266aba4a-credential-socket\") pod \"e2e-api-b-da009d-openshift-default-756f669c5c-9b27h\" (UID: \"f7073af4-ef2d-482f-b1a4-647d266aba4a\") " pod="openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h" Jul 15 21:42:44.355121 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:44.355036 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7073af4-ef2d-482f-b1a4-647d266aba4a-istio-token\") pod \"e2e-api-b-da009d-openshift-default-756f669c5c-9b27h\" (UID: \"f7073af4-ef2d-482f-b1a4-647d266aba4a\") " pod="openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h" Jul 15 21:42:44.355121 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:44.355077 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7073af4-ef2d-482f-b1a4-647d266aba4a-istio-data\") pod \"e2e-api-b-da009d-openshift-default-756f669c5c-9b27h\" (UID: \"f7073af4-ef2d-482f-b1a4-647d266aba4a\") " pod="openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h" Jul 15 21:42:44.355454 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:44.355169 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7073af4-ef2d-482f-b1a4-647d266aba4a-istio-podinfo\") pod \"e2e-api-b-da009d-openshift-default-756f669c5c-9b27h\" (UID: \"f7073af4-ef2d-482f-b1a4-647d266aba4a\") " pod="openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h" Jul 15 21:42:44.355454 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:44.355211 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7073af4-ef2d-482f-b1a4-647d266aba4a-istiod-ca-cert\") pod \"e2e-api-b-da009d-openshift-default-756f669c5c-9b27h\" (UID: \"f7073af4-ef2d-482f-b1a4-647d266aba4a\") " pod="openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h" Jul 15 21:42:44.355454 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:44.355294 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7073af4-ef2d-482f-b1a4-647d266aba4a-workload-certs\") pod \"e2e-api-b-da009d-openshift-default-756f669c5c-9b27h\" (UID: \"f7073af4-ef2d-482f-b1a4-647d266aba4a\") " pod="openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h" Jul 15 21:42:44.355454 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:44.355340 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7073af4-ef2d-482f-b1a4-647d266aba4a-workload-socket\") pod \"e2e-api-b-da009d-openshift-default-756f669c5c-9b27h\" (UID: \"f7073af4-ef2d-482f-b1a4-647d266aba4a\") " pod="openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h" Jul 15 21:42:44.355668 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:44.355618 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7073af4-ef2d-482f-b1a4-647d266aba4a-credential-socket\") pod \"e2e-api-b-da009d-openshift-default-756f669c5c-9b27h\" (UID: \"f7073af4-ef2d-482f-b1a4-647d266aba4a\") " pod="openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h" Jul 15 21:42:44.355668 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:44.355660 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7073af4-ef2d-482f-b1a4-647d266aba4a-istio-data\") pod \"e2e-api-b-da009d-openshift-default-756f669c5c-9b27h\" (UID: \"f7073af4-ef2d-482f-b1a4-647d266aba4a\") " pod="openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h" Jul 15 21:42:44.355911 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:44.355891 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7073af4-ef2d-482f-b1a4-647d266aba4a-istiod-ca-cert\") pod \"e2e-api-b-da009d-openshift-default-756f669c5c-9b27h\" (UID: \"f7073af4-ef2d-482f-b1a4-647d266aba4a\") " pod="openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h" Jul 15 21:42:44.357459 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:44.357442 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7073af4-ef2d-482f-b1a4-647d266aba4a-istio-envoy\") pod \"e2e-api-b-da009d-openshift-default-756f669c5c-9b27h\" (UID: \"f7073af4-ef2d-482f-b1a4-647d266aba4a\") " pod="openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h" Jul 15 21:42:44.357669 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:44.357649 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7073af4-ef2d-482f-b1a4-647d266aba4a-istio-podinfo\") pod \"e2e-api-b-da009d-openshift-default-756f669c5c-9b27h\" (UID: \"f7073af4-ef2d-482f-b1a4-647d266aba4a\") " pod="openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h" Jul 15 21:42:44.363764 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:44.363738 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxqtw\" (UniqueName: \"kubernetes.io/projected/f7073af4-ef2d-482f-b1a4-647d266aba4a-kube-api-access-fxqtw\") pod \"e2e-api-b-da009d-openshift-default-756f669c5c-9b27h\" (UID: \"f7073af4-ef2d-482f-b1a4-647d266aba4a\") " pod="openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h" Jul 15 21:42:44.364444 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:44.364421 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7073af4-ef2d-482f-b1a4-647d266aba4a-istio-token\") pod \"e2e-api-b-da009d-openshift-default-756f669c5c-9b27h\" (UID: \"f7073af4-ef2d-482f-b1a4-647d266aba4a\") " pod="openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h" Jul 15 21:42:44.440579 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:44.440483 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h" Jul 15 21:42:44.475293 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:44.475257 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:42:44.582001 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:44.581973 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h"] Jul 15 21:42:44.583777 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:42:44.583751 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7073af4_ef2d_482f_b1a4_647d266aba4a.slice/crio-7688a0e2afc7facf56c9f86f76511f3b20612fa16ed4f039d1b9859e2f35be75 WatchSource:0}: Error finding container 7688a0e2afc7facf56c9f86f76511f3b20612fa16ed4f039d1b9859e2f35be75: Status 404 returned error can't find the container with id 7688a0e2afc7facf56c9f86f76511f3b20612fa16ed4f039d1b9859e2f35be75 Jul 15 21:42:44.586261 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:44.586233 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:42:44.586329 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:44.586300 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:42:44.586369 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:44.586330 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:42:44.937884 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:44.937834 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h" event={"ID":"f7073af4-ef2d-482f-b1a4-647d266aba4a","Type":"ContainerStarted","Data":"c0a150a9e3e693dd4248cf420903df05a545ec7b986c3914912c6348099cf13f"} Jul 15 21:42:44.938099 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:44.937945 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h" event={"ID":"f7073af4-ef2d-482f-b1a4-647d266aba4a","Type":"ContainerStarted","Data":"7688a0e2afc7facf56c9f86f76511f3b20612fa16ed4f039d1b9859e2f35be75"} Jul 15 21:42:44.962159 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:44.962096 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h" podStartSLOduration=0.96205051 podStartE2EDuration="962.05051ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:42:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:42:44.961673259 +0000 UTC m=+2469.502985841" watchObservedRunningTime="2026-07-15 21:42:44.96205051 +0000 UTC m=+2469.503363061" Jul 15 21:42:45.440837 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:45.440787 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h" Jul 15 21:42:45.445918 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:45.445891 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h" Jul 15 21:42:45.943327 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:45.943297 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h" Jul 15 21:42:45.944234 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:45.944216 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h" Jul 15 21:42:46.185823 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:46.185778 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:42:46.938826 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:46.938758 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:42:47.042635 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:47.042603 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:42:47.216484 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:47.216399 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:42:47.519056 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:47.518953 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:42:47.742684 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:47.742649 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:42:47.842879 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:47.842786 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:42:51.446922 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:51.446884 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:42:53.935247 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:53.935196 2579 patch_prober.go:28] interesting pod/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 15 21:42:53.935775 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:53.935273 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl" podUID="e852dd0a-92d3-45de-8922-5e801112b481" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 15 21:42:55.002700 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:55.002663 2579 generic.go:356] "Generic (PLEG): container finished" podID="e852dd0a-92d3-45de-8922-5e801112b481" containerID="448dc604e87decf5e0cf3d194bdc3f5b7ac7ace4d2636b066e6fb2b444461ae9" exitCode=0 Jul 15 21:42:55.003328 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:55.002725 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl" event={"ID":"e852dd0a-92d3-45de-8922-5e801112b481","Type":"ContainerDied","Data":"448dc604e87decf5e0cf3d194bdc3f5b7ac7ace4d2636b066e6fb2b444461ae9"} Jul 15 21:42:55.003328 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:55.003123 2579 scope.go:117] "RemoveContainer" containerID="448dc604e87decf5e0cf3d194bdc3f5b7ac7ace4d2636b066e6fb2b444461ae9" Jul 15 21:42:55.004141 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:55.004110 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:42:55.004233 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:55.004179 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:42:55.004233 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:55.004210 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:42:56.009687 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:56.009649 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl" event={"ID":"e852dd0a-92d3-45de-8922-5e801112b481","Type":"ContainerStarted","Data":"8a312cd87646dd0000bbe01737bc9fb31c7a5a2ebf1f1d38f2f23a9902a31987"} Jul 15 21:42:56.619845 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:56.619801 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl" Jul 15 21:42:56.620030 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:56.619977 2579 patch_prober.go:28] interesting pod/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 15 21:42:56.620111 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:56.620047 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl" podUID="e852dd0a-92d3-45de-8922-5e801112b481" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 15 21:42:57.620081 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:57.620024 2579 patch_prober.go:28] interesting pod/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 15 21:42:57.620491 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:57.620120 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl" podUID="e852dd0a-92d3-45de-8922-5e801112b481" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 15 21:42:58.620304 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:58.620267 2579 patch_prober.go:28] interesting pod/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 15 21:42:58.620709 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:58.620353 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl" podUID="e852dd0a-92d3-45de-8922-5e801112b481" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 15 21:42:59.619705 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:59.619666 2579 patch_prober.go:28] interesting pod/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 15 21:42:59.619900 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:42:59.619720 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl" podUID="e852dd0a-92d3-45de-8922-5e801112b481" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 15 21:43:00.180052 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:00.179999 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:43:00.278724 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:00.278685 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:43:00.623507 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:00.623480 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl" Jul 15 21:43:00.623904 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:00.623882 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl" Jul 15 21:43:00.624696 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:00.624677 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl" Jul 15 21:43:01.986250 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:01.986194 2579 patch_prober.go:28] interesting pod/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 15 21:43:01.986730 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:01.986303 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h" podUID="f7073af4-ef2d-482f-b1a4-647d266aba4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 15 21:43:04.065625 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:04.065594 2579 generic.go:356] "Generic (PLEG): container finished" podID="f7073af4-ef2d-482f-b1a4-647d266aba4a" containerID="c0a150a9e3e693dd4248cf420903df05a545ec7b986c3914912c6348099cf13f" exitCode=0 Jul 15 21:43:04.066096 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:04.065661 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h" event={"ID":"f7073af4-ef2d-482f-b1a4-647d266aba4a","Type":"ContainerDied","Data":"c0a150a9e3e693dd4248cf420903df05a545ec7b986c3914912c6348099cf13f"} Jul 15 21:43:04.066096 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:04.066046 2579 scope.go:117] "RemoveContainer" containerID="c0a150a9e3e693dd4248cf420903df05a545ec7b986c3914912c6348099cf13f" Jul 15 21:43:04.067105 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:04.067057 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:43:04.067182 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:04.067148 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:43:04.067221 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:04.067191 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:43:04.599369 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:04.599338 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:43:05.072616 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:05.072577 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h" event={"ID":"f7073af4-ef2d-482f-b1a4-647d266aba4a","Type":"ContainerStarted","Data":"3769d8d418a076a3a2d7317b40ecf6c738755ba544301c279ec11ae290e550ab"} Jul 15 21:43:05.441877 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:05.441779 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h" Jul 15 21:43:05.442230 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:05.442193 2579 patch_prober.go:28] interesting pod/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 15 21:43:05.442436 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:05.442407 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h" podUID="f7073af4-ef2d-482f-b1a4-647d266aba4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 15 21:43:06.441239 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:06.441200 2579 patch_prober.go:28] interesting pod/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 15 21:43:06.442264 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:06.441820 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h" podUID="f7073af4-ef2d-482f-b1a4-647d266aba4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 15 21:43:07.441152 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:07.441106 2579 patch_prober.go:28] interesting pod/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 15 21:43:07.441367 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:07.441175 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h" podUID="f7073af4-ef2d-482f-b1a4-647d266aba4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 15 21:43:08.442040 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:08.442001 2579 patch_prober.go:28] interesting pod/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 15 21:43:08.443152 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:08.442663 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h" podUID="f7073af4-ef2d-482f-b1a4-647d266aba4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 15 21:43:08.559114 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:08.559084 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:43:08.659646 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:08.659607 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:43:09.444946 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:09.444917 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h" Jul 15 21:43:09.445359 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:09.445262 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h" Jul 15 21:43:09.446028 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:09.446010 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h" Jul 15 21:43:10.219921 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:10.219877 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:43:10.320038 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:10.320003 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:43:10.505808 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:10.505719 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:43:14.597223 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:14.597187 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:43:16.187334 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:16.187298 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:43:20.535427 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:20.535393 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:43:20.599636 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:20.599603 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:43:25.323211 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:25.323174 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl"] Jul 15 21:43:25.323677 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:25.323428 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl" podUID="e852dd0a-92d3-45de-8922-5e801112b481" containerName="istio-proxy" containerID="cri-o://8a312cd87646dd0000bbe01737bc9fb31c7a5a2ebf1f1d38f2f23a9902a31987" gracePeriod=30 Jul 15 21:43:25.350461 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:25.350423 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:43:25.567405 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:25.567373 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:43:26.535274 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:26.535241 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:43:26.636047 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:26.636010 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:43:30.582704 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:30.582674 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl" Jul 15 21:43:30.681485 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:30.681445 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e852dd0a-92d3-45de-8922-5e801112b481-credential-socket\") pod \"e852dd0a-92d3-45de-8922-5e801112b481\" (UID: \"e852dd0a-92d3-45de-8922-5e801112b481\") " Jul 15 21:43:30.681485 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:30.681489 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e852dd0a-92d3-45de-8922-5e801112b481-istiod-ca-cert\") pod \"e852dd0a-92d3-45de-8922-5e801112b481\" (UID: \"e852dd0a-92d3-45de-8922-5e801112b481\") " Jul 15 21:43:30.681749 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:30.681506 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e852dd0a-92d3-45de-8922-5e801112b481-workload-socket\") pod \"e852dd0a-92d3-45de-8922-5e801112b481\" (UID: \"e852dd0a-92d3-45de-8922-5e801112b481\") " Jul 15 21:43:30.681749 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:30.681536 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e852dd0a-92d3-45de-8922-5e801112b481-istio-token\") pod \"e852dd0a-92d3-45de-8922-5e801112b481\" (UID: \"e852dd0a-92d3-45de-8922-5e801112b481\") " Jul 15 21:43:30.681749 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:30.681576 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e852dd0a-92d3-45de-8922-5e801112b481-istio-podinfo\") pod \"e852dd0a-92d3-45de-8922-5e801112b481\" (UID: \"e852dd0a-92d3-45de-8922-5e801112b481\") " Jul 15 21:43:30.681749 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:30.681615 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e852dd0a-92d3-45de-8922-5e801112b481-istio-data\") pod \"e852dd0a-92d3-45de-8922-5e801112b481\" (UID: \"e852dd0a-92d3-45de-8922-5e801112b481\") " Jul 15 21:43:30.681749 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:30.681638 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e852dd0a-92d3-45de-8922-5e801112b481-workload-certs\") pod \"e852dd0a-92d3-45de-8922-5e801112b481\" (UID: \"e852dd0a-92d3-45de-8922-5e801112b481\") " Jul 15 21:43:30.681749 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:30.681713 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q9dxw\" (UniqueName: \"kubernetes.io/projected/e852dd0a-92d3-45de-8922-5e801112b481-kube-api-access-q9dxw\") pod \"e852dd0a-92d3-45de-8922-5e801112b481\" (UID: \"e852dd0a-92d3-45de-8922-5e801112b481\") " Jul 15 21:43:30.682101 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:30.681757 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e852dd0a-92d3-45de-8922-5e801112b481-istio-envoy\") pod \"e852dd0a-92d3-45de-8922-5e801112b481\" (UID: \"e852dd0a-92d3-45de-8922-5e801112b481\") " Jul 15 21:43:30.682101 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:30.681787 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e852dd0a-92d3-45de-8922-5e801112b481-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e852dd0a-92d3-45de-8922-5e801112b481" (UID: "e852dd0a-92d3-45de-8922-5e801112b481"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:43:30.682101 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:30.681854 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e852dd0a-92d3-45de-8922-5e801112b481-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e852dd0a-92d3-45de-8922-5e801112b481" (UID: "e852dd0a-92d3-45de-8922-5e801112b481"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:43:30.682101 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:30.681992 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e852dd0a-92d3-45de-8922-5e801112b481-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e852dd0a-92d3-45de-8922-5e801112b481" (UID: "e852dd0a-92d3-45de-8922-5e801112b481"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:43:30.682101 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:30.682018 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e852dd0a-92d3-45de-8922-5e801112b481-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e852dd0a-92d3-45de-8922-5e801112b481" (UID: "e852dd0a-92d3-45de-8922-5e801112b481"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 21:43:30.682347 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:30.682138 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e852dd0a-92d3-45de-8922-5e801112b481-workload-certs\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:43:30.682347 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:30.682249 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e852dd0a-92d3-45de-8922-5e801112b481-credential-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:43:30.682347 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:30.682263 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e852dd0a-92d3-45de-8922-5e801112b481-istiod-ca-cert\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:43:30.682347 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:30.682275 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e852dd0a-92d3-45de-8922-5e801112b481-workload-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:43:30.682759 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:30.682741 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e852dd0a-92d3-45de-8922-5e801112b481-istio-data" (OuterVolumeSpecName: "istio-data") pod "e852dd0a-92d3-45de-8922-5e801112b481" (UID: "e852dd0a-92d3-45de-8922-5e801112b481"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:43:30.684510 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:30.684485 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e852dd0a-92d3-45de-8922-5e801112b481-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e852dd0a-92d3-45de-8922-5e801112b481" (UID: "e852dd0a-92d3-45de-8922-5e801112b481"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:43:30.684510 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:30.684495 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e852dd0a-92d3-45de-8922-5e801112b481-istio-token" (OuterVolumeSpecName: "istio-token") pod "e852dd0a-92d3-45de-8922-5e801112b481" (UID: "e852dd0a-92d3-45de-8922-5e801112b481"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:43:30.684652 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:30.684546 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e852dd0a-92d3-45de-8922-5e801112b481-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e852dd0a-92d3-45de-8922-5e801112b481" (UID: "e852dd0a-92d3-45de-8922-5e801112b481"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 21:43:30.684652 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:30.684552 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e852dd0a-92d3-45de-8922-5e801112b481-kube-api-access-q9dxw" (OuterVolumeSpecName: "kube-api-access-q9dxw") pod "e852dd0a-92d3-45de-8922-5e801112b481" (UID: "e852dd0a-92d3-45de-8922-5e801112b481"). InnerVolumeSpecName "kube-api-access-q9dxw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:43:30.782813 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:30.782765 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e852dd0a-92d3-45de-8922-5e801112b481-istio-podinfo\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:43:30.782813 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:30.782809 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e852dd0a-92d3-45de-8922-5e801112b481-istio-data\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:43:30.782813 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:30.782819 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q9dxw\" (UniqueName: \"kubernetes.io/projected/e852dd0a-92d3-45de-8922-5e801112b481-kube-api-access-q9dxw\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:43:30.782813 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:30.782829 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e852dd0a-92d3-45de-8922-5e801112b481-istio-envoy\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:43:30.783120 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:30.782837 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e852dd0a-92d3-45de-8922-5e801112b481-istio-token\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:43:31.217562 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:31.217528 2579 generic.go:356] "Generic (PLEG): container finished" podID="e852dd0a-92d3-45de-8922-5e801112b481" containerID="8a312cd87646dd0000bbe01737bc9fb31c7a5a2ebf1f1d38f2f23a9902a31987" exitCode=0 Jul 15 21:43:31.217773 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:31.217574 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl" event={"ID":"e852dd0a-92d3-45de-8922-5e801112b481","Type":"ContainerDied","Data":"8a312cd87646dd0000bbe01737bc9fb31c7a5a2ebf1f1d38f2f23a9902a31987"} Jul 15 21:43:31.217773 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:31.217577 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl" Jul 15 21:43:31.217773 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:31.217660 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl" event={"ID":"e852dd0a-92d3-45de-8922-5e801112b481","Type":"ContainerDied","Data":"71002235fe5f6dc99cfb1d7829ef54786114d0be455c158d592a7571bf031923"} Jul 15 21:43:31.217773 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:31.217689 2579 scope.go:117] "RemoveContainer" containerID="8a312cd87646dd0000bbe01737bc9fb31c7a5a2ebf1f1d38f2f23a9902a31987" Jul 15 21:43:31.228217 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:31.228195 2579 scope.go:117] "RemoveContainer" containerID="448dc604e87decf5e0cf3d194bdc3f5b7ac7ace4d2636b066e6fb2b444461ae9" Jul 15 21:43:31.236480 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:31.236457 2579 scope.go:117] "RemoveContainer" containerID="8a312cd87646dd0000bbe01737bc9fb31c7a5a2ebf1f1d38f2f23a9902a31987" Jul 15 21:43:31.236721 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:43:31.236704 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a312cd87646dd0000bbe01737bc9fb31c7a5a2ebf1f1d38f2f23a9902a31987\": container with ID starting with 8a312cd87646dd0000bbe01737bc9fb31c7a5a2ebf1f1d38f2f23a9902a31987 not found: ID does not exist" containerID="8a312cd87646dd0000bbe01737bc9fb31c7a5a2ebf1f1d38f2f23a9902a31987" Jul 15 21:43:31.236801 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:31.236736 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a312cd87646dd0000bbe01737bc9fb31c7a5a2ebf1f1d38f2f23a9902a31987"} err="failed to get container status \"8a312cd87646dd0000bbe01737bc9fb31c7a5a2ebf1f1d38f2f23a9902a31987\": rpc error: code = NotFound desc = could not find container \"8a312cd87646dd0000bbe01737bc9fb31c7a5a2ebf1f1d38f2f23a9902a31987\": container with ID starting with 8a312cd87646dd0000bbe01737bc9fb31c7a5a2ebf1f1d38f2f23a9902a31987 not found: ID does not exist" Jul 15 21:43:31.236801 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:31.236761 2579 scope.go:117] "RemoveContainer" containerID="448dc604e87decf5e0cf3d194bdc3f5b7ac7ace4d2636b066e6fb2b444461ae9" Jul 15 21:43:31.237009 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:43:31.236992 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"448dc604e87decf5e0cf3d194bdc3f5b7ac7ace4d2636b066e6fb2b444461ae9\": container with ID starting with 448dc604e87decf5e0cf3d194bdc3f5b7ac7ace4d2636b066e6fb2b444461ae9 not found: ID does not exist" containerID="448dc604e87decf5e0cf3d194bdc3f5b7ac7ace4d2636b066e6fb2b444461ae9" Jul 15 21:43:31.237049 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:31.237016 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"448dc604e87decf5e0cf3d194bdc3f5b7ac7ace4d2636b066e6fb2b444461ae9"} err="failed to get container status \"448dc604e87decf5e0cf3d194bdc3f5b7ac7ace4d2636b066e6fb2b444461ae9\": rpc error: code = NotFound desc = could not find container \"448dc604e87decf5e0cf3d194bdc3f5b7ac7ace4d2636b066e6fb2b444461ae9\": container with ID starting with 448dc604e87decf5e0cf3d194bdc3f5b7ac7ace4d2636b066e6fb2b444461ae9 not found: ID does not exist" Jul 15 21:43:31.266334 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:31.266298 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl"] Jul 15 21:43:31.275940 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:31.275914 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-652862-openshift-default-c9d95f897-mzsgl"] Jul 15 21:43:31.997073 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:31.997041 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e852dd0a-92d3-45de-8922-5e801112b481" path="/var/lib/kubelet/pods/e852dd0a-92d3-45de-8922-5e801112b481/volumes" Jul 15 21:43:36.202657 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:36.202621 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:43:39.610289 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:39.610242 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:43:40.790637 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:40.790597 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:43:45.465349 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:45.465310 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:43:45.535368 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:45.535331 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:43:46.947848 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:46.947812 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:43:47.050840 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:47.050811 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:43:51.707641 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:51.707605 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h"] Jul 15 21:43:51.708108 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:51.707891 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h" podUID="f7073af4-ef2d-482f-b1a4-647d266aba4a" containerName="istio-proxy" containerID="cri-o://3769d8d418a076a3a2d7317b40ecf6c738755ba544301c279ec11ae290e550ab" gracePeriod=30 Jul 15 21:43:51.742784 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:51.742733 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:43:52.008098 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:52.007992 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:43:52.393524 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:52.393489 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:43:52.702191 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:52.702110 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g"] Jul 15 21:43:52.702575 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:52.702550 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e852dd0a-92d3-45de-8922-5e801112b481" containerName="istio-proxy" Jul 15 21:43:52.702575 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:52.702573 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e852dd0a-92d3-45de-8922-5e801112b481" containerName="istio-proxy" Jul 15 21:43:52.702766 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:52.702585 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e852dd0a-92d3-45de-8922-5e801112b481" containerName="istio-proxy" Jul 15 21:43:52.702766 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:52.702594 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e852dd0a-92d3-45de-8922-5e801112b481" containerName="istio-proxy" Jul 15 21:43:52.702766 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:52.702682 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e852dd0a-92d3-45de-8922-5e801112b481" containerName="istio-proxy" Jul 15 21:43:52.702766 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:52.702692 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e852dd0a-92d3-45de-8922-5e801112b481" containerName="istio-proxy" Jul 15 21:43:52.704595 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:52.704579 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g" Jul 15 21:43:52.708522 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:52.708495 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-31dfa2-openshift-default-dockercfg-nq2vs\"" Jul 15 21:43:52.722028 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:52.722002 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g"] Jul 15 21:43:52.737634 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:52.737598 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:43:52.773855 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:52.773812 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6d86286-9979-41b8-89cd-dea1057b2dff-istio-token\") pod \"e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g\" (UID: \"f6d86286-9979-41b8-89cd-dea1057b2dff\") " pod="openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g" Jul 15 21:43:52.773855 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:52.773856 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6d86286-9979-41b8-89cd-dea1057b2dff-workload-socket\") pod \"e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g\" (UID: \"f6d86286-9979-41b8-89cd-dea1057b2dff\") " pod="openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g" Jul 15 21:43:52.774149 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:52.773942 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6d86286-9979-41b8-89cd-dea1057b2dff-workload-certs\") pod \"e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g\" (UID: \"f6d86286-9979-41b8-89cd-dea1057b2dff\") " pod="openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g" Jul 15 21:43:52.774149 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:52.773964 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qxm6\" (UniqueName: \"kubernetes.io/projected/f6d86286-9979-41b8-89cd-dea1057b2dff-kube-api-access-8qxm6\") pod \"e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g\" (UID: \"f6d86286-9979-41b8-89cd-dea1057b2dff\") " pod="openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g" Jul 15 21:43:52.774149 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:52.773993 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6d86286-9979-41b8-89cd-dea1057b2dff-istiod-ca-cert\") pod \"e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g\" (UID: \"f6d86286-9979-41b8-89cd-dea1057b2dff\") " pod="openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g" Jul 15 21:43:52.774149 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:52.774031 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6d86286-9979-41b8-89cd-dea1057b2dff-credential-socket\") pod \"e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g\" (UID: \"f6d86286-9979-41b8-89cd-dea1057b2dff\") " pod="openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g" Jul 15 21:43:52.774149 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:52.774107 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6d86286-9979-41b8-89cd-dea1057b2dff-istio-podinfo\") pod \"e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g\" (UID: \"f6d86286-9979-41b8-89cd-dea1057b2dff\") " pod="openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g" Jul 15 21:43:52.774352 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:52.774148 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6d86286-9979-41b8-89cd-dea1057b2dff-istio-data\") pod \"e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g\" (UID: \"f6d86286-9979-41b8-89cd-dea1057b2dff\") " pod="openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g" Jul 15 21:43:52.774352 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:52.774179 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6d86286-9979-41b8-89cd-dea1057b2dff-istio-envoy\") pod \"e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g\" (UID: \"f6d86286-9979-41b8-89cd-dea1057b2dff\") " pod="openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g" Jul 15 21:43:52.874945 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:52.874912 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6d86286-9979-41b8-89cd-dea1057b2dff-istio-podinfo\") pod \"e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g\" (UID: \"f6d86286-9979-41b8-89cd-dea1057b2dff\") " pod="openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g" Jul 15 21:43:52.874945 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:52.874955 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6d86286-9979-41b8-89cd-dea1057b2dff-istio-data\") pod \"e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g\" (UID: \"f6d86286-9979-41b8-89cd-dea1057b2dff\") " pod="openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g" Jul 15 21:43:52.875257 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:52.874983 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6d86286-9979-41b8-89cd-dea1057b2dff-istio-envoy\") pod \"e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g\" (UID: \"f6d86286-9979-41b8-89cd-dea1057b2dff\") " pod="openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g" Jul 15 21:43:52.875257 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:52.875027 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6d86286-9979-41b8-89cd-dea1057b2dff-istio-token\") pod \"e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g\" (UID: \"f6d86286-9979-41b8-89cd-dea1057b2dff\") " pod="openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g" Jul 15 21:43:52.875257 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:52.875053 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6d86286-9979-41b8-89cd-dea1057b2dff-workload-socket\") pod \"e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g\" (UID: \"f6d86286-9979-41b8-89cd-dea1057b2dff\") " pod="openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g" Jul 15 21:43:52.875257 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:52.875151 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6d86286-9979-41b8-89cd-dea1057b2dff-workload-certs\") pod \"e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g\" (UID: \"f6d86286-9979-41b8-89cd-dea1057b2dff\") " pod="openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g" Jul 15 21:43:52.875257 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:52.875176 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qxm6\" (UniqueName: \"kubernetes.io/projected/f6d86286-9979-41b8-89cd-dea1057b2dff-kube-api-access-8qxm6\") pod \"e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g\" (UID: \"f6d86286-9979-41b8-89cd-dea1057b2dff\") " pod="openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g" Jul 15 21:43:52.875257 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:52.875206 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6d86286-9979-41b8-89cd-dea1057b2dff-istiod-ca-cert\") pod \"e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g\" (UID: \"f6d86286-9979-41b8-89cd-dea1057b2dff\") " pod="openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g" Jul 15 21:43:52.875257 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:52.875254 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6d86286-9979-41b8-89cd-dea1057b2dff-credential-socket\") pod \"e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g\" (UID: \"f6d86286-9979-41b8-89cd-dea1057b2dff\") " pod="openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g" Jul 15 21:43:52.875653 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:52.875399 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6d86286-9979-41b8-89cd-dea1057b2dff-istio-data\") pod \"e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g\" (UID: \"f6d86286-9979-41b8-89cd-dea1057b2dff\") " pod="openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g" Jul 15 21:43:52.875653 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:52.875499 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6d86286-9979-41b8-89cd-dea1057b2dff-workload-socket\") pod \"e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g\" (UID: \"f6d86286-9979-41b8-89cd-dea1057b2dff\") " pod="openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g" Jul 15 21:43:52.875653 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:52.875564 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6d86286-9979-41b8-89cd-dea1057b2dff-credential-socket\") pod \"e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g\" (UID: \"f6d86286-9979-41b8-89cd-dea1057b2dff\") " pod="openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g" Jul 15 21:43:52.875816 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:52.875655 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6d86286-9979-41b8-89cd-dea1057b2dff-workload-certs\") pod \"e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g\" (UID: \"f6d86286-9979-41b8-89cd-dea1057b2dff\") " pod="openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g" Jul 15 21:43:52.876003 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:52.875981 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6d86286-9979-41b8-89cd-dea1057b2dff-istiod-ca-cert\") pod \"e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g\" (UID: \"f6d86286-9979-41b8-89cd-dea1057b2dff\") " pod="openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g" Jul 15 21:43:52.877371 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:52.877339 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6d86286-9979-41b8-89cd-dea1057b2dff-istio-envoy\") pod \"e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g\" (UID: \"f6d86286-9979-41b8-89cd-dea1057b2dff\") " pod="openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g" Jul 15 21:43:52.877810 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:52.877788 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6d86286-9979-41b8-89cd-dea1057b2dff-istio-podinfo\") pod \"e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g\" (UID: \"f6d86286-9979-41b8-89cd-dea1057b2dff\") " pod="openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g" Jul 15 21:43:52.890555 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:52.890527 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qxm6\" (UniqueName: \"kubernetes.io/projected/f6d86286-9979-41b8-89cd-dea1057b2dff-kube-api-access-8qxm6\") pod \"e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g\" (UID: \"f6d86286-9979-41b8-89cd-dea1057b2dff\") " pod="openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g" Jul 15 21:43:52.891729 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:52.891707 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6d86286-9979-41b8-89cd-dea1057b2dff-istio-token\") pod \"e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g\" (UID: \"f6d86286-9979-41b8-89cd-dea1057b2dff\") " pod="openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g" Jul 15 21:43:53.016357 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:53.016266 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g" Jul 15 21:43:53.163620 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:53.163594 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g"] Jul 15 21:43:53.165438 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:43:53.165411 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6d86286_9979_41b8_89cd_dea1057b2dff.slice/crio-f7fc1255e07f106c1a6b326015696a861dd0a6c2dc90aba53404c803ee1fd062 WatchSource:0}: Error finding container f7fc1255e07f106c1a6b326015696a861dd0a6c2dc90aba53404c803ee1fd062: Status 404 returned error can't find the container with id f7fc1255e07f106c1a6b326015696a861dd0a6c2dc90aba53404c803ee1fd062 Jul 15 21:43:53.167859 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:53.167828 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:43:53.167919 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:53.167896 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:43:53.167960 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:53.167924 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:43:53.356347 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:53.356312 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g" event={"ID":"f6d86286-9979-41b8-89cd-dea1057b2dff","Type":"ContainerStarted","Data":"7a707e09a8ffa1f0e6a33a90611635ba2342af5c4b8468541eba0e474581dfb5"} Jul 15 21:43:53.356546 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:53.356375 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g" event={"ID":"f6d86286-9979-41b8-89cd-dea1057b2dff","Type":"ContainerStarted","Data":"f7fc1255e07f106c1a6b326015696a861dd0a6c2dc90aba53404c803ee1fd062"} Jul 15 21:43:53.378971 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:53.378911 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g" podStartSLOduration=1.378894168 podStartE2EDuration="1.378894168s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:43:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:43:53.37795549 +0000 UTC m=+2537.919268042" watchObservedRunningTime="2026-07-15 21:43:53.378894168 +0000 UTC m=+2537.920206718" Jul 15 21:43:53.689187 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:53.689084 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:43:54.017264 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:54.017178 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g" Jul 15 21:43:54.022200 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:54.022174 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g" Jul 15 21:43:54.362463 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:54.362438 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g" Jul 15 21:43:54.363541 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:54.363519 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g" Jul 15 21:43:54.447845 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:54.447810 2579 patch_prober.go:28] interesting pod/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 15 21:43:54.447998 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:54.447867 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h" podUID="f7073af4-ef2d-482f-b1a4-647d266aba4a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 21:43:55.421407 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:55.421375 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:43:55.503617 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:55.503575 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:43:55.573897 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:55.573855 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:43:55.824495 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:55.824406 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:43:56.112282 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:56.112245 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:43:56.214983 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:56.214948 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:43:56.956459 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:56.956427 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h" Jul 15 21:43:57.011722 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:57.011636 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fxqtw\" (UniqueName: \"kubernetes.io/projected/f7073af4-ef2d-482f-b1a4-647d266aba4a-kube-api-access-fxqtw\") pod \"f7073af4-ef2d-482f-b1a4-647d266aba4a\" (UID: \"f7073af4-ef2d-482f-b1a4-647d266aba4a\") " Jul 15 21:43:57.011722 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:57.011684 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7073af4-ef2d-482f-b1a4-647d266aba4a-istio-token\") pod \"f7073af4-ef2d-482f-b1a4-647d266aba4a\" (UID: \"f7073af4-ef2d-482f-b1a4-647d266aba4a\") " Jul 15 21:43:57.011722 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:57.011713 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7073af4-ef2d-482f-b1a4-647d266aba4a-istio-podinfo\") pod \"f7073af4-ef2d-482f-b1a4-647d266aba4a\" (UID: \"f7073af4-ef2d-482f-b1a4-647d266aba4a\") " Jul 15 21:43:57.011996 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:57.011773 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7073af4-ef2d-482f-b1a4-647d266aba4a-istiod-ca-cert\") pod \"f7073af4-ef2d-482f-b1a4-647d266aba4a\" (UID: \"f7073af4-ef2d-482f-b1a4-647d266aba4a\") " Jul 15 21:43:57.012199 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:57.012173 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7073af4-ef2d-482f-b1a4-647d266aba4a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f7073af4-ef2d-482f-b1a4-647d266aba4a" (UID: "f7073af4-ef2d-482f-b1a4-647d266aba4a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 21:43:57.012357 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:57.012225 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7073af4-ef2d-482f-b1a4-647d266aba4a-credential-socket\") pod \"f7073af4-ef2d-482f-b1a4-647d266aba4a\" (UID: \"f7073af4-ef2d-482f-b1a4-647d266aba4a\") " Jul 15 21:43:57.012357 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:57.012271 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7073af4-ef2d-482f-b1a4-647d266aba4a-workload-socket\") pod \"f7073af4-ef2d-482f-b1a4-647d266aba4a\" (UID: \"f7073af4-ef2d-482f-b1a4-647d266aba4a\") " Jul 15 21:43:57.012357 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:57.012315 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7073af4-ef2d-482f-b1a4-647d266aba4a-istio-envoy\") pod \"f7073af4-ef2d-482f-b1a4-647d266aba4a\" (UID: \"f7073af4-ef2d-482f-b1a4-647d266aba4a\") " Jul 15 21:43:57.012357 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:57.012340 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7073af4-ef2d-482f-b1a4-647d266aba4a-workload-certs\") pod \"f7073af4-ef2d-482f-b1a4-647d266aba4a\" (UID: \"f7073af4-ef2d-482f-b1a4-647d266aba4a\") " Jul 15 21:43:57.012589 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:57.012373 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7073af4-ef2d-482f-b1a4-647d266aba4a-istio-data\") pod \"f7073af4-ef2d-482f-b1a4-647d266aba4a\" (UID: \"f7073af4-ef2d-482f-b1a4-647d266aba4a\") " Jul 15 21:43:57.012874 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:57.012450 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7073af4-ef2d-482f-b1a4-647d266aba4a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f7073af4-ef2d-482f-b1a4-647d266aba4a" (UID: "f7073af4-ef2d-482f-b1a4-647d266aba4a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:43:57.012874 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:57.012532 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7073af4-ef2d-482f-b1a4-647d266aba4a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f7073af4-ef2d-482f-b1a4-647d266aba4a" (UID: "f7073af4-ef2d-482f-b1a4-647d266aba4a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:43:57.012874 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:57.012761 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7073af4-ef2d-482f-b1a4-647d266aba4a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f7073af4-ef2d-482f-b1a4-647d266aba4a" (UID: "f7073af4-ef2d-482f-b1a4-647d266aba4a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:43:57.013183 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:57.013098 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7073af4-ef2d-482f-b1a4-647d266aba4a-workload-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:43:57.013183 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:57.013121 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7073af4-ef2d-482f-b1a4-647d266aba4a-workload-certs\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:43:57.013183 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:57.013136 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7073af4-ef2d-482f-b1a4-647d266aba4a-istiod-ca-cert\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:43:57.013183 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:57.013150 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7073af4-ef2d-482f-b1a4-647d266aba4a-credential-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:43:57.014259 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:57.014235 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7073af4-ef2d-482f-b1a4-647d266aba4a-istio-data" (OuterVolumeSpecName: "istio-data") pod "f7073af4-ef2d-482f-b1a4-647d266aba4a" (UID: "f7073af4-ef2d-482f-b1a4-647d266aba4a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:43:57.014952 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:57.014912 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7073af4-ef2d-482f-b1a4-647d266aba4a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f7073af4-ef2d-482f-b1a4-647d266aba4a" (UID: "f7073af4-ef2d-482f-b1a4-647d266aba4a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:43:57.014952 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:57.014921 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f7073af4-ef2d-482f-b1a4-647d266aba4a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f7073af4-ef2d-482f-b1a4-647d266aba4a" (UID: "f7073af4-ef2d-482f-b1a4-647d266aba4a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 21:43:57.015143 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:57.014926 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7073af4-ef2d-482f-b1a4-647d266aba4a-istio-token" (OuterVolumeSpecName: "istio-token") pod "f7073af4-ef2d-482f-b1a4-647d266aba4a" (UID: "f7073af4-ef2d-482f-b1a4-647d266aba4a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:43:57.015232 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:57.015213 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7073af4-ef2d-482f-b1a4-647d266aba4a-kube-api-access-fxqtw" (OuterVolumeSpecName: "kube-api-access-fxqtw") pod "f7073af4-ef2d-482f-b1a4-647d266aba4a" (UID: "f7073af4-ef2d-482f-b1a4-647d266aba4a"). InnerVolumeSpecName "kube-api-access-fxqtw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:43:57.114019 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:57.113980 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7073af4-ef2d-482f-b1a4-647d266aba4a-istio-envoy\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:43:57.114019 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:57.114012 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7073af4-ef2d-482f-b1a4-647d266aba4a-istio-data\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:43:57.114019 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:57.114024 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fxqtw\" (UniqueName: \"kubernetes.io/projected/f7073af4-ef2d-482f-b1a4-647d266aba4a-kube-api-access-fxqtw\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:43:57.114301 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:57.114036 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7073af4-ef2d-482f-b1a4-647d266aba4a-istio-token\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:43:57.114301 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:57.114048 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7073af4-ef2d-482f-b1a4-647d266aba4a-istio-podinfo\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:43:57.379714 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:57.379681 2579 generic.go:356] "Generic (PLEG): container finished" podID="f7073af4-ef2d-482f-b1a4-647d266aba4a" containerID="3769d8d418a076a3a2d7317b40ecf6c738755ba544301c279ec11ae290e550ab" exitCode=0 Jul 15 21:43:57.379899 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:57.379728 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h" Jul 15 21:43:57.379899 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:57.379768 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h" event={"ID":"f7073af4-ef2d-482f-b1a4-647d266aba4a","Type":"ContainerDied","Data":"3769d8d418a076a3a2d7317b40ecf6c738755ba544301c279ec11ae290e550ab"} Jul 15 21:43:57.379899 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:57.379820 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h" event={"ID":"f7073af4-ef2d-482f-b1a4-647d266aba4a","Type":"ContainerDied","Data":"7688a0e2afc7facf56c9f86f76511f3b20612fa16ed4f039d1b9859e2f35be75"} Jul 15 21:43:57.379899 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:57.379841 2579 scope.go:117] "RemoveContainer" containerID="3769d8d418a076a3a2d7317b40ecf6c738755ba544301c279ec11ae290e550ab" Jul 15 21:43:57.389752 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:57.389733 2579 scope.go:117] "RemoveContainer" containerID="c0a150a9e3e693dd4248cf420903df05a545ec7b986c3914912c6348099cf13f" Jul 15 21:43:57.397722 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:57.397707 2579 scope.go:117] "RemoveContainer" containerID="3769d8d418a076a3a2d7317b40ecf6c738755ba544301c279ec11ae290e550ab" Jul 15 21:43:57.397956 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:43:57.397940 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3769d8d418a076a3a2d7317b40ecf6c738755ba544301c279ec11ae290e550ab\": container with ID starting with 3769d8d418a076a3a2d7317b40ecf6c738755ba544301c279ec11ae290e550ab not found: ID does not exist" containerID="3769d8d418a076a3a2d7317b40ecf6c738755ba544301c279ec11ae290e550ab" Jul 15 21:43:57.398029 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:57.397964 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3769d8d418a076a3a2d7317b40ecf6c738755ba544301c279ec11ae290e550ab"} err="failed to get container status \"3769d8d418a076a3a2d7317b40ecf6c738755ba544301c279ec11ae290e550ab\": rpc error: code = NotFound desc = could not find container \"3769d8d418a076a3a2d7317b40ecf6c738755ba544301c279ec11ae290e550ab\": container with ID starting with 3769d8d418a076a3a2d7317b40ecf6c738755ba544301c279ec11ae290e550ab not found: ID does not exist" Jul 15 21:43:57.398029 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:57.397978 2579 scope.go:117] "RemoveContainer" containerID="c0a150a9e3e693dd4248cf420903df05a545ec7b986c3914912c6348099cf13f" Jul 15 21:43:57.398208 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:43:57.398198 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0a150a9e3e693dd4248cf420903df05a545ec7b986c3914912c6348099cf13f\": container with ID starting with c0a150a9e3e693dd4248cf420903df05a545ec7b986c3914912c6348099cf13f not found: ID does not exist" containerID="c0a150a9e3e693dd4248cf420903df05a545ec7b986c3914912c6348099cf13f" Jul 15 21:43:57.398256 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:57.398214 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0a150a9e3e693dd4248cf420903df05a545ec7b986c3914912c6348099cf13f"} err="failed to get container status \"c0a150a9e3e693dd4248cf420903df05a545ec7b986c3914912c6348099cf13f\": rpc error: code = NotFound desc = could not find container \"c0a150a9e3e693dd4248cf420903df05a545ec7b986c3914912c6348099cf13f\": container with ID starting with c0a150a9e3e693dd4248cf420903df05a545ec7b986c3914912c6348099cf13f not found: ID does not exist" Jul 15 21:43:57.403905 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:57.403885 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h"] Jul 15 21:43:57.408452 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:57.408429 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-da009d-openshift-default-756f669c5c-9b27h"] Jul 15 21:43:57.997227 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:57.997193 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7073af4-ef2d-482f-b1a4-647d266aba4a" path="/var/lib/kubelet/pods/f7073af4-ef2d-482f-b1a4-647d266aba4a/volumes" Jul 15 21:43:59.909831 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:43:59.909791 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:44:00.235352 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:00.235254 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:44:00.512394 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:00.512298 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b"] Jul 15 21:44:00.512872 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:00.512851 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7073af4-ef2d-482f-b1a4-647d266aba4a" containerName="istio-proxy" Jul 15 21:44:00.512872 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:00.512871 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7073af4-ef2d-482f-b1a4-647d266aba4a" containerName="istio-proxy" Jul 15 21:44:00.513022 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:00.512890 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7073af4-ef2d-482f-b1a4-647d266aba4a" containerName="istio-proxy" Jul 15 21:44:00.513022 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:00.512900 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7073af4-ef2d-482f-b1a4-647d266aba4a" containerName="istio-proxy" Jul 15 21:44:00.513022 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:00.513008 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7073af4-ef2d-482f-b1a4-647d266aba4a" containerName="istio-proxy" Jul 15 21:44:00.513022 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:00.513020 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7073af4-ef2d-482f-b1a4-647d266aba4a" containerName="istio-proxy" Jul 15 21:44:00.516357 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:00.516338 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b" Jul 15 21:44:00.519245 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:00.519228 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-b7f88c-openshift-default-dockercfg-7mg2r\"" Jul 15 21:44:00.528178 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:00.528154 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b"] Jul 15 21:44:00.540814 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:00.540777 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:44:00.543947 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:00.543919 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bac6a42-a26f-4a6e-85e3-f0fd79346a9d-credential-socket\") pod \"e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b\" (UID: \"8bac6a42-a26f-4a6e-85e3-f0fd79346a9d\") " pod="openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b" Jul 15 21:44:00.544032 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:00.543963 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8bac6a42-a26f-4a6e-85e3-f0fd79346a9d-istio-data\") pod \"e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b\" (UID: \"8bac6a42-a26f-4a6e-85e3-f0fd79346a9d\") " pod="openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b" Jul 15 21:44:00.544032 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:00.543999 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8bac6a42-a26f-4a6e-85e3-f0fd79346a9d-istio-podinfo\") pod \"e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b\" (UID: \"8bac6a42-a26f-4a6e-85e3-f0fd79346a9d\") " pod="openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b" Jul 15 21:44:00.544170 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:00.544041 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8bac6a42-a26f-4a6e-85e3-f0fd79346a9d-istio-envoy\") pod \"e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b\" (UID: \"8bac6a42-a26f-4a6e-85e3-f0fd79346a9d\") " pod="openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b" Jul 15 21:44:00.544170 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:00.544080 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8bac6a42-a26f-4a6e-85e3-f0fd79346a9d-istio-token\") pod \"e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b\" (UID: \"8bac6a42-a26f-4a6e-85e3-f0fd79346a9d\") " pod="openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b" Jul 15 21:44:00.544170 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:00.544118 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8bac6a42-a26f-4a6e-85e3-f0fd79346a9d-istiod-ca-cert\") pod \"e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b\" (UID: \"8bac6a42-a26f-4a6e-85e3-f0fd79346a9d\") " pod="openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b" Jul 15 21:44:00.544170 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:00.544153 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4twbs\" (UniqueName: \"kubernetes.io/projected/8bac6a42-a26f-4a6e-85e3-f0fd79346a9d-kube-api-access-4twbs\") pod \"e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b\" (UID: \"8bac6a42-a26f-4a6e-85e3-f0fd79346a9d\") " pod="openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b" Jul 15 21:44:00.544378 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:00.544195 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8bac6a42-a26f-4a6e-85e3-f0fd79346a9d-workload-certs\") pod \"e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b\" (UID: \"8bac6a42-a26f-4a6e-85e3-f0fd79346a9d\") " pod="openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b" Jul 15 21:44:00.544378 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:00.544246 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bac6a42-a26f-4a6e-85e3-f0fd79346a9d-workload-socket\") pod \"e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b\" (UID: \"8bac6a42-a26f-4a6e-85e3-f0fd79346a9d\") " pod="openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b" Jul 15 21:44:00.645155 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:00.645122 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8bac6a42-a26f-4a6e-85e3-f0fd79346a9d-istiod-ca-cert\") pod \"e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b\" (UID: \"8bac6a42-a26f-4a6e-85e3-f0fd79346a9d\") " pod="openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b" Jul 15 21:44:00.645370 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:00.645161 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4twbs\" (UniqueName: \"kubernetes.io/projected/8bac6a42-a26f-4a6e-85e3-f0fd79346a9d-kube-api-access-4twbs\") pod \"e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b\" (UID: \"8bac6a42-a26f-4a6e-85e3-f0fd79346a9d\") " pod="openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b" Jul 15 21:44:00.645370 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:00.645189 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8bac6a42-a26f-4a6e-85e3-f0fd79346a9d-workload-certs\") pod \"e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b\" (UID: \"8bac6a42-a26f-4a6e-85e3-f0fd79346a9d\") " pod="openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b" Jul 15 21:44:00.645370 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:00.645226 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bac6a42-a26f-4a6e-85e3-f0fd79346a9d-workload-socket\") pod \"e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b\" (UID: \"8bac6a42-a26f-4a6e-85e3-f0fd79346a9d\") " pod="openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b" Jul 15 21:44:00.645370 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:00.645286 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bac6a42-a26f-4a6e-85e3-f0fd79346a9d-credential-socket\") pod \"e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b\" (UID: \"8bac6a42-a26f-4a6e-85e3-f0fd79346a9d\") " pod="openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b" Jul 15 21:44:00.645370 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:00.645309 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8bac6a42-a26f-4a6e-85e3-f0fd79346a9d-istio-data\") pod \"e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b\" (UID: \"8bac6a42-a26f-4a6e-85e3-f0fd79346a9d\") " pod="openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b" Jul 15 21:44:00.645370 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:00.645337 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8bac6a42-a26f-4a6e-85e3-f0fd79346a9d-istio-podinfo\") pod \"e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b\" (UID: \"8bac6a42-a26f-4a6e-85e3-f0fd79346a9d\") " pod="openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b" Jul 15 21:44:00.645677 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:00.645375 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8bac6a42-a26f-4a6e-85e3-f0fd79346a9d-istio-envoy\") pod \"e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b\" (UID: \"8bac6a42-a26f-4a6e-85e3-f0fd79346a9d\") " pod="openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b" Jul 15 21:44:00.645677 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:00.645399 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8bac6a42-a26f-4a6e-85e3-f0fd79346a9d-istio-token\") pod \"e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b\" (UID: \"8bac6a42-a26f-4a6e-85e3-f0fd79346a9d\") " pod="openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b" Jul 15 21:44:00.645777 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:00.645738 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bac6a42-a26f-4a6e-85e3-f0fd79346a9d-workload-socket\") pod \"e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b\" (UID: \"8bac6a42-a26f-4a6e-85e3-f0fd79346a9d\") " pod="openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b" Jul 15 21:44:00.645831 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:00.645791 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bac6a42-a26f-4a6e-85e3-f0fd79346a9d-credential-socket\") pod \"e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b\" (UID: \"8bac6a42-a26f-4a6e-85e3-f0fd79346a9d\") " pod="openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b" Jul 15 21:44:00.645885 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:00.645835 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8bac6a42-a26f-4a6e-85e3-f0fd79346a9d-istio-data\") pod \"e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b\" (UID: \"8bac6a42-a26f-4a6e-85e3-f0fd79346a9d\") " pod="openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b" Jul 15 21:44:00.645959 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:00.645931 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8bac6a42-a26f-4a6e-85e3-f0fd79346a9d-istiod-ca-cert\") pod \"e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b\" (UID: \"8bac6a42-a26f-4a6e-85e3-f0fd79346a9d\") " pod="openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b" Jul 15 21:44:00.646143 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:00.646127 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8bac6a42-a26f-4a6e-85e3-f0fd79346a9d-workload-certs\") pod \"e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b\" (UID: \"8bac6a42-a26f-4a6e-85e3-f0fd79346a9d\") " pod="openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b" Jul 15 21:44:00.647801 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:00.647772 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8bac6a42-a26f-4a6e-85e3-f0fd79346a9d-istio-envoy\") pod \"e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b\" (UID: \"8bac6a42-a26f-4a6e-85e3-f0fd79346a9d\") " pod="openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b" Jul 15 21:44:00.648008 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:00.647992 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8bac6a42-a26f-4a6e-85e3-f0fd79346a9d-istio-podinfo\") pod \"e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b\" (UID: \"8bac6a42-a26f-4a6e-85e3-f0fd79346a9d\") " pod="openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b" Jul 15 21:44:00.659286 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:00.659257 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8bac6a42-a26f-4a6e-85e3-f0fd79346a9d-istio-token\") pod \"e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b\" (UID: \"8bac6a42-a26f-4a6e-85e3-f0fd79346a9d\") " pod="openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b" Jul 15 21:44:00.659623 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:00.659604 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4twbs\" (UniqueName: \"kubernetes.io/projected/8bac6a42-a26f-4a6e-85e3-f0fd79346a9d-kube-api-access-4twbs\") pod \"e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b\" (UID: \"8bac6a42-a26f-4a6e-85e3-f0fd79346a9d\") " pod="openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b" Jul 15 21:44:00.800502 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:00.800414 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:44:00.828353 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:00.828314 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b" Jul 15 21:44:00.923273 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:00.923234 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:44:00.971632 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:00.971600 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b"] Jul 15 21:44:00.973131 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:44:00.973090 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bac6a42_a26f_4a6e_85e3_f0fd79346a9d.slice/crio-c57867ebbade6619b1583fcfdfc5176c20c23cd46424ed03d7a7def9134a5c51 WatchSource:0}: Error finding container c57867ebbade6619b1583fcfdfc5176c20c23cd46424ed03d7a7def9134a5c51: Status 404 returned error can't find the container with id c57867ebbade6619b1583fcfdfc5176c20c23cd46424ed03d7a7def9134a5c51 Jul 15 21:44:00.975262 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:00.975228 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:44:00.975371 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:00.975312 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:44:00.975371 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:00.975355 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:44:01.406221 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:01.406181 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b" event={"ID":"8bac6a42-a26f-4a6e-85e3-f0fd79346a9d","Type":"ContainerStarted","Data":"be30ec63d42443406f475eec59fc5c00ac14c33e0a3c4b4730ed6b936b280060"} Jul 15 21:44:01.406394 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:01.406252 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b" event={"ID":"8bac6a42-a26f-4a6e-85e3-f0fd79346a9d","Type":"ContainerStarted","Data":"c57867ebbade6619b1583fcfdfc5176c20c23cd46424ed03d7a7def9134a5c51"} Jul 15 21:44:01.434231 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:01.434179 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b" podStartSLOduration=1.434153388 podStartE2EDuration="1.434153388s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:44:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:44:01.433351943 +0000 UTC m=+2545.974664493" watchObservedRunningTime="2026-07-15 21:44:01.434153388 +0000 UTC m=+2545.975465938" Jul 15 21:44:01.828858 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:01.828746 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b" Jul 15 21:44:02.848017 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:02.847973 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b" Jul 15 21:44:03.416512 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:03.416478 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b" Jul 15 21:44:03.417453 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:03.417430 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b" Jul 15 21:44:03.515593 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:03.515556 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:44:03.671995 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:03.671897 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:44:03.772744 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:03.772704 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:44:03.898420 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:03.898387 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:44:04.234813 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:04.234778 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:44:04.333967 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:04.333933 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:44:07.843368 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:07.843273 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:44:09.363913 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:09.363878 2579 patch_prober.go:28] interesting pod/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 15 21:44:09.364338 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:09.363946 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g" podUID="f6d86286-9979-41b8-89cd-dea1057b2dff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 15 21:44:09.456570 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:09.456535 2579 generic.go:356] "Generic (PLEG): container finished" podID="f6d86286-9979-41b8-89cd-dea1057b2dff" containerID="7a707e09a8ffa1f0e6a33a90611635ba2342af5c4b8468541eba0e474581dfb5" exitCode=0 Jul 15 21:44:09.456735 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:09.456580 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g" event={"ID":"f6d86286-9979-41b8-89cd-dea1057b2dff","Type":"ContainerDied","Data":"7a707e09a8ffa1f0e6a33a90611635ba2342af5c4b8468541eba0e474581dfb5"} Jul 15 21:44:09.457142 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:09.457125 2579 scope.go:117] "RemoveContainer" containerID="7a707e09a8ffa1f0e6a33a90611635ba2342af5c4b8468541eba0e474581dfb5" Jul 15 21:44:09.458342 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:09.458315 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:44:09.458420 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:09.458399 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:44:09.458458 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:09.458441 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:44:10.468184 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:10.468143 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g" event={"ID":"f6d86286-9979-41b8-89cd-dea1057b2dff","Type":"ContainerStarted","Data":"3a0c7335b5753b7a0315a687263c620d5a8994d428ef259bd5ee4442ffecaeac"} Jul 15 21:44:11.017398 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:11.017358 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g" Jul 15 21:44:11.017677 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:11.017657 2579 patch_prober.go:28] interesting pod/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 15 21:44:11.017803 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:11.017784 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g" podUID="f6d86286-9979-41b8-89cd-dea1057b2dff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 15 21:44:12.016982 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:12.016947 2579 patch_prober.go:28] interesting pod/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 15 21:44:12.017411 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:12.017007 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g" podUID="f6d86286-9979-41b8-89cd-dea1057b2dff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 15 21:44:13.017184 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:13.017078 2579 patch_prober.go:28] interesting pod/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 15 21:44:13.017184 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:13.017138 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g" podUID="f6d86286-9979-41b8-89cd-dea1057b2dff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 15 21:44:14.016929 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:14.016891 2579 patch_prober.go:28] interesting pod/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 15 21:44:14.017227 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:14.016957 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g" podUID="f6d86286-9979-41b8-89cd-dea1057b2dff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 15 21:44:15.021248 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:15.021212 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g" Jul 15 21:44:15.021629 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:15.021510 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g" Jul 15 21:44:15.022236 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:15.022216 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g" Jul 15 21:44:16.508683 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:16.508650 2579 generic.go:356] "Generic (PLEG): container finished" podID="8bac6a42-a26f-4a6e-85e3-f0fd79346a9d" containerID="be30ec63d42443406f475eec59fc5c00ac14c33e0a3c4b4730ed6b936b280060" exitCode=0 Jul 15 21:44:16.509209 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:16.508726 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b" event={"ID":"8bac6a42-a26f-4a6e-85e3-f0fd79346a9d","Type":"ContainerDied","Data":"be30ec63d42443406f475eec59fc5c00ac14c33e0a3c4b4730ed6b936b280060"} Jul 15 21:44:16.509333 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:16.509316 2579 scope.go:117] "RemoveContainer" containerID="be30ec63d42443406f475eec59fc5c00ac14c33e0a3c4b4730ed6b936b280060" Jul 15 21:44:16.510263 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:16.510237 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:44:16.510360 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:16.510319 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:44:16.510360 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:16.510352 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 21:44:17.203892 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:17.203854 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:44:17.279298 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:17.279262 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:44:17.382244 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:17.382211 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:44:17.516175 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:17.516078 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b" event={"ID":"8bac6a42-a26f-4a6e-85e3-f0fd79346a9d","Type":"ContainerStarted","Data":"89271f5af60c3c36c38b8db31e2e7a2b2dda0e9496307f117e5e07b239c4d9cc"} Jul 15 21:44:17.830886 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:17.830790 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b" Jul 15 21:44:17.831333 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:17.831307 2579 patch_prober.go:28] interesting pod/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 15 21:44:17.831539 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:17.831513 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b" podUID="8bac6a42-a26f-4a6e-85e3-f0fd79346a9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 15 21:44:18.416867 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:18.416819 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b" Jul 15 21:44:18.829489 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:18.829382 2579 patch_prober.go:28] interesting pod/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 15 21:44:18.829489 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:18.829467 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b" podUID="8bac6a42-a26f-4a6e-85e3-f0fd79346a9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 15 21:44:19.829385 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:19.829343 2579 patch_prober.go:28] interesting pod/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 15 21:44:19.829694 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:19.829659 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b" podUID="8bac6a42-a26f-4a6e-85e3-f0fd79346a9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 15 21:44:20.828974 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:20.828935 2579 patch_prober.go:28] interesting pod/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 15 21:44:20.829343 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:20.829300 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b" podUID="8bac6a42-a26f-4a6e-85e3-f0fd79346a9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 15 21:44:21.833428 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:21.833395 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b" Jul 15 21:44:21.834593 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:21.834570 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b" Jul 15 21:44:25.123078 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:25.123028 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:44:25.219994 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:25.219958 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:44:39.069260 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:39.069213 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:44:39.144919 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:39.144882 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:44:45.305234 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:45.305190 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:44:48.889823 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:44:48.889784 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:45:10.920860 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:45:10.920816 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:45:14.491222 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:45:14.491188 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:45:40.357204 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:45:40.357160 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:45:58.821027 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:45:58.820986 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:45:59.043556 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:45:59.043523 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:46:15.191476 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:46:15.191439 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:46:20.206761 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:46:20.206724 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:46:21.527643 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:46:21.527601 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:46:26.141348 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:46:26.141312 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:46:26.350268 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:46:26.350231 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:46:33.289985 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:46:33.289890 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:46:36.216807 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:46:36.216774 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m5r2m_0db09c1e-d24d-4519-93c8-c185018c3d24/ovn-acl-logging/0.log" Jul 15 21:46:36.220422 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:46:36.220399 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m5r2m_0db09c1e-d24d-4519-93c8-c185018c3d24/ovn-acl-logging/0.log" Jul 15 21:46:54.581278 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:46:54.581244 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:47:12.311302 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:47:12.311266 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:47:23.829432 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:47:23.829393 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:47:24.718252 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:47:24.718216 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:47:26.615207 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:47:26.615165 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:47:30.541278 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:47:30.541245 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:47:55.716372 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:47:55.716337 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:47:59.881380 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:47:59.881288 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:48:21.087179 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:48:21.087144 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:48:27.989863 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:48:27.989819 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:48:42.301101 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:48:42.301051 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:48:46.285240 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:48:46.285167 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:48:46.381642 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:48:46.381604 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:48:48.032295 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:48:48.032258 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:48:48.929441 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:48:48.929401 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:48:58.518989 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:48:58.518924 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:49:02.564074 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:49:02.564017 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:49:18.257232 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:49:18.257199 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:49:18.419427 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:49:18.419392 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:49:20.795004 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:49:20.794962 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:49:25.195095 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:49:25.195039 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:49:26.023850 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:49:26.023811 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:49:26.123617 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:49:26.123582 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:49:40.939461 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:49:40.939428 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:50:00.884009 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:00.883965 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:50:05.098760 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:05.098718 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:50:05.937797 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:05.937754 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:50:06.849200 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:06.849164 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7346-139642136431792"] Jul 15 21:50:06.852837 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:06.852812 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7346-139642136431792" Jul 15 21:50:06.855610 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:06.855585 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-655jp\"" Jul 15 21:50:06.858173 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:06.858145 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7346-139642136431792"] Jul 15 21:50:06.951984 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:06.951942 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlwks\" (UniqueName: \"kubernetes.io/projected/070d207e-0204-4d3f-bca5-59d1def38c5f-kube-api-access-jlwks\") pod \"test-curl-7346-139642136431792\" (UID: \"070d207e-0204-4d3f-bca5-59d1def38c5f\") " pod="opendatahub/test-curl-7346-139642136431792" Jul 15 21:50:07.052802 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:07.052764 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlwks\" (UniqueName: \"kubernetes.io/projected/070d207e-0204-4d3f-bca5-59d1def38c5f-kube-api-access-jlwks\") pod \"test-curl-7346-139642136431792\" (UID: \"070d207e-0204-4d3f-bca5-59d1def38c5f\") " pod="opendatahub/test-curl-7346-139642136431792" Jul 15 21:50:07.062011 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:07.061979 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlwks\" (UniqueName: \"kubernetes.io/projected/070d207e-0204-4d3f-bca5-59d1def38c5f-kube-api-access-jlwks\") pod \"test-curl-7346-139642136431792\" (UID: \"070d207e-0204-4d3f-bca5-59d1def38c5f\") " pod="opendatahub/test-curl-7346-139642136431792" Jul 15 21:50:07.164461 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:07.164422 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7346-139642136431792" Jul 15 21:50:07.305948 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:07.305922 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7346-139642136431792"] Jul 15 21:50:07.308137 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:50:07.308104 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod070d207e_0204_4d3f_bca5_59d1def38c5f.slice/crio-134d8135908ca509110e6ab8faa5d9c0e866b8426c664620d5845a6a5f22c062 WatchSource:0}: Error finding container 134d8135908ca509110e6ab8faa5d9c0e866b8426c664620d5845a6a5f22c062: Status 404 returned error can't find the container with id 134d8135908ca509110e6ab8faa5d9c0e866b8426c664620d5845a6a5f22c062 Jul 15 21:50:07.309861 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:07.309842 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 21:50:07.452793 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:07.452699 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7346-139642136431792" event={"ID":"070d207e-0204-4d3f-bca5-59d1def38c5f","Type":"ContainerStarted","Data":"134d8135908ca509110e6ab8faa5d9c0e866b8426c664620d5845a6a5f22c062"} Jul 15 21:50:08.459852 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:08.459815 2579 generic.go:356] "Generic (PLEG): container finished" podID="070d207e-0204-4d3f-bca5-59d1def38c5f" containerID="9675de97c0cdf8ef29522fa9c3bd55486d6e0bebec4728267770dc28b56140ce" exitCode=0 Jul 15 21:50:08.460325 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:08.459902 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7346-139642136431792" event={"ID":"070d207e-0204-4d3f-bca5-59d1def38c5f","Type":"ContainerDied","Data":"9675de97c0cdf8ef29522fa9c3bd55486d6e0bebec4728267770dc28b56140ce"} Jul 15 21:50:09.597886 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:09.597862 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7346-139642136431792" Jul 15 21:50:09.618268 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:09.618226 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7346-139642136431792_070d207e-0204-4d3f-bca5-59d1def38c5f/test-curl-7346-139642136431792/0.log" Jul 15 21:50:09.639890 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:09.639857 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7346-139642136431792"] Jul 15 21:50:09.643512 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:09.643488 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7346-139642136431792"] Jul 15 21:50:09.677509 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:09.677474 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jlwks\" (UniqueName: \"kubernetes.io/projected/070d207e-0204-4d3f-bca5-59d1def38c5f-kube-api-access-jlwks\") pod \"070d207e-0204-4d3f-bca5-59d1def38c5f\" (UID: \"070d207e-0204-4d3f-bca5-59d1def38c5f\") " Jul 15 21:50:09.679791 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:09.679754 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/070d207e-0204-4d3f-bca5-59d1def38c5f-kube-api-access-jlwks" (OuterVolumeSpecName: "kube-api-access-jlwks") pod "070d207e-0204-4d3f-bca5-59d1def38c5f" (UID: "070d207e-0204-4d3f-bca5-59d1def38c5f"). InnerVolumeSpecName "kube-api-access-jlwks". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:50:09.761809 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:09.761723 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7346-139642136430128"] Jul 15 21:50:09.762141 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:09.762127 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="070d207e-0204-4d3f-bca5-59d1def38c5f" containerName="test-curl-7346-139642136431792" Jul 15 21:50:09.762201 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:09.762142 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="070d207e-0204-4d3f-bca5-59d1def38c5f" containerName="test-curl-7346-139642136431792" Jul 15 21:50:09.762238 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:09.762210 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="070d207e-0204-4d3f-bca5-59d1def38c5f" containerName="test-curl-7346-139642136431792" Jul 15 21:50:09.765170 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:09.765153 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7346-139642136430128" Jul 15 21:50:09.771490 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:09.771466 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7346-139642136430128"] Jul 15 21:50:09.778397 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:09.778370 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jlwks\" (UniqueName: \"kubernetes.io/projected/070d207e-0204-4d3f-bca5-59d1def38c5f-kube-api-access-jlwks\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:50:09.879478 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:09.879438 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w26z4\" (UniqueName: \"kubernetes.io/projected/a1cb866a-20db-4445-a211-714d3b642cff-kube-api-access-w26z4\") pod \"test-curl-7346-139642136430128\" (UID: \"a1cb866a-20db-4445-a211-714d3b642cff\") " pod="opendatahub/test-curl-7346-139642136430128" Jul 15 21:50:09.980429 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:09.980395 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w26z4\" (UniqueName: \"kubernetes.io/projected/a1cb866a-20db-4445-a211-714d3b642cff-kube-api-access-w26z4\") pod \"test-curl-7346-139642136430128\" (UID: \"a1cb866a-20db-4445-a211-714d3b642cff\") " pod="opendatahub/test-curl-7346-139642136430128" Jul 15 21:50:09.988965 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:09.988944 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w26z4\" (UniqueName: \"kubernetes.io/projected/a1cb866a-20db-4445-a211-714d3b642cff-kube-api-access-w26z4\") pod \"test-curl-7346-139642136430128\" (UID: \"a1cb866a-20db-4445-a211-714d3b642cff\") " pod="opendatahub/test-curl-7346-139642136430128" Jul 15 21:50:09.997298 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:09.997270 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="070d207e-0204-4d3f-bca5-59d1def38c5f" path="/var/lib/kubelet/pods/070d207e-0204-4d3f-bca5-59d1def38c5f/volumes" Jul 15 21:50:10.076162 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:10.076053 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7346-139642136430128" Jul 15 21:50:10.214553 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:10.214523 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7346-139642136430128"] Jul 15 21:50:10.217225 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:50:10.217189 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda1cb866a_20db_4445_a211_714d3b642cff.slice/crio-5d8cea0df7ad229211f809dfbd090e4f3bbfebcb942034c1d706ff3925000235 WatchSource:0}: Error finding container 5d8cea0df7ad229211f809dfbd090e4f3bbfebcb942034c1d706ff3925000235: Status 404 returned error can't find the container with id 5d8cea0df7ad229211f809dfbd090e4f3bbfebcb942034c1d706ff3925000235 Jul 15 21:50:10.472214 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:10.472186 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7346-139642136431792" Jul 15 21:50:10.472414 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:10.472191 2579 scope.go:117] "RemoveContainer" containerID="9675de97c0cdf8ef29522fa9c3bd55486d6e0bebec4728267770dc28b56140ce" Jul 15 21:50:10.473593 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:10.473443 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7346-139642136430128" event={"ID":"a1cb866a-20db-4445-a211-714d3b642cff","Type":"ContainerStarted","Data":"5d8cea0df7ad229211f809dfbd090e4f3bbfebcb942034c1d706ff3925000235"} Jul 15 21:50:11.481860 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:11.481828 2579 generic.go:356] "Generic (PLEG): container finished" podID="a1cb866a-20db-4445-a211-714d3b642cff" containerID="a9cecea96c15c0c3838c8f183d582127c7e2abc601e03944ab187abd40dddec5" exitCode=0 Jul 15 21:50:11.482326 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:11.481912 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7346-139642136430128" event={"ID":"a1cb866a-20db-4445-a211-714d3b642cff","Type":"ContainerDied","Data":"a9cecea96c15c0c3838c8f183d582127c7e2abc601e03944ab187abd40dddec5"} Jul 15 21:50:12.628954 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:12.628922 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7346-139642136430128" Jul 15 21:50:12.649090 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:12.649049 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7346-139642136430128_a1cb866a-20db-4445-a211-714d3b642cff/test-curl-7346-139642136430128/0.log" Jul 15 21:50:12.680525 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:12.680483 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7346-139642136430128"] Jul 15 21:50:12.682726 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:12.682697 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7346-139642136430128"] Jul 15 21:50:12.806753 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:12.806660 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w26z4\" (UniqueName: \"kubernetes.io/projected/a1cb866a-20db-4445-a211-714d3b642cff-kube-api-access-w26z4\") pod \"a1cb866a-20db-4445-a211-714d3b642cff\" (UID: \"a1cb866a-20db-4445-a211-714d3b642cff\") " Jul 15 21:50:12.808796 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:12.808770 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1cb866a-20db-4445-a211-714d3b642cff-kube-api-access-w26z4" (OuterVolumeSpecName: "kube-api-access-w26z4") pod "a1cb866a-20db-4445-a211-714d3b642cff" (UID: "a1cb866a-20db-4445-a211-714d3b642cff"). InnerVolumeSpecName "kube-api-access-w26z4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:50:12.843670 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:12.843634 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7346-139642131304640"] Jul 15 21:50:12.844107 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:12.844089 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1cb866a-20db-4445-a211-714d3b642cff" containerName="test-curl-7346-139642136430128" Jul 15 21:50:12.844202 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:12.844109 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1cb866a-20db-4445-a211-714d3b642cff" containerName="test-curl-7346-139642136430128" Jul 15 21:50:12.844262 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:12.844211 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1cb866a-20db-4445-a211-714d3b642cff" containerName="test-curl-7346-139642136430128" Jul 15 21:50:12.847495 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:12.847474 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7346-139642131304640" Jul 15 21:50:12.850709 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:12.850674 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7346-139642131304640"] Jul 15 21:50:12.907647 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:12.907617 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w26z4\" (UniqueName: \"kubernetes.io/projected/a1cb866a-20db-4445-a211-714d3b642cff-kube-api-access-w26z4\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:50:13.008595 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:13.008547 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5rbr\" (UniqueName: \"kubernetes.io/projected/b474e480-c4fe-484e-80b0-a90a991e8938-kube-api-access-m5rbr\") pod \"test-curl-7346-139642131304640\" (UID: \"b474e480-c4fe-484e-80b0-a90a991e8938\") " pod="opendatahub/test-curl-7346-139642131304640" Jul 15 21:50:13.109452 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:13.109418 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5rbr\" (UniqueName: \"kubernetes.io/projected/b474e480-c4fe-484e-80b0-a90a991e8938-kube-api-access-m5rbr\") pod \"test-curl-7346-139642131304640\" (UID: \"b474e480-c4fe-484e-80b0-a90a991e8938\") " pod="opendatahub/test-curl-7346-139642131304640" Jul 15 21:50:13.117656 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:13.117621 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5rbr\" (UniqueName: \"kubernetes.io/projected/b474e480-c4fe-484e-80b0-a90a991e8938-kube-api-access-m5rbr\") pod \"test-curl-7346-139642131304640\" (UID: \"b474e480-c4fe-484e-80b0-a90a991e8938\") " pod="opendatahub/test-curl-7346-139642131304640" Jul 15 21:50:13.157286 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:13.157235 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7346-139642131304640" Jul 15 21:50:13.292480 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:13.292452 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7346-139642131304640"] Jul 15 21:50:13.293559 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:50:13.293534 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb474e480_c4fe_484e_80b0_a90a991e8938.slice/crio-a51eb05dd00f781514f453c114d7ffffc88844b5226f9c849b0131d01f4eea8e WatchSource:0}: Error finding container a51eb05dd00f781514f453c114d7ffffc88844b5226f9c849b0131d01f4eea8e: Status 404 returned error can't find the container with id a51eb05dd00f781514f453c114d7ffffc88844b5226f9c849b0131d01f4eea8e Jul 15 21:50:13.494766 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:13.494672 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7346-139642136430128" Jul 15 21:50:13.494766 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:13.494689 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5d8cea0df7ad229211f809dfbd090e4f3bbfebcb942034c1d706ff3925000235" Jul 15 21:50:13.495994 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:13.495959 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7346-139642131304640" event={"ID":"b474e480-c4fe-484e-80b0-a90a991e8938","Type":"ContainerStarted","Data":"a51eb05dd00f781514f453c114d7ffffc88844b5226f9c849b0131d01f4eea8e"} Jul 15 21:50:13.506273 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:50:13.506234 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7346-139642136430128\" is forbidden: User \"system:node:ip-10-0-137-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-185.ec2.internal' and this object" podUID="a1cb866a-20db-4445-a211-714d3b642cff" pod="opendatahub/test-curl-7346-139642136430128" Jul 15 21:50:13.997429 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:13.997396 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1cb866a-20db-4445-a211-714d3b642cff" path="/var/lib/kubelet/pods/a1cb866a-20db-4445-a211-714d3b642cff/volumes" Jul 15 21:50:14.503113 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:14.503052 2579 generic.go:356] "Generic (PLEG): container finished" podID="b474e480-c4fe-484e-80b0-a90a991e8938" containerID="c980ff4c85da3ece62027f867e7f1fc00e2db51a88f4bd777c3dea7467747cde" exitCode=0 Jul 15 21:50:14.503303 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:14.503145 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7346-139642131304640" event={"ID":"b474e480-c4fe-484e-80b0-a90a991e8938","Type":"ContainerDied","Data":"c980ff4c85da3ece62027f867e7f1fc00e2db51a88f4bd777c3dea7467747cde"} Jul 15 21:50:15.649735 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:15.649710 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7346-139642131304640" Jul 15 21:50:15.670647 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:15.670615 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7346-139642131304640_b474e480-c4fe-484e-80b0-a90a991e8938/test-curl-7346-139642131304640/0.log" Jul 15 21:50:15.699512 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:15.699478 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7346-139642131304640"] Jul 15 21:50:15.701507 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:15.701485 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7346-139642131304640"] Jul 15 21:50:15.734441 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:15.734404 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m5rbr\" (UniqueName: \"kubernetes.io/projected/b474e480-c4fe-484e-80b0-a90a991e8938-kube-api-access-m5rbr\") pod \"b474e480-c4fe-484e-80b0-a90a991e8938\" (UID: \"b474e480-c4fe-484e-80b0-a90a991e8938\") " Jul 15 21:50:15.736554 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:15.736526 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b474e480-c4fe-484e-80b0-a90a991e8938-kube-api-access-m5rbr" (OuterVolumeSpecName: "kube-api-access-m5rbr") pod "b474e480-c4fe-484e-80b0-a90a991e8938" (UID: "b474e480-c4fe-484e-80b0-a90a991e8938"). InnerVolumeSpecName "kube-api-access-m5rbr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:50:15.832629 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:15.832549 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7346-139642131304640"] Jul 15 21:50:15.832905 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:15.832892 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b474e480-c4fe-484e-80b0-a90a991e8938" containerName="test-curl-7346-139642131304640" Jul 15 21:50:15.832949 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:15.832907 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b474e480-c4fe-484e-80b0-a90a991e8938" containerName="test-curl-7346-139642131304640" Jul 15 21:50:15.832985 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:15.832979 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b474e480-c4fe-484e-80b0-a90a991e8938" containerName="test-curl-7346-139642131304640" Jul 15 21:50:15.835347 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:15.835322 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m5rbr\" (UniqueName: \"kubernetes.io/projected/b474e480-c4fe-484e-80b0-a90a991e8938-kube-api-access-m5rbr\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:50:15.836052 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:15.836037 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7346-139642131304640" Jul 15 21:50:15.841939 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:15.841915 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7346-139642131304640"] Jul 15 21:50:15.936016 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:15.935980 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgbpt\" (UniqueName: \"kubernetes.io/projected/92fc69a3-1f38-415a-a26b-b54c06642dd3-kube-api-access-tgbpt\") pod \"test-curl-7346-139642131304640\" (UID: \"92fc69a3-1f38-415a-a26b-b54c06642dd3\") " pod="opendatahub/test-curl-7346-139642131304640" Jul 15 21:50:15.996702 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:15.996667 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b474e480-c4fe-484e-80b0-a90a991e8938" path="/var/lib/kubelet/pods/b474e480-c4fe-484e-80b0-a90a991e8938/volumes" Jul 15 21:50:16.036997 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:16.036962 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgbpt\" (UniqueName: \"kubernetes.io/projected/92fc69a3-1f38-415a-a26b-b54c06642dd3-kube-api-access-tgbpt\") pod \"test-curl-7346-139642131304640\" (UID: \"92fc69a3-1f38-415a-a26b-b54c06642dd3\") " pod="opendatahub/test-curl-7346-139642131304640" Jul 15 21:50:16.045719 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:16.045694 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgbpt\" (UniqueName: \"kubernetes.io/projected/92fc69a3-1f38-415a-a26b-b54c06642dd3-kube-api-access-tgbpt\") pod \"test-curl-7346-139642131304640\" (UID: \"92fc69a3-1f38-415a-a26b-b54c06642dd3\") " pod="opendatahub/test-curl-7346-139642131304640" Jul 15 21:50:16.146212 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:16.146169 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7346-139642131304640" Jul 15 21:50:16.280834 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:16.280808 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7346-139642131304640"] Jul 15 21:50:16.281667 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:50:16.281643 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod92fc69a3_1f38_415a_a26b_b54c06642dd3.slice/crio-3640dfd5bedce17d02d8197bdf5a58f2962f9170854c656cc1f0de310eb6417e WatchSource:0}: Error finding container 3640dfd5bedce17d02d8197bdf5a58f2962f9170854c656cc1f0de310eb6417e: Status 404 returned error can't find the container with id 3640dfd5bedce17d02d8197bdf5a58f2962f9170854c656cc1f0de310eb6417e Jul 15 21:50:16.483281 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:16.483203 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:50:16.515900 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:16.515866 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7346-139642131304640" event={"ID":"92fc69a3-1f38-415a-a26b-b54c06642dd3","Type":"ContainerStarted","Data":"3640dfd5bedce17d02d8197bdf5a58f2962f9170854c656cc1f0de310eb6417e"} Jul 15 21:50:16.517170 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:16.517148 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7346-139642131304640" Jul 15 21:50:16.517296 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:16.517211 2579 scope.go:117] "RemoveContainer" containerID="c980ff4c85da3ece62027f867e7f1fc00e2db51a88f4bd777c3dea7467747cde" Jul 15 21:50:17.524965 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:17.524929 2579 generic.go:356] "Generic (PLEG): container finished" podID="92fc69a3-1f38-415a-a26b-b54c06642dd3" containerID="e23fd9b8b446a3a3f8aedbbf10ddf3ab0c9f17177965b0236f07ba2ddd32c335" exitCode=0 Jul 15 21:50:17.525445 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:17.525010 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7346-139642131304640" event={"ID":"92fc69a3-1f38-415a-a26b-b54c06642dd3","Type":"ContainerDied","Data":"e23fd9b8b446a3a3f8aedbbf10ddf3ab0c9f17177965b0236f07ba2ddd32c335"} Jul 15 21:50:18.663969 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:18.663943 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7346-139642131304640" Jul 15 21:50:18.684772 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:18.684740 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7346-139642131304640_92fc69a3-1f38-415a-a26b-b54c06642dd3/test-curl-7346-139642131304640/0.log" Jul 15 21:50:18.712046 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:18.712009 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7346-139642131304640"] Jul 15 21:50:18.716290 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:18.716267 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7346-139642131304640"] Jul 15 21:50:18.760802 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:18.760776 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tgbpt\" (UniqueName: \"kubernetes.io/projected/92fc69a3-1f38-415a-a26b-b54c06642dd3-kube-api-access-tgbpt\") pod \"92fc69a3-1f38-415a-a26b-b54c06642dd3\" (UID: \"92fc69a3-1f38-415a-a26b-b54c06642dd3\") " Jul 15 21:50:18.762891 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:18.762863 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92fc69a3-1f38-415a-a26b-b54c06642dd3-kube-api-access-tgbpt" (OuterVolumeSpecName: "kube-api-access-tgbpt") pod "92fc69a3-1f38-415a-a26b-b54c06642dd3" (UID: "92fc69a3-1f38-415a-a26b-b54c06642dd3"). InnerVolumeSpecName "kube-api-access-tgbpt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:50:18.833043 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:18.832956 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7346-139642131305648"] Jul 15 21:50:18.833394 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:18.833378 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92fc69a3-1f38-415a-a26b-b54c06642dd3" containerName="test-curl-7346-139642131304640" Jul 15 21:50:18.833457 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:18.833397 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="92fc69a3-1f38-415a-a26b-b54c06642dd3" containerName="test-curl-7346-139642131304640" Jul 15 21:50:18.833491 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:18.833457 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="92fc69a3-1f38-415a-a26b-b54c06642dd3" containerName="test-curl-7346-139642131304640" Jul 15 21:50:18.836618 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:18.836601 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7346-139642131305648" Jul 15 21:50:18.840873 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:18.840845 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7346-139642131305648"] Jul 15 21:50:18.862262 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:18.862228 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tgbpt\" (UniqueName: \"kubernetes.io/projected/92fc69a3-1f38-415a-a26b-b54c06642dd3-kube-api-access-tgbpt\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:50:18.962997 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:18.962965 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9qx5\" (UniqueName: \"kubernetes.io/projected/d00f0578-e72c-4ace-891b-1f14acf2bdf2-kube-api-access-k9qx5\") pod \"test-curl-7346-139642131305648\" (UID: \"d00f0578-e72c-4ace-891b-1f14acf2bdf2\") " pod="opendatahub/test-curl-7346-139642131305648" Jul 15 21:50:19.063720 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:19.063677 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9qx5\" (UniqueName: \"kubernetes.io/projected/d00f0578-e72c-4ace-891b-1f14acf2bdf2-kube-api-access-k9qx5\") pod \"test-curl-7346-139642131305648\" (UID: \"d00f0578-e72c-4ace-891b-1f14acf2bdf2\") " pod="opendatahub/test-curl-7346-139642131305648" Jul 15 21:50:19.073444 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:19.073408 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9qx5\" (UniqueName: \"kubernetes.io/projected/d00f0578-e72c-4ace-891b-1f14acf2bdf2-kube-api-access-k9qx5\") pod \"test-curl-7346-139642131305648\" (UID: \"d00f0578-e72c-4ace-891b-1f14acf2bdf2\") " pod="opendatahub/test-curl-7346-139642131305648" Jul 15 21:50:19.146364 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:19.146326 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7346-139642131305648" Jul 15 21:50:19.283704 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:19.283668 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7346-139642131305648"] Jul 15 21:50:19.286501 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:50:19.286471 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd00f0578_e72c_4ace_891b_1f14acf2bdf2.slice/crio-94a32da4c6d57fbf7aa8b989a72ba920911425c7b2f40319b99c02d6e6de5aba WatchSource:0}: Error finding container 94a32da4c6d57fbf7aa8b989a72ba920911425c7b2f40319b99c02d6e6de5aba: Status 404 returned error can't find the container with id 94a32da4c6d57fbf7aa8b989a72ba920911425c7b2f40319b99c02d6e6de5aba Jul 15 21:50:19.539680 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:19.539590 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7346-139642131304640" Jul 15 21:50:19.539680 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:19.539615 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3640dfd5bedce17d02d8197bdf5a58f2962f9170854c656cc1f0de310eb6417e" Jul 15 21:50:19.540871 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:19.540846 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7346-139642131305648" event={"ID":"d00f0578-e72c-4ace-891b-1f14acf2bdf2","Type":"ContainerStarted","Data":"94a32da4c6d57fbf7aa8b989a72ba920911425c7b2f40319b99c02d6e6de5aba"} Jul 15 21:50:19.550640 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:50:19.550611 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7346-139642131304640\" is forbidden: User \"system:node:ip-10-0-137-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-185.ec2.internal' and this object" podUID="92fc69a3-1f38-415a-a26b-b54c06642dd3" pod="opendatahub/test-curl-7346-139642131304640" Jul 15 21:50:19.997153 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:19.997123 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92fc69a3-1f38-415a-a26b-b54c06642dd3" path="/var/lib/kubelet/pods/92fc69a3-1f38-415a-a26b-b54c06642dd3/volumes" Jul 15 21:50:20.547800 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:20.547765 2579 generic.go:356] "Generic (PLEG): container finished" podID="d00f0578-e72c-4ace-891b-1f14acf2bdf2" containerID="1114a0aca9a73eb77d9e561f7cf021f135f4b936782fe128872dd6aa8e78bfd7" exitCode=0 Jul 15 21:50:20.547950 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:20.547846 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7346-139642131305648" event={"ID":"d00f0578-e72c-4ace-891b-1f14acf2bdf2","Type":"ContainerDied","Data":"1114a0aca9a73eb77d9e561f7cf021f135f4b936782fe128872dd6aa8e78bfd7"} Jul 15 21:50:21.691885 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:21.691860 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7346-139642131305648" Jul 15 21:50:21.711236 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:21.711207 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7346-139642131305648_d00f0578-e72c-4ace-891b-1f14acf2bdf2/test-curl-7346-139642131305648/0.log" Jul 15 21:50:21.734639 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:21.734607 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7346-139642131305648"] Jul 15 21:50:21.737366 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:21.737339 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7346-139642131305648"] Jul 15 21:50:21.853855 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:21.853758 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7346-139642131305648"] Jul 15 21:50:21.854260 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:21.854234 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d00f0578-e72c-4ace-891b-1f14acf2bdf2" containerName="test-curl-7346-139642131305648" Jul 15 21:50:21.854375 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:21.854262 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d00f0578-e72c-4ace-891b-1f14acf2bdf2" containerName="test-curl-7346-139642131305648" Jul 15 21:50:21.854375 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:21.854363 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d00f0578-e72c-4ace-891b-1f14acf2bdf2" containerName="test-curl-7346-139642131305648" Jul 15 21:50:21.858521 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:21.858500 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7346-139642131305648" Jul 15 21:50:21.862521 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:21.862500 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7346-139642131305648"] Jul 15 21:50:21.889309 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:21.889283 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k9qx5\" (UniqueName: \"kubernetes.io/projected/d00f0578-e72c-4ace-891b-1f14acf2bdf2-kube-api-access-k9qx5\") pod \"d00f0578-e72c-4ace-891b-1f14acf2bdf2\" (UID: \"d00f0578-e72c-4ace-891b-1f14acf2bdf2\") " Jul 15 21:50:21.891366 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:21.891337 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d00f0578-e72c-4ace-891b-1f14acf2bdf2-kube-api-access-k9qx5" (OuterVolumeSpecName: "kube-api-access-k9qx5") pod "d00f0578-e72c-4ace-891b-1f14acf2bdf2" (UID: "d00f0578-e72c-4ace-891b-1f14acf2bdf2"). InnerVolumeSpecName "kube-api-access-k9qx5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:50:21.990891 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:21.990848 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65rch\" (UniqueName: \"kubernetes.io/projected/7d946864-862a-4cdf-9365-6e1a05f6bc30-kube-api-access-65rch\") pod \"test-curl-7346-139642131305648\" (UID: \"7d946864-862a-4cdf-9365-6e1a05f6bc30\") " pod="opendatahub/test-curl-7346-139642131305648" Jul 15 21:50:21.991045 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:21.991006 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k9qx5\" (UniqueName: \"kubernetes.io/projected/d00f0578-e72c-4ace-891b-1f14acf2bdf2-kube-api-access-k9qx5\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:50:21.997292 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:21.997263 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d00f0578-e72c-4ace-891b-1f14acf2bdf2" path="/var/lib/kubelet/pods/d00f0578-e72c-4ace-891b-1f14acf2bdf2/volumes" Jul 15 21:50:22.092309 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:22.092270 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65rch\" (UniqueName: \"kubernetes.io/projected/7d946864-862a-4cdf-9365-6e1a05f6bc30-kube-api-access-65rch\") pod \"test-curl-7346-139642131305648\" (UID: \"7d946864-862a-4cdf-9365-6e1a05f6bc30\") " pod="opendatahub/test-curl-7346-139642131305648" Jul 15 21:50:22.101205 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:22.101180 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-65rch\" (UniqueName: \"kubernetes.io/projected/7d946864-862a-4cdf-9365-6e1a05f6bc30-kube-api-access-65rch\") pod \"test-curl-7346-139642131305648\" (UID: \"7d946864-862a-4cdf-9365-6e1a05f6bc30\") " pod="opendatahub/test-curl-7346-139642131305648" Jul 15 21:50:22.168815 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:22.168778 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7346-139642131305648" Jul 15 21:50:22.304106 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:22.304079 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7346-139642131305648"] Jul 15 21:50:22.305350 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:50:22.305324 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7d946864_862a_4cdf_9365_6e1a05f6bc30.slice/crio-c0992ce59123fcdbf798e2f161410becd60a3c5d1530ed35e05edf7a8acb1f50 WatchSource:0}: Error finding container c0992ce59123fcdbf798e2f161410becd60a3c5d1530ed35e05edf7a8acb1f50: Status 404 returned error can't find the container with id c0992ce59123fcdbf798e2f161410becd60a3c5d1530ed35e05edf7a8acb1f50 Jul 15 21:50:22.566212 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:22.566134 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7346-139642131305648" Jul 15 21:50:22.566212 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:22.566145 2579 scope.go:117] "RemoveContainer" containerID="1114a0aca9a73eb77d9e561f7cf021f135f4b936782fe128872dd6aa8e78bfd7" Jul 15 21:50:22.567565 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:22.567534 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7346-139642131305648" event={"ID":"7d946864-862a-4cdf-9365-6e1a05f6bc30","Type":"ContainerStarted","Data":"c0992ce59123fcdbf798e2f161410becd60a3c5d1530ed35e05edf7a8acb1f50"} Jul 15 21:50:23.577362 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:23.577325 2579 generic.go:356] "Generic (PLEG): container finished" podID="7d946864-862a-4cdf-9365-6e1a05f6bc30" containerID="8957a242c20cfebba158c9e8ace7970ab51ee556c4632f5fc1916c0a46a34a12" exitCode=0 Jul 15 21:50:23.577767 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:23.577406 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7346-139642131305648" event={"ID":"7d946864-862a-4cdf-9365-6e1a05f6bc30","Type":"ContainerDied","Data":"8957a242c20cfebba158c9e8ace7970ab51ee556c4632f5fc1916c0a46a34a12"} Jul 15 21:50:24.725961 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:24.725937 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7346-139642131305648" Jul 15 21:50:24.747298 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:24.747267 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7346-139642131305648_7d946864-862a-4cdf-9365-6e1a05f6bc30/test-curl-7346-139642131305648/0.log" Jul 15 21:50:24.770839 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:24.770809 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7346-139642131305648"] Jul 15 21:50:24.774406 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:24.774381 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7346-139642131305648"] Jul 15 21:50:24.813261 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:24.813236 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-65rch\" (UniqueName: \"kubernetes.io/projected/7d946864-862a-4cdf-9365-6e1a05f6bc30-kube-api-access-65rch\") pod \"7d946864-862a-4cdf-9365-6e1a05f6bc30\" (UID: \"7d946864-862a-4cdf-9365-6e1a05f6bc30\") " Jul 15 21:50:24.815392 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:24.815358 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d946864-862a-4cdf-9365-6e1a05f6bc30-kube-api-access-65rch" (OuterVolumeSpecName: "kube-api-access-65rch") pod "7d946864-862a-4cdf-9365-6e1a05f6bc30" (UID: "7d946864-862a-4cdf-9365-6e1a05f6bc30"). InnerVolumeSpecName "kube-api-access-65rch". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:50:24.914478 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:24.914446 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-65rch\" (UniqueName: \"kubernetes.io/projected/7d946864-862a-4cdf-9365-6e1a05f6bc30-kube-api-access-65rch\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:50:25.487334 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:25.487296 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:50:25.587548 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:25.587479 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:50:25.591583 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:25.591560 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7346-139642131305648" Jul 15 21:50:25.591726 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:25.591567 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c0992ce59123fcdbf798e2f161410becd60a3c5d1530ed35e05edf7a8acb1f50" Jul 15 21:50:25.602144 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:50:25.602117 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7346-139642131305648\" is forbidden: User \"system:node:ip-10-0-137-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-185.ec2.internal' and this object" podUID="7d946864-862a-4cdf-9365-6e1a05f6bc30" pod="opendatahub/test-curl-7346-139642131305648" Jul 15 21:50:25.996532 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:50:25.996479 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7346-139642131305648\" is forbidden: User \"system:node:ip-10-0-137-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-185.ec2.internal' and this object" podUID="7d946864-862a-4cdf-9365-6e1a05f6bc30" pod="opendatahub/test-curl-7346-139642131305648" Jul 15 21:50:25.997671 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:25.997649 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d946864-862a-4cdf-9365-6e1a05f6bc30" path="/var/lib/kubelet/pods/7d946864-862a-4cdf-9365-6e1a05f6bc30/volumes" Jul 15 21:50:35.426745 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:35.426705 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:50:38.878994 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:38.878955 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:50:40.057059 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:40.057010 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:50:46.003144 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:46.003103 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:50:46.064479 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:46.064440 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:50:50.740996 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:50.740959 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g"] Jul 15 21:50:50.741409 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:50.741238 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g" podUID="f6d86286-9979-41b8-89cd-dea1057b2dff" containerName="istio-proxy" containerID="cri-o://3a0c7335b5753b7a0315a687263c620d5a8994d428ef259bd5ee4442ffecaeac" gracePeriod=30 Jul 15 21:50:50.756693 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:50.756656 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:50:51.010671 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:51.010591 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:50:51.664244 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:51.664199 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:50:51.763871 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:51.763837 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:50:55.999485 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:55.999459 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g" Jul 15 21:50:56.067095 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:56.066975 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6d86286-9979-41b8-89cd-dea1057b2dff-istio-podinfo\") pod \"f6d86286-9979-41b8-89cd-dea1057b2dff\" (UID: \"f6d86286-9979-41b8-89cd-dea1057b2dff\") " Jul 15 21:50:56.067095 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:56.067047 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6d86286-9979-41b8-89cd-dea1057b2dff-workload-socket\") pod \"f6d86286-9979-41b8-89cd-dea1057b2dff\" (UID: \"f6d86286-9979-41b8-89cd-dea1057b2dff\") " Jul 15 21:50:56.067349 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:56.067111 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6d86286-9979-41b8-89cd-dea1057b2dff-istio-token\") pod \"f6d86286-9979-41b8-89cd-dea1057b2dff\" (UID: \"f6d86286-9979-41b8-89cd-dea1057b2dff\") " Jul 15 21:50:56.067349 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:56.067145 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6d86286-9979-41b8-89cd-dea1057b2dff-credential-socket\") pod \"f6d86286-9979-41b8-89cd-dea1057b2dff\" (UID: \"f6d86286-9979-41b8-89cd-dea1057b2dff\") " Jul 15 21:50:56.067349 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:56.067172 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6d86286-9979-41b8-89cd-dea1057b2dff-workload-certs\") pod \"f6d86286-9979-41b8-89cd-dea1057b2dff\" (UID: \"f6d86286-9979-41b8-89cd-dea1057b2dff\") " Jul 15 21:50:56.067349 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:56.067198 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6d86286-9979-41b8-89cd-dea1057b2dff-istio-data\") pod \"f6d86286-9979-41b8-89cd-dea1057b2dff\" (UID: \"f6d86286-9979-41b8-89cd-dea1057b2dff\") " Jul 15 21:50:56.067349 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:56.067219 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6d86286-9979-41b8-89cd-dea1057b2dff-istio-envoy\") pod \"f6d86286-9979-41b8-89cd-dea1057b2dff\" (UID: \"f6d86286-9979-41b8-89cd-dea1057b2dff\") " Jul 15 21:50:56.067349 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:56.067264 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8qxm6\" (UniqueName: \"kubernetes.io/projected/f6d86286-9979-41b8-89cd-dea1057b2dff-kube-api-access-8qxm6\") pod \"f6d86286-9979-41b8-89cd-dea1057b2dff\" (UID: \"f6d86286-9979-41b8-89cd-dea1057b2dff\") " Jul 15 21:50:56.067349 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:56.067309 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6d86286-9979-41b8-89cd-dea1057b2dff-istiod-ca-cert\") pod \"f6d86286-9979-41b8-89cd-dea1057b2dff\" (UID: \"f6d86286-9979-41b8-89cd-dea1057b2dff\") " Jul 15 21:50:56.067709 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:56.067412 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6d86286-9979-41b8-89cd-dea1057b2dff-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f6d86286-9979-41b8-89cd-dea1057b2dff" (UID: "f6d86286-9979-41b8-89cd-dea1057b2dff"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:50:56.067709 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:56.067529 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6d86286-9979-41b8-89cd-dea1057b2dff-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f6d86286-9979-41b8-89cd-dea1057b2dff" (UID: "f6d86286-9979-41b8-89cd-dea1057b2dff"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:50:56.067709 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:56.067540 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6d86286-9979-41b8-89cd-dea1057b2dff-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f6d86286-9979-41b8-89cd-dea1057b2dff" (UID: "f6d86286-9979-41b8-89cd-dea1057b2dff"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:50:56.067709 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:56.067666 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6d86286-9979-41b8-89cd-dea1057b2dff-credential-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:50:56.067935 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:56.067715 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6d86286-9979-41b8-89cd-dea1057b2dff-workload-certs\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:50:56.067935 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:56.067732 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6d86286-9979-41b8-89cd-dea1057b2dff-workload-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:50:56.067935 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:56.067846 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6d86286-9979-41b8-89cd-dea1057b2dff-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f6d86286-9979-41b8-89cd-dea1057b2dff" (UID: "f6d86286-9979-41b8-89cd-dea1057b2dff"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 21:50:56.068288 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:56.068258 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6d86286-9979-41b8-89cd-dea1057b2dff-istio-data" (OuterVolumeSpecName: "istio-data") pod "f6d86286-9979-41b8-89cd-dea1057b2dff" (UID: "f6d86286-9979-41b8-89cd-dea1057b2dff"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:50:56.069495 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:56.069468 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f6d86286-9979-41b8-89cd-dea1057b2dff-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f6d86286-9979-41b8-89cd-dea1057b2dff" (UID: "f6d86286-9979-41b8-89cd-dea1057b2dff"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 21:50:56.069608 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:56.069493 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6d86286-9979-41b8-89cd-dea1057b2dff-istio-token" (OuterVolumeSpecName: "istio-token") pod "f6d86286-9979-41b8-89cd-dea1057b2dff" (UID: "f6d86286-9979-41b8-89cd-dea1057b2dff"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:50:56.069608 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:56.069591 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6d86286-9979-41b8-89cd-dea1057b2dff-kube-api-access-8qxm6" (OuterVolumeSpecName: "kube-api-access-8qxm6") pod "f6d86286-9979-41b8-89cd-dea1057b2dff" (UID: "f6d86286-9979-41b8-89cd-dea1057b2dff"). InnerVolumeSpecName "kube-api-access-8qxm6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:50:56.069686 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:56.069623 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6d86286-9979-41b8-89cd-dea1057b2dff-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f6d86286-9979-41b8-89cd-dea1057b2dff" (UID: "f6d86286-9979-41b8-89cd-dea1057b2dff"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:50:56.168634 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:56.168598 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6d86286-9979-41b8-89cd-dea1057b2dff-istio-token\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:50:56.168634 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:56.168626 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6d86286-9979-41b8-89cd-dea1057b2dff-istio-data\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:50:56.168634 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:56.168635 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6d86286-9979-41b8-89cd-dea1057b2dff-istio-envoy\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:50:56.168634 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:56.168644 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8qxm6\" (UniqueName: \"kubernetes.io/projected/f6d86286-9979-41b8-89cd-dea1057b2dff-kube-api-access-8qxm6\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:50:56.168902 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:56.168653 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6d86286-9979-41b8-89cd-dea1057b2dff-istiod-ca-cert\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:50:56.168902 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:56.168662 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6d86286-9979-41b8-89cd-dea1057b2dff-istio-podinfo\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:50:56.772376 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:56.772338 2579 generic.go:356] "Generic (PLEG): container finished" podID="f6d86286-9979-41b8-89cd-dea1057b2dff" containerID="3a0c7335b5753b7a0315a687263c620d5a8994d428ef259bd5ee4442ffecaeac" exitCode=0 Jul 15 21:50:56.772567 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:56.772395 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g" Jul 15 21:50:56.772567 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:56.772432 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g" event={"ID":"f6d86286-9979-41b8-89cd-dea1057b2dff","Type":"ContainerDied","Data":"3a0c7335b5753b7a0315a687263c620d5a8994d428ef259bd5ee4442ffecaeac"} Jul 15 21:50:56.772567 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:56.772496 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g" event={"ID":"f6d86286-9979-41b8-89cd-dea1057b2dff","Type":"ContainerDied","Data":"f7fc1255e07f106c1a6b326015696a861dd0a6c2dc90aba53404c803ee1fd062"} Jul 15 21:50:56.772567 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:56.772512 2579 scope.go:117] "RemoveContainer" containerID="3a0c7335b5753b7a0315a687263c620d5a8994d428ef259bd5ee4442ffecaeac" Jul 15 21:50:56.782878 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:56.782858 2579 scope.go:117] "RemoveContainer" containerID="7a707e09a8ffa1f0e6a33a90611635ba2342af5c4b8468541eba0e474581dfb5" Jul 15 21:50:56.791358 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:56.791337 2579 scope.go:117] "RemoveContainer" containerID="3a0c7335b5753b7a0315a687263c620d5a8994d428ef259bd5ee4442ffecaeac" Jul 15 21:50:56.791619 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:50:56.791600 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a0c7335b5753b7a0315a687263c620d5a8994d428ef259bd5ee4442ffecaeac\": container with ID starting with 3a0c7335b5753b7a0315a687263c620d5a8994d428ef259bd5ee4442ffecaeac not found: ID does not exist" containerID="3a0c7335b5753b7a0315a687263c620d5a8994d428ef259bd5ee4442ffecaeac" Jul 15 21:50:56.791692 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:56.791631 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a0c7335b5753b7a0315a687263c620d5a8994d428ef259bd5ee4442ffecaeac"} err="failed to get container status \"3a0c7335b5753b7a0315a687263c620d5a8994d428ef259bd5ee4442ffecaeac\": rpc error: code = NotFound desc = could not find container \"3a0c7335b5753b7a0315a687263c620d5a8994d428ef259bd5ee4442ffecaeac\": container with ID starting with 3a0c7335b5753b7a0315a687263c620d5a8994d428ef259bd5ee4442ffecaeac not found: ID does not exist" Jul 15 21:50:56.791692 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:56.791657 2579 scope.go:117] "RemoveContainer" containerID="7a707e09a8ffa1f0e6a33a90611635ba2342af5c4b8468541eba0e474581dfb5" Jul 15 21:50:56.791917 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:50:56.791898 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a707e09a8ffa1f0e6a33a90611635ba2342af5c4b8468541eba0e474581dfb5\": container with ID starting with 7a707e09a8ffa1f0e6a33a90611635ba2342af5c4b8468541eba0e474581dfb5 not found: ID does not exist" containerID="7a707e09a8ffa1f0e6a33a90611635ba2342af5c4b8468541eba0e474581dfb5" Jul 15 21:50:56.791964 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:56.791925 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a707e09a8ffa1f0e6a33a90611635ba2342af5c4b8468541eba0e474581dfb5"} err="failed to get container status \"7a707e09a8ffa1f0e6a33a90611635ba2342af5c4b8468541eba0e474581dfb5\": rpc error: code = NotFound desc = could not find container \"7a707e09a8ffa1f0e6a33a90611635ba2342af5c4b8468541eba0e474581dfb5\": container with ID starting with 7a707e09a8ffa1f0e6a33a90611635ba2342af5c4b8468541eba0e474581dfb5 not found: ID does not exist" Jul 15 21:50:56.796734 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:56.796712 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g"] Jul 15 21:50:56.800272 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:56.800251 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-31dfa2-openshift-default-774b4fb675-lb75g"] Jul 15 21:50:57.997825 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:50:57.997795 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6d86286-9979-41b8-89cd-dea1057b2dff" path="/var/lib/kubelet/pods/f6d86286-9979-41b8-89cd-dea1057b2dff/volumes" Jul 15 21:51:01.599126 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:01.599050 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:51:05.381845 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:05.381803 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nqhnc"] Jul 15 21:51:06.230084 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:06.230020 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:51:12.381459 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:12.381423 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:51:12.440011 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:12.439970 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:51:16.989124 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:16.989088 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b"] Jul 15 21:51:16.989633 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:16.989371 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b" podUID="8bac6a42-a26f-4a6e-85e3-f0fd79346a9d" containerName="istio-proxy" containerID="cri-o://89271f5af60c3c36c38b8db31e2e7a2b2dda0e9496307f117e5e07b239c4d9cc" gracePeriod=30 Jul 15 21:51:17.015507 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:17.015457 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:51:17.271252 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:17.271149 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:51:17.779773 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:17.779739 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-98759bf69-rll7s_57610c56-136e-473b-9609-fc51ce6729dc/authorino/0.log" Jul 15 21:51:21.209397 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:21.209359 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:51:21.307547 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:21.307506 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:51:21.836618 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:21.836572 2579 patch_prober.go:28] interesting pod/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 15 21:51:21.836789 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:21.836645 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b" podUID="8bac6a42-a26f-4a6e-85e3-f0fd79346a9d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 21:51:22.246183 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:22.246159 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b" Jul 15 21:51:22.402911 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:22.402814 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bac6a42-a26f-4a6e-85e3-f0fd79346a9d-credential-socket\") pod \"8bac6a42-a26f-4a6e-85e3-f0fd79346a9d\" (UID: \"8bac6a42-a26f-4a6e-85e3-f0fd79346a9d\") " Jul 15 21:51:22.402911 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:22.402876 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8bac6a42-a26f-4a6e-85e3-f0fd79346a9d-workload-certs\") pod \"8bac6a42-a26f-4a6e-85e3-f0fd79346a9d\" (UID: \"8bac6a42-a26f-4a6e-85e3-f0fd79346a9d\") " Jul 15 21:51:22.402911 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:22.402898 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8bac6a42-a26f-4a6e-85e3-f0fd79346a9d-istio-data\") pod \"8bac6a42-a26f-4a6e-85e3-f0fd79346a9d\" (UID: \"8bac6a42-a26f-4a6e-85e3-f0fd79346a9d\") " Jul 15 21:51:22.403235 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:22.402923 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8bac6a42-a26f-4a6e-85e3-f0fd79346a9d-istiod-ca-cert\") pod \"8bac6a42-a26f-4a6e-85e3-f0fd79346a9d\" (UID: \"8bac6a42-a26f-4a6e-85e3-f0fd79346a9d\") " Jul 15 21:51:22.403235 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:22.402957 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8bac6a42-a26f-4a6e-85e3-f0fd79346a9d-istio-envoy\") pod \"8bac6a42-a26f-4a6e-85e3-f0fd79346a9d\" (UID: \"8bac6a42-a26f-4a6e-85e3-f0fd79346a9d\") " Jul 15 21:51:22.403235 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:22.403000 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4twbs\" (UniqueName: \"kubernetes.io/projected/8bac6a42-a26f-4a6e-85e3-f0fd79346a9d-kube-api-access-4twbs\") pod \"8bac6a42-a26f-4a6e-85e3-f0fd79346a9d\" (UID: \"8bac6a42-a26f-4a6e-85e3-f0fd79346a9d\") " Jul 15 21:51:22.403235 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:22.403022 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bac6a42-a26f-4a6e-85e3-f0fd79346a9d-workload-socket\") pod \"8bac6a42-a26f-4a6e-85e3-f0fd79346a9d\" (UID: \"8bac6a42-a26f-4a6e-85e3-f0fd79346a9d\") " Jul 15 21:51:22.403235 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:22.403053 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8bac6a42-a26f-4a6e-85e3-f0fd79346a9d-istio-podinfo\") pod \"8bac6a42-a26f-4a6e-85e3-f0fd79346a9d\" (UID: \"8bac6a42-a26f-4a6e-85e3-f0fd79346a9d\") " Jul 15 21:51:22.403235 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:22.403098 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bac6a42-a26f-4a6e-85e3-f0fd79346a9d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8bac6a42-a26f-4a6e-85e3-f0fd79346a9d" (UID: "8bac6a42-a26f-4a6e-85e3-f0fd79346a9d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:51:22.403235 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:22.403117 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8bac6a42-a26f-4a6e-85e3-f0fd79346a9d-istio-token\") pod \"8bac6a42-a26f-4a6e-85e3-f0fd79346a9d\" (UID: \"8bac6a42-a26f-4a6e-85e3-f0fd79346a9d\") " Jul 15 21:51:22.403589 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:22.403370 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bac6a42-a26f-4a6e-85e3-f0fd79346a9d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8bac6a42-a26f-4a6e-85e3-f0fd79346a9d" (UID: "8bac6a42-a26f-4a6e-85e3-f0fd79346a9d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:51:22.403589 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:22.403400 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bac6a42-a26f-4a6e-85e3-f0fd79346a9d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8bac6a42-a26f-4a6e-85e3-f0fd79346a9d" (UID: "8bac6a42-a26f-4a6e-85e3-f0fd79346a9d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 21:51:22.403589 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:22.403582 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bac6a42-a26f-4a6e-85e3-f0fd79346a9d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8bac6a42-a26f-4a6e-85e3-f0fd79346a9d" (UID: "8bac6a42-a26f-4a6e-85e3-f0fd79346a9d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:51:22.403733 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:22.403634 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8bac6a42-a26f-4a6e-85e3-f0fd79346a9d-istiod-ca-cert\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:51:22.403733 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:22.403656 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bac6a42-a26f-4a6e-85e3-f0fd79346a9d-workload-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:51:22.403733 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:22.403671 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bac6a42-a26f-4a6e-85e3-f0fd79346a9d-credential-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:51:22.403861 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:22.403828 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bac6a42-a26f-4a6e-85e3-f0fd79346a9d-istio-data" (OuterVolumeSpecName: "istio-data") pod "8bac6a42-a26f-4a6e-85e3-f0fd79346a9d" (UID: "8bac6a42-a26f-4a6e-85e3-f0fd79346a9d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:51:22.405295 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:22.405265 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bac6a42-a26f-4a6e-85e3-f0fd79346a9d-istio-token" (OuterVolumeSpecName: "istio-token") pod "8bac6a42-a26f-4a6e-85e3-f0fd79346a9d" (UID: "8bac6a42-a26f-4a6e-85e3-f0fd79346a9d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:51:22.405425 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:22.405389 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bac6a42-a26f-4a6e-85e3-f0fd79346a9d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8bac6a42-a26f-4a6e-85e3-f0fd79346a9d" (UID: "8bac6a42-a26f-4a6e-85e3-f0fd79346a9d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:51:22.405566 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:22.405546 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bac6a42-a26f-4a6e-85e3-f0fd79346a9d-kube-api-access-4twbs" (OuterVolumeSpecName: "kube-api-access-4twbs") pod "8bac6a42-a26f-4a6e-85e3-f0fd79346a9d" (UID: "8bac6a42-a26f-4a6e-85e3-f0fd79346a9d"). InnerVolumeSpecName "kube-api-access-4twbs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:51:22.405635 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:22.405616 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8bac6a42-a26f-4a6e-85e3-f0fd79346a9d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8bac6a42-a26f-4a6e-85e3-f0fd79346a9d" (UID: "8bac6a42-a26f-4a6e-85e3-f0fd79346a9d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 21:51:22.504656 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:22.504610 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8bac6a42-a26f-4a6e-85e3-f0fd79346a9d-istio-podinfo\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:51:22.504656 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:22.504647 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8bac6a42-a26f-4a6e-85e3-f0fd79346a9d-istio-token\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:51:22.504656 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:22.504657 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8bac6a42-a26f-4a6e-85e3-f0fd79346a9d-workload-certs\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:51:22.504656 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:22.504665 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8bac6a42-a26f-4a6e-85e3-f0fd79346a9d-istio-data\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:51:22.504656 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:22.504672 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8bac6a42-a26f-4a6e-85e3-f0fd79346a9d-istio-envoy\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:51:22.504968 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:22.504681 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4twbs\" (UniqueName: \"kubernetes.io/projected/8bac6a42-a26f-4a6e-85e3-f0fd79346a9d-kube-api-access-4twbs\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 15 21:51:22.934714 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:22.934676 2579 generic.go:356] "Generic (PLEG): container finished" podID="8bac6a42-a26f-4a6e-85e3-f0fd79346a9d" containerID="89271f5af60c3c36c38b8db31e2e7a2b2dda0e9496307f117e5e07b239c4d9cc" exitCode=0 Jul 15 21:51:22.934922 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:22.934744 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b" Jul 15 21:51:22.934922 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:22.934764 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b" event={"ID":"8bac6a42-a26f-4a6e-85e3-f0fd79346a9d","Type":"ContainerDied","Data":"89271f5af60c3c36c38b8db31e2e7a2b2dda0e9496307f117e5e07b239c4d9cc"} Jul 15 21:51:22.934922 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:22.934858 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b" event={"ID":"8bac6a42-a26f-4a6e-85e3-f0fd79346a9d","Type":"ContainerDied","Data":"c57867ebbade6619b1583fcfdfc5176c20c23cd46424ed03d7a7def9134a5c51"} Jul 15 21:51:22.934922 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:22.934889 2579 scope.go:117] "RemoveContainer" containerID="89271f5af60c3c36c38b8db31e2e7a2b2dda0e9496307f117e5e07b239c4d9cc" Jul 15 21:51:22.944894 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:22.944875 2579 scope.go:117] "RemoveContainer" containerID="be30ec63d42443406f475eec59fc5c00ac14c33e0a3c4b4730ed6b936b280060" Jul 15 21:51:22.953607 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:22.953591 2579 scope.go:117] "RemoveContainer" containerID="89271f5af60c3c36c38b8db31e2e7a2b2dda0e9496307f117e5e07b239c4d9cc" Jul 15 21:51:22.953883 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:51:22.953864 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89271f5af60c3c36c38b8db31e2e7a2b2dda0e9496307f117e5e07b239c4d9cc\": container with ID starting with 89271f5af60c3c36c38b8db31e2e7a2b2dda0e9496307f117e5e07b239c4d9cc not found: ID does not exist" containerID="89271f5af60c3c36c38b8db31e2e7a2b2dda0e9496307f117e5e07b239c4d9cc" Jul 15 21:51:22.953923 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:22.953893 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89271f5af60c3c36c38b8db31e2e7a2b2dda0e9496307f117e5e07b239c4d9cc"} err="failed to get container status \"89271f5af60c3c36c38b8db31e2e7a2b2dda0e9496307f117e5e07b239c4d9cc\": rpc error: code = NotFound desc = could not find container \"89271f5af60c3c36c38b8db31e2e7a2b2dda0e9496307f117e5e07b239c4d9cc\": container with ID starting with 89271f5af60c3c36c38b8db31e2e7a2b2dda0e9496307f117e5e07b239c4d9cc not found: ID does not exist" Jul 15 21:51:22.953923 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:22.953912 2579 scope.go:117] "RemoveContainer" containerID="be30ec63d42443406f475eec59fc5c00ac14c33e0a3c4b4730ed6b936b280060" Jul 15 21:51:22.954184 ip-10-0-137-185 kubenswrapper[2579]: E0715 21:51:22.954164 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be30ec63d42443406f475eec59fc5c00ac14c33e0a3c4b4730ed6b936b280060\": container with ID starting with be30ec63d42443406f475eec59fc5c00ac14c33e0a3c4b4730ed6b936b280060 not found: ID does not exist" containerID="be30ec63d42443406f475eec59fc5c00ac14c33e0a3c4b4730ed6b936b280060" Jul 15 21:51:22.954260 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:22.954195 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be30ec63d42443406f475eec59fc5c00ac14c33e0a3c4b4730ed6b936b280060"} err="failed to get container status \"be30ec63d42443406f475eec59fc5c00ac14c33e0a3c4b4730ed6b936b280060\": rpc error: code = NotFound desc = could not find container \"be30ec63d42443406f475eec59fc5c00ac14c33e0a3c4b4730ed6b936b280060\": container with ID starting with be30ec63d42443406f475eec59fc5c00ac14c33e0a3c4b4730ed6b936b280060 not found: ID does not exist" Jul 15 21:51:22.957942 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:22.957923 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b"] Jul 15 21:51:22.959943 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:22.959925 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-b7f88c-openshift-default-7649cf89dc-2zx4b"] Jul 15 21:51:23.996521 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:23.996486 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8bac6a42-a26f-4a6e-85e3-f0fd79346a9d" path="/var/lib/kubelet/pods/8bac6a42-a26f-4a6e-85e3-f0fd79346a9d/volumes" Jul 15 21:51:24.074992 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:24.074953 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-gxtc2_46fc8345-bb02-4935-bfa9-227aba6b8077/manager/2.log" Jul 15 21:51:24.216579 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:24.216544 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-mcldd_ccbdd9b9-e1f6-4075-b80d-9ea7827580cf/manager/1.log" Jul 15 21:51:24.645893 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:24.645861 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-2mq85_c75adccd-fdbc-47a6-a744-c7020160ebb5/manager/1.log" Jul 15 21:51:25.077784 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:25.077698 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-65c67f598f-wb248_7193df13-36d0-4f6b-826b-4c876d9c2299/manager/1.log" Jul 15 21:51:25.089055 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:25.089030 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-65c67f598f-wb248_7193df13-36d0-4f6b-826b-4c876d9c2299/copy-manifests/0.log" Jul 15 21:51:28.514206 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:28.514171 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597plq7_cf45cf6f-0a81-470a-b31d-26d7e6591639/util/0.log" Jul 15 21:51:28.522624 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:28.522595 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597plq7_cf45cf6f-0a81-470a-b31d-26d7e6591639/pull/0.log" Jul 15 21:51:28.530264 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:28.530232 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597plq7_cf45cf6f-0a81-470a-b31d-26d7e6591639/extract/0.log" Jul 15 21:51:28.663837 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:28.663806 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tmgpz_a5f59b43-6fed-4fa8-b782-ed5457723590/util/0.log" Jul 15 21:51:28.671548 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:28.671523 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tmgpz_a5f59b43-6fed-4fa8-b782-ed5457723590/pull/0.log" Jul 15 21:51:28.678974 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:28.678946 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tmgpz_a5f59b43-6fed-4fa8-b782-ed5457723590/extract/0.log" Jul 15 21:51:28.811467 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:28.811391 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dp99w_a720aa3c-7268-4fbf-8fc9-ef8739108fdf/util/0.log" Jul 15 21:51:28.818503 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:28.818474 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dp99w_a720aa3c-7268-4fbf-8fc9-ef8739108fdf/pull/0.log" Jul 15 21:51:28.824753 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:28.824728 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dp99w_a720aa3c-7268-4fbf-8fc9-ef8739108fdf/extract/0.log" Jul 15 21:51:28.955027 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:28.954998 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zlc2_563f133b-0a69-4af8-b620-c7e5dcb1f38e/pull/0.log" Jul 15 21:51:28.961791 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:28.961768 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zlc2_563f133b-0a69-4af8-b620-c7e5dcb1f38e/extract/0.log" Jul 15 21:51:28.967455 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:28.967430 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zlc2_563f133b-0a69-4af8-b620-c7e5dcb1f38e/util/0.log" Jul 15 21:51:29.118209 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:29.118177 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-98759bf69-rll7s_57610c56-136e-473b-9609-fc51ce6729dc/authorino/0.log" Jul 15 21:51:29.261783 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:29.261753 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-7bzzs_cd1829fe-534c-44fc-b6cf-5761acfa46a0/manager/1.log" Jul 15 21:51:29.387812 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:29.387734 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-4cj4n_799804cd-f071-4c3f-8217-7a20956aeac7/manager/1.log" Jul 15 21:51:29.660232 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:29.660149 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-kbgvd_2152674f-5bf6-4df0-a6d7-5c3d4c6f01b9/registry-server/0.log" Jul 15 21:51:29.807535 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:29.807502 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-t8w9q_ee17d5c5-5655-460e-a148-0e4b4e95ee2a/manager/1.log" Jul 15 21:51:29.943776 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:29.943691 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-nqhnc_1a841697-9292-45f5-8fbb-3dfb012e0eff/limitador/0.log" Jul 15 21:51:30.087418 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:30.087393 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-fpp7b_86c51d13-9e24-46d2-af54-3002ca89dec3/manager/1.log" Jul 15 21:51:30.500131 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:30.500097 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4zskbp_0aa66e88-9447-4b63-8141-7f3bedff0ae5/istio-proxy/0.log" Jul 15 21:51:30.761644 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:30.761557 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6f6cc69c87-gvww8_1273fbd3-a61d-44b9-91cf-b796085ddffa/kube-auth-proxy/0.log" Jul 15 21:51:31.028947 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:31.028854 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-622mb_c9de6c58-b9b9-479a-b22a-57e8b98286be/istio-proxy/1.log" Jul 15 21:51:36.253164 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:36.253129 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m5r2m_0db09c1e-d24d-4519-93c8-c185018c3d24/ovn-acl-logging/0.log" Jul 15 21:51:36.257133 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:36.257110 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m5r2m_0db09c1e-d24d-4519-93c8-c185018c3d24/ovn-acl-logging/0.log" Jul 15 21:51:39.047769 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:39.047729 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-pqvxt/must-gather-jfdss"] Jul 15 21:51:39.048221 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:39.048096 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6d86286-9979-41b8-89cd-dea1057b2dff" containerName="istio-proxy" Jul 15 21:51:39.048221 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:39.048111 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6d86286-9979-41b8-89cd-dea1057b2dff" containerName="istio-proxy" Jul 15 21:51:39.048221 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:39.048122 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bac6a42-a26f-4a6e-85e3-f0fd79346a9d" containerName="istio-proxy" Jul 15 21:51:39.048221 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:39.048131 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bac6a42-a26f-4a6e-85e3-f0fd79346a9d" containerName="istio-proxy" Jul 15 21:51:39.048221 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:39.048149 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6d86286-9979-41b8-89cd-dea1057b2dff" containerName="istio-proxy" Jul 15 21:51:39.048221 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:39.048155 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6d86286-9979-41b8-89cd-dea1057b2dff" containerName="istio-proxy" Jul 15 21:51:39.048221 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:39.048162 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bac6a42-a26f-4a6e-85e3-f0fd79346a9d" containerName="istio-proxy" Jul 15 21:51:39.048221 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:39.048168 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bac6a42-a26f-4a6e-85e3-f0fd79346a9d" containerName="istio-proxy" Jul 15 21:51:39.048221 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:39.048179 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d946864-862a-4cdf-9365-6e1a05f6bc30" containerName="test-curl-7346-139642131305648" Jul 15 21:51:39.048221 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:39.048185 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d946864-862a-4cdf-9365-6e1a05f6bc30" containerName="test-curl-7346-139642131305648" Jul 15 21:51:39.048574 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:39.048247 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bac6a42-a26f-4a6e-85e3-f0fd79346a9d" containerName="istio-proxy" Jul 15 21:51:39.048574 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:39.048255 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d946864-862a-4cdf-9365-6e1a05f6bc30" containerName="test-curl-7346-139642131305648" Jul 15 21:51:39.048574 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:39.048264 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6d86286-9979-41b8-89cd-dea1057b2dff" containerName="istio-proxy" Jul 15 21:51:39.048574 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:39.048270 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6d86286-9979-41b8-89cd-dea1057b2dff" containerName="istio-proxy" Jul 15 21:51:39.048574 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:39.048373 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bac6a42-a26f-4a6e-85e3-f0fd79346a9d" containerName="istio-proxy" Jul 15 21:51:39.052921 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:39.052896 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pqvxt/must-gather-jfdss" Jul 15 21:51:39.055703 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:39.055677 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pqvxt\"/\"kube-root-ca.crt\"" Jul 15 21:51:39.055848 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:39.055677 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pqvxt\"/\"openshift-service-ca.crt\"" Jul 15 21:51:39.055848 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:39.055749 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-pqvxt\"/\"default-dockercfg-c2jrj\"" Jul 15 21:51:39.059792 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:39.059763 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pqvxt/must-gather-jfdss"] Jul 15 21:51:39.063841 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:39.063810 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:51:39.152139 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:39.152100 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/37e4c325-6492-4a21-97d4-98a1bea2440f-must-gather-output\") pod \"must-gather-jfdss\" (UID: \"37e4c325-6492-4a21-97d4-98a1bea2440f\") " pod="openshift-must-gather-pqvxt/must-gather-jfdss" Jul 15 21:51:39.152139 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:39.152146 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7npqr\" (UniqueName: \"kubernetes.io/projected/37e4c325-6492-4a21-97d4-98a1bea2440f-kube-api-access-7npqr\") pod \"must-gather-jfdss\" (UID: \"37e4c325-6492-4a21-97d4-98a1bea2440f\") " pod="openshift-must-gather-pqvxt/must-gather-jfdss" Jul 15 21:51:39.253542 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:39.253498 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/37e4c325-6492-4a21-97d4-98a1bea2440f-must-gather-output\") pod \"must-gather-jfdss\" (UID: \"37e4c325-6492-4a21-97d4-98a1bea2440f\") " pod="openshift-must-gather-pqvxt/must-gather-jfdss" Jul 15 21:51:39.253542 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:39.253550 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7npqr\" (UniqueName: \"kubernetes.io/projected/37e4c325-6492-4a21-97d4-98a1bea2440f-kube-api-access-7npqr\") pod \"must-gather-jfdss\" (UID: \"37e4c325-6492-4a21-97d4-98a1bea2440f\") " pod="openshift-must-gather-pqvxt/must-gather-jfdss" Jul 15 21:51:39.253855 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:39.253836 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/37e4c325-6492-4a21-97d4-98a1bea2440f-must-gather-output\") pod \"must-gather-jfdss\" (UID: \"37e4c325-6492-4a21-97d4-98a1bea2440f\") " pod="openshift-must-gather-pqvxt/must-gather-jfdss" Jul 15 21:51:39.262903 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:39.262875 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7npqr\" (UniqueName: \"kubernetes.io/projected/37e4c325-6492-4a21-97d4-98a1bea2440f-kube-api-access-7npqr\") pod \"must-gather-jfdss\" (UID: \"37e4c325-6492-4a21-97d4-98a1bea2440f\") " pod="openshift-must-gather-pqvxt/must-gather-jfdss" Jul 15 21:51:39.363603 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:39.363569 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pqvxt/must-gather-jfdss" Jul 15 21:51:39.499041 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:39.499018 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pqvxt/must-gather-jfdss"] Jul 15 21:51:39.501446 ip-10-0-137-185 kubenswrapper[2579]: W0715 21:51:39.501421 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod37e4c325_6492_4a21_97d4_98a1bea2440f.slice/crio-db044707e302f8f6b095ab748ad76ba5a8f016326e385b7b1a4d4fcd5f1048b6 WatchSource:0}: Error finding container db044707e302f8f6b095ab748ad76ba5a8f016326e385b7b1a4d4fcd5f1048b6: Status 404 returned error can't find the container with id db044707e302f8f6b095ab748ad76ba5a8f016326e385b7b1a4d4fcd5f1048b6 Jul 15 21:51:40.029023 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:40.028985 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pqvxt/must-gather-jfdss" event={"ID":"37e4c325-6492-4a21-97d4-98a1bea2440f","Type":"ContainerStarted","Data":"db044707e302f8f6b095ab748ad76ba5a8f016326e385b7b1a4d4fcd5f1048b6"} Jul 15 21:51:40.515371 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:40.515329 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" limit=1024 Jul 15 21:51:41.037490 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:41.037448 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pqvxt/must-gather-jfdss" event={"ID":"37e4c325-6492-4a21-97d4-98a1bea2440f","Type":"ContainerStarted","Data":"adf738084ac9a0983a8b429761f0edaf71ad951b2b09ef1f7ed01549386c87c6"} Jul 15 21:51:41.037672 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:41.037508 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pqvxt/must-gather-jfdss" event={"ID":"37e4c325-6492-4a21-97d4-98a1bea2440f","Type":"ContainerStarted","Data":"c98507773d32209daaa8a7ff28c30c8140e853b674d809fafa187e31f68d05a1"} Jul 15 21:51:42.030881 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:42.030846 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-4ctrp_3f26570b-5abd-4460-ba74-e59ba65032a1/global-pull-secret-syncer/0.log" Jul 15 21:51:42.169189 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:42.169159 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-g65nf_f7dba3ba-aec0-4137-b84e-4ce93d09c5a0/konnectivity-agent/0.log" Jul 15 21:51:42.302890 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:42.302804 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-185.ec2.internal_b338db47c4937c088ae739177bb426e7/haproxy/0.log" Jul 15 21:51:47.338691 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:47.338663 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597plq7_cf45cf6f-0a81-470a-b31d-26d7e6591639/extract/0.log" Jul 15 21:51:47.361831 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:47.361799 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597plq7_cf45cf6f-0a81-470a-b31d-26d7e6591639/util/0.log" Jul 15 21:51:47.383266 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:47.383232 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597plq7_cf45cf6f-0a81-470a-b31d-26d7e6591639/pull/0.log" Jul 15 21:51:47.415999 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:47.415949 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tmgpz_a5f59b43-6fed-4fa8-b782-ed5457723590/extract/0.log" Jul 15 21:51:47.438961 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:47.438933 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tmgpz_a5f59b43-6fed-4fa8-b782-ed5457723590/util/0.log" Jul 15 21:51:47.462809 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:47.462769 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tmgpz_a5f59b43-6fed-4fa8-b782-ed5457723590/pull/0.log" Jul 15 21:51:47.493700 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:47.493658 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dp99w_a720aa3c-7268-4fbf-8fc9-ef8739108fdf/extract/0.log" Jul 15 21:51:47.516563 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:47.516524 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dp99w_a720aa3c-7268-4fbf-8fc9-ef8739108fdf/util/0.log" Jul 15 21:51:47.544841 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:47.544809 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dp99w_a720aa3c-7268-4fbf-8fc9-ef8739108fdf/pull/0.log" Jul 15 21:51:47.572901 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:47.572866 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zlc2_563f133b-0a69-4af8-b620-c7e5dcb1f38e/extract/0.log" Jul 15 21:51:47.595025 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:47.594942 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zlc2_563f133b-0a69-4af8-b620-c7e5dcb1f38e/util/0.log" Jul 15 21:51:47.616821 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:47.616785 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zlc2_563f133b-0a69-4af8-b620-c7e5dcb1f38e/pull/0.log" Jul 15 21:51:48.201651 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:48.201618 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-98759bf69-rll7s_57610c56-136e-473b-9609-fc51ce6729dc/authorino/0.log" Jul 15 21:51:48.231045 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:48.231011 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-7bzzs_cd1829fe-534c-44fc-b6cf-5761acfa46a0/manager/1.log" Jul 15 21:51:48.236661 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:48.236441 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-7bzzs_cd1829fe-534c-44fc-b6cf-5761acfa46a0/manager/0.log" Jul 15 21:51:48.265198 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:48.265167 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-4cj4n_799804cd-f071-4c3f-8217-7a20956aeac7/manager/1.log" Jul 15 21:51:48.265894 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:48.265862 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-4cj4n_799804cd-f071-4c3f-8217-7a20956aeac7/manager/0.log" Jul 15 21:51:48.360795 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:48.360756 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-kbgvd_2152674f-5bf6-4df0-a6d7-5c3d4c6f01b9/registry-server/0.log" Jul 15 21:51:48.447872 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:48.447833 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-t8w9q_ee17d5c5-5655-460e-a148-0e4b4e95ee2a/manager/0.log" Jul 15 21:51:48.568139 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:48.567932 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-t8w9q_ee17d5c5-5655-460e-a148-0e4b4e95ee2a/manager/1.log" Jul 15 21:51:48.606993 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:48.606965 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-nqhnc_1a841697-9292-45f5-8fbb-3dfb012e0eff/limitador/0.log" Jul 15 21:51:48.711877 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:48.711840 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-fpp7b_86c51d13-9e24-46d2-af54-3002ca89dec3/manager/0.log" Jul 15 21:51:48.721830 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:48.721793 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-fpp7b_86c51d13-9e24-46d2-af54-3002ca89dec3/manager/1.log" Jul 15 21:51:50.672823 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:50.672792 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5ngtg_9331cdf3-604a-475b-a036-afe088bdea64/node-exporter/0.log" Jul 15 21:51:50.701266 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:50.701225 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5ngtg_9331cdf3-604a-475b-a036-afe088bdea64/kube-rbac-proxy/0.log" Jul 15 21:51:50.725725 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:50.725688 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5ngtg_9331cdf3-604a-475b-a036-afe088bdea64/init-textfile/0.log" Jul 15 21:51:51.213774 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:51.213737 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5494675598-c64bz_111ca5e6-b48c-44ea-a3ae-52f1a9572b11/telemeter-client/0.log" Jul 15 21:51:51.234825 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:51.234786 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5494675598-c64bz_111ca5e6-b48c-44ea-a3ae-52f1a9572b11/reload/0.log" Jul 15 21:51:51.256017 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:51.255976 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5494675598-c64bz_111ca5e6-b48c-44ea-a3ae-52f1a9572b11/kube-rbac-proxy/0.log" Jul 15 21:51:52.356832 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:52.356760 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-pqvxt/must-gather-jfdss" podStartSLOduration=12.597811112 podStartE2EDuration="13.356734923s" totalImagesPullingTime="758.923811ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:51:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:51:41.056956641 +0000 UTC m=+3005.598269191" watchObservedRunningTime="2026-07-15 21:51:52.356734923 +0000 UTC m=+3016.898047476" Jul 15 21:51:52.357613 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:52.357583 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-pqvxt/perf-node-gather-daemonset-5fnbq"] Jul 15 21:51:52.361677 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:52.361655 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pqvxt/perf-node-gather-daemonset-5fnbq" Jul 15 21:51:52.370458 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:52.370435 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pqvxt/perf-node-gather-daemonset-5fnbq"] Jul 15 21:51:52.479024 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:52.478977 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/00cc2492-5360-4a05-b4cc-0f50ccad3b19-lib-modules\") pod \"perf-node-gather-daemonset-5fnbq\" (UID: \"00cc2492-5360-4a05-b4cc-0f50ccad3b19\") " pod="openshift-must-gather-pqvxt/perf-node-gather-daemonset-5fnbq" Jul 15 21:51:52.479242 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:52.479052 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/00cc2492-5360-4a05-b4cc-0f50ccad3b19-podres\") pod \"perf-node-gather-daemonset-5fnbq\" (UID: \"00cc2492-5360-4a05-b4cc-0f50ccad3b19\") " pod="openshift-must-gather-pqvxt/perf-node-gather-daemonset-5fnbq" Jul 15 21:51:52.479242 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:52.479159 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/00cc2492-5360-4a05-b4cc-0f50ccad3b19-proc\") pod \"perf-node-gather-daemonset-5fnbq\" (UID: \"00cc2492-5360-4a05-b4cc-0f50ccad3b19\") " pod="openshift-must-gather-pqvxt/perf-node-gather-daemonset-5fnbq" Jul 15 21:51:52.479242 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:52.479186 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/00cc2492-5360-4a05-b4cc-0f50ccad3b19-sys\") pod \"perf-node-gather-daemonset-5fnbq\" (UID: \"00cc2492-5360-4a05-b4cc-0f50ccad3b19\") " pod="openshift-must-gather-pqvxt/perf-node-gather-daemonset-5fnbq" Jul 15 21:51:52.479242 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:52.479222 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nd5h\" (UniqueName: \"kubernetes.io/projected/00cc2492-5360-4a05-b4cc-0f50ccad3b19-kube-api-access-6nd5h\") pod \"perf-node-gather-daemonset-5fnbq\" (UID: \"00cc2492-5360-4a05-b4cc-0f50ccad3b19\") " pod="openshift-must-gather-pqvxt/perf-node-gather-daemonset-5fnbq" Jul 15 21:51:52.579876 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:52.579831 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/00cc2492-5360-4a05-b4cc-0f50ccad3b19-proc\") pod \"perf-node-gather-daemonset-5fnbq\" (UID: \"00cc2492-5360-4a05-b4cc-0f50ccad3b19\") " pod="openshift-must-gather-pqvxt/perf-node-gather-daemonset-5fnbq" Jul 15 21:51:52.580094 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:52.579885 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/00cc2492-5360-4a05-b4cc-0f50ccad3b19-sys\") pod \"perf-node-gather-daemonset-5fnbq\" (UID: \"00cc2492-5360-4a05-b4cc-0f50ccad3b19\") " pod="openshift-must-gather-pqvxt/perf-node-gather-daemonset-5fnbq" Jul 15 21:51:52.580094 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:52.579920 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nd5h\" (UniqueName: \"kubernetes.io/projected/00cc2492-5360-4a05-b4cc-0f50ccad3b19-kube-api-access-6nd5h\") pod \"perf-node-gather-daemonset-5fnbq\" (UID: \"00cc2492-5360-4a05-b4cc-0f50ccad3b19\") " pod="openshift-must-gather-pqvxt/perf-node-gather-daemonset-5fnbq" Jul 15 21:51:52.580094 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:52.579985 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/00cc2492-5360-4a05-b4cc-0f50ccad3b19-lib-modules\") pod \"perf-node-gather-daemonset-5fnbq\" (UID: \"00cc2492-5360-4a05-b4cc-0f50ccad3b19\") " pod="openshift-must-gather-pqvxt/perf-node-gather-daemonset-5fnbq" Jul 15 21:51:52.580094 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:52.580027 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/00cc2492-5360-4a05-b4cc-0f50ccad3b19-podres\") pod \"perf-node-gather-daemonset-5fnbq\" (UID: \"00cc2492-5360-4a05-b4cc-0f50ccad3b19\") " pod="openshift-must-gather-pqvxt/perf-node-gather-daemonset-5fnbq" Jul 15 21:51:52.580315 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:52.580221 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/00cc2492-5360-4a05-b4cc-0f50ccad3b19-podres\") pod \"perf-node-gather-daemonset-5fnbq\" (UID: \"00cc2492-5360-4a05-b4cc-0f50ccad3b19\") " pod="openshift-must-gather-pqvxt/perf-node-gather-daemonset-5fnbq" Jul 15 21:51:52.580315 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:52.580282 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/00cc2492-5360-4a05-b4cc-0f50ccad3b19-proc\") pod \"perf-node-gather-daemonset-5fnbq\" (UID: \"00cc2492-5360-4a05-b4cc-0f50ccad3b19\") " pod="openshift-must-gather-pqvxt/perf-node-gather-daemonset-5fnbq" Jul 15 21:51:52.580421 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:52.580328 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/00cc2492-5360-4a05-b4cc-0f50ccad3b19-sys\") pod \"perf-node-gather-daemonset-5fnbq\" (UID: \"00cc2492-5360-4a05-b4cc-0f50ccad3b19\") " pod="openshift-must-gather-pqvxt/perf-node-gather-daemonset-5fnbq" Jul 15 21:51:52.580730 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:52.580706 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/00cc2492-5360-4a05-b4cc-0f50ccad3b19-lib-modules\") pod \"perf-node-gather-daemonset-5fnbq\" (UID: \"00cc2492-5360-4a05-b4cc-0f50ccad3b19\") " pod="openshift-must-gather-pqvxt/perf-node-gather-daemonset-5fnbq" Jul 15 21:51:52.589095 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:52.589032 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nd5h\" (UniqueName: \"kubernetes.io/projected/00cc2492-5360-4a05-b4cc-0f50ccad3b19-kube-api-access-6nd5h\") pod \"perf-node-gather-daemonset-5fnbq\" (UID: \"00cc2492-5360-4a05-b4cc-0f50ccad3b19\") " pod="openshift-must-gather-pqvxt/perf-node-gather-daemonset-5fnbq" Jul 15 21:51:52.679292 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:52.679207 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pqvxt/perf-node-gather-daemonset-5fnbq" Jul 15 21:51:52.853263 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:52.852496 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pqvxt/perf-node-gather-daemonset-5fnbq"] Jul 15 21:51:53.146172 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:53.146128 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pqvxt/perf-node-gather-daemonset-5fnbq" event={"ID":"00cc2492-5360-4a05-b4cc-0f50ccad3b19","Type":"ContainerStarted","Data":"a55032a92b8a952a0397d9ee2b80d5d27948917f32549eebc997158301f9814a"} Jul 15 21:51:53.146384 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:53.146205 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-pqvxt/perf-node-gather-daemonset-5fnbq" Jul 15 21:51:53.146384 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:53.146225 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pqvxt/perf-node-gather-daemonset-5fnbq" event={"ID":"00cc2492-5360-4a05-b4cc-0f50ccad3b19","Type":"ContainerStarted","Data":"a20d3bd0add1e5ceefbb779ce80bb80a15fa37e24b81e6a29ebc963a6d35120f"} Jul 15 21:51:53.164005 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:53.163956 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-pqvxt/perf-node-gather-daemonset-5fnbq" podStartSLOduration=1.1639388720000001 podStartE2EDuration="1.163938872s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 21:51:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:51:53.161630065 +0000 UTC m=+3017.702942615" watchObservedRunningTime="2026-07-15 21:51:53.163938872 +0000 UTC m=+3017.705251422" Jul 15 21:51:55.369319 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:55.369284 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nhh6c_2511b73d-3502-447a-9175-60d60bd9b862/dns/0.log" Jul 15 21:51:55.387891 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:55.387860 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nhh6c_2511b73d-3502-447a-9175-60d60bd9b862/kube-rbac-proxy/0.log" Jul 15 21:51:55.431948 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:55.431913 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jw6xs_05324e48-e436-4e6a-9b35-5d0c661a82a4/dns-node-resolver/0.log" Jul 15 21:51:56.091321 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:56.091288 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-w2rr5_e24afff0-0ef0-4463-92fc-848462106a1b/node-ca/0.log" Jul 15 21:51:57.077676 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:57.077622 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4zskbp_0aa66e88-9447-4b63-8141-7f3bedff0ae5/istio-proxy/0.log" Jul 15 21:51:57.221302 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:57.221275 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6f6cc69c87-gvww8_1273fbd3-a61d-44b9-91cf-b796085ddffa/kube-auth-proxy/0.log" Jul 15 21:51:57.275167 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:57.275132 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-622mb_c9de6c58-b9b9-479a-b22a-57e8b98286be/istio-proxy/0.log" Jul 15 21:51:57.338793 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:57.338710 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-622mb_c9de6c58-b9b9-479a-b22a-57e8b98286be/istio-proxy/1.log" Jul 15 21:51:58.111906 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:58.111869 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-9mhlt_44d76b49-7290-4217-b4b3-2ec82286aa5c/serve-healthcheck-canary/0.log" Jul 15 21:51:58.745291 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:58.745261 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mmxvl_50d5a72a-1db9-492b-ab5c-21c12a0efca2/kube-rbac-proxy/0.log" Jul 15 21:51:58.766604 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:58.766575 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mmxvl_50d5a72a-1db9-492b-ab5c-21c12a0efca2/exporter/0.log" Jul 15 21:51:58.787913 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:58.787879 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mmxvl_50d5a72a-1db9-492b-ab5c-21c12a0efca2/extractor/0.log" Jul 15 21:51:59.163879 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:51:59.163853 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-pqvxt/perf-node-gather-daemonset-5fnbq" Jul 15 21:52:01.520151 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:52:01.520014 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-gxtc2_46fc8345-bb02-4935-bfa9-227aba6b8077/manager/1.log" Jul 15 21:52:01.520151 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:52:01.520045 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-gxtc2_46fc8345-bb02-4935-bfa9-227aba6b8077/manager/2.log" Jul 15 21:52:01.669525 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:52:01.669488 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-mcldd_ccbdd9b9-e1f6-4075-b80d-9ea7827580cf/manager/0.log" Jul 15 21:52:01.794390 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:52:01.794303 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-mcldd_ccbdd9b9-e1f6-4075-b80d-9ea7827580cf/manager/1.log" Jul 15 21:52:02.216540 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:52:02.216494 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-2mq85_c75adccd-fdbc-47a6-a744-c7020160ebb5/manager/0.log" Jul 15 21:52:02.219332 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:52:02.219303 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-2mq85_c75adccd-fdbc-47a6-a744-c7020160ebb5/manager/1.log" Jul 15 21:52:02.510057 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:52:02.509939 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-65c67f598f-wb248_7193df13-36d0-4f6b-826b-4c876d9c2299/manager/1.log" Jul 15 21:52:02.567043 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:52:02.566982 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-65c67f598f-wb248_7193df13-36d0-4f6b-826b-4c876d9c2299/manager/0.log" Jul 15 21:52:02.600900 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:52:02.600863 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-65c67f598f-wb248_7193df13-36d0-4f6b-826b-4c876d9c2299/copy-manifests/0.log" Jul 15 21:52:02.892086 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:52:02.892021 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4zskbp_0aa66e88-9447-4b63-8141-7f3bedff0ae5/istio-proxy/0.log" Jul 15 21:52:03.034098 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:52:03.034045 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6f6cc69c87-gvww8_1273fbd3-a61d-44b9-91cf-b796085ddffa/kube-auth-proxy/0.log" Jul 15 21:52:03.085685 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:52:03.085655 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-622mb_c9de6c58-b9b9-479a-b22a-57e8b98286be/istio-proxy/0.log" Jul 15 21:52:03.149251 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:52:03.149097 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-622mb_c9de6c58-b9b9-479a-b22a-57e8b98286be/istio-proxy/1.log" Jul 15 21:52:04.030783 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:52:04.030751 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-8mrng_f91d181a-4185-4d11-acbd-d91ae771a9d1/openshift-lws-operator/0.log" Jul 15 21:52:04.444942 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:52:04.444912 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4zskbp_0aa66e88-9447-4b63-8141-7f3bedff0ae5/istio-proxy/0.log" Jul 15 21:52:04.585644 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:52:04.585609 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6f6cc69c87-gvww8_1273fbd3-a61d-44b9-91cf-b796085ddffa/kube-auth-proxy/0.log" Jul 15 21:52:04.634476 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:52:04.634447 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-622mb_c9de6c58-b9b9-479a-b22a-57e8b98286be/istio-proxy/0.log" Jul 15 21:52:04.701898 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:52:04.701803 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-622mb_c9de6c58-b9b9-479a-b22a-57e8b98286be/istio-proxy/1.log" Jul 15 21:52:08.927702 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:52:08.927672 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-974pc_92770f0f-ea66-4ff1-906b-32fff3b4447f/migrator/0.log" Jul 15 21:52:08.949475 ip-10-0-137-185 kubenswrapper[2579]: I0715 21:52:08.949449 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-974pc_92770f0f-ea66-4ff1-906b-32fff3b4447f/graceful-termination/0.log"