Jul 05 13:16:40.148742 ip-10-0-139-155 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 05 13:16:40.148759 ip-10-0-139-155 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 05 13:16:40.148769 ip-10-0-139-155 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 05 13:16:40.149109 ip-10-0-139-155 systemd[1]: Failed to start Kubernetes Kubelet. Jul 05 13:16:50.277748 ip-10-0-139-155 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 05 13:16:50.277767 ip-10-0-139-155 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 924a9b1daea94e499fc2cd5ef412b4c8 -- Jul 05 13:18:45.781200 ip-10-0-139-155 systemd[1]: Starting Kubernetes Kubelet... Jul 05 13:18:46.252553 ip-10-0-139-155 kubenswrapper[2573]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 05 13:18:46.252553 ip-10-0-139-155 kubenswrapper[2573]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 05 13:18:46.252553 ip-10-0-139-155 kubenswrapper[2573]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 05 13:18:46.252553 ip-10-0-139-155 kubenswrapper[2573]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 05 13:18:46.252553 ip-10-0-139-155 kubenswrapper[2573]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 05 13:18:46.323324 ip-10-0-139-155 systemd[1]: Started Kubernetes Kubelet. Jul 05 13:18:46.347138 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.253961 2573 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 05 13:18:46.347138 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259716 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 05 13:18:46.347138 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259728 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 05 13:18:46.347138 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259732 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 05 13:18:46.347138 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259735 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 05 13:18:46.347138 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259738 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 05 13:18:46.347138 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259743 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 05 13:18:46.347138 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259746 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 05 13:18:46.347138 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259748 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 05 13:18:46.347138 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259751 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 05 13:18:46.347138 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259755 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 05 13:18:46.347138 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259760 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 05 13:18:46.347138 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259764 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 05 13:18:46.347138 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259767 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 05 13:18:46.347138 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259770 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 05 13:18:46.347138 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259773 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 05 13:18:46.347138 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259776 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 05 13:18:46.347138 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259780 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 05 13:18:46.348314 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259783 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 05 13:18:46.348314 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259786 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 05 13:18:46.348314 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259788 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 05 13:18:46.348314 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259791 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 05 13:18:46.348314 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259793 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 05 13:18:46.348314 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259796 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 05 13:18:46.348314 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259799 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 05 13:18:46.348314 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259801 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 05 13:18:46.348314 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259804 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 05 13:18:46.348314 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259806 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 05 13:18:46.348314 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259809 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 05 13:18:46.348314 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259819 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 05 13:18:46.348314 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259822 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 05 13:18:46.348314 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259824 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 05 13:18:46.348314 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259827 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 05 13:18:46.348314 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259830 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 05 13:18:46.348314 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259832 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 05 13:18:46.348314 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259835 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 05 13:18:46.348314 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259838 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 05 13:18:46.348314 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259841 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 05 13:18:46.349639 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259844 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 05 13:18:46.349639 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259846 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 05 13:18:46.349639 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259849 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 05 13:18:46.349639 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259851 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 05 13:18:46.349639 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259854 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 05 13:18:46.349639 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259857 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 05 13:18:46.349639 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259859 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 05 13:18:46.349639 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259862 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 05 13:18:46.349639 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259865 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 05 13:18:46.349639 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259867 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 05 13:18:46.349639 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259871 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 05 13:18:46.349639 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259874 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 05 13:18:46.349639 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259877 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 05 13:18:46.349639 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259880 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 05 13:18:46.349639 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259883 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 05 13:18:46.349639 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259885 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 05 13:18:46.349639 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259888 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 05 13:18:46.349639 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259891 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 05 13:18:46.349639 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259893 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 05 13:18:46.349639 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259896 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 05 13:18:46.350680 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259900 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 05 13:18:46.350680 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259902 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 05 13:18:46.350680 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259905 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 05 13:18:46.350680 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259908 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 05 13:18:46.350680 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259911 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 05 13:18:46.350680 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259913 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 05 13:18:46.350680 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259930 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 05 13:18:46.350680 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259933 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 05 13:18:46.350680 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259936 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 05 13:18:46.350680 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259938 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 05 13:18:46.350680 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259941 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 05 13:18:46.350680 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259945 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 05 13:18:46.350680 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259948 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 05 13:18:46.350680 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259951 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 05 13:18:46.350680 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259954 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 05 13:18:46.350680 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259957 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 05 13:18:46.350680 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259960 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 05 13:18:46.350680 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259963 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 05 13:18:46.350680 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259965 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 05 13:18:46.351521 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259968 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 05 13:18:46.351521 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259971 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 05 13:18:46.351521 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259974 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 05 13:18:46.351521 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259977 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 05 13:18:46.351521 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259981 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 05 13:18:46.351521 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259983 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 05 13:18:46.351521 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259986 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 05 13:18:46.351521 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259989 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 05 13:18:46.351521 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259991 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 05 13:18:46.351521 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259994 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 05 13:18:46.351521 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.259997 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 05 13:18:46.351521 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260001 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 05 13:18:46.351521 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260004 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 05 13:18:46.351521 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260006 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 05 13:18:46.351521 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260009 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 05 13:18:46.351521 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260013 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 05 13:18:46.351521 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260017 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 05 13:18:46.351521 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260020 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 05 13:18:46.351521 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260023 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 05 13:18:46.351521 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260025 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 05 13:18:46.352293 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260028 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 05 13:18:46.352293 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260030 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 05 13:18:46.352293 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260033 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 05 13:18:46.352293 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260748 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 05 13:18:46.352293 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260755 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 05 13:18:46.352293 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260759 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 05 13:18:46.352293 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260762 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 05 13:18:46.352293 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260766 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 05 13:18:46.352293 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260769 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 05 13:18:46.352293 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260772 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 05 13:18:46.352293 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260775 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 05 13:18:46.352293 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260777 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 05 13:18:46.352293 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260780 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 05 13:18:46.352293 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260783 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 05 13:18:46.352293 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260786 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 05 13:18:46.352293 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260788 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 05 13:18:46.352293 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260791 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 05 13:18:46.352293 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260794 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 05 13:18:46.352293 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260797 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 05 13:18:46.352293 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260799 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 05 13:18:46.352891 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260802 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 05 13:18:46.352891 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260804 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 05 13:18:46.352891 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260807 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 05 13:18:46.352891 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260809 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 05 13:18:46.352891 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260813 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 05 13:18:46.352891 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260815 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 05 13:18:46.352891 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260818 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 05 13:18:46.352891 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260820 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 05 13:18:46.352891 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260823 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 05 13:18:46.352891 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260825 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 05 13:18:46.352891 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260828 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 05 13:18:46.352891 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260831 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 05 13:18:46.352891 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260834 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 05 13:18:46.352891 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260837 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 05 13:18:46.352891 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260839 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 05 13:18:46.352891 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260842 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 05 13:18:46.352891 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260845 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 05 13:18:46.352891 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260847 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 05 13:18:46.352891 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260850 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 05 13:18:46.352891 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260852 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 05 13:18:46.353526 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260855 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 05 13:18:46.353526 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260858 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 05 13:18:46.353526 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260861 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 05 13:18:46.353526 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260863 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 05 13:18:46.353526 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260867 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 05 13:18:46.353526 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260870 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 05 13:18:46.353526 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260872 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 05 13:18:46.353526 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260875 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 05 13:18:46.353526 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260877 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 05 13:18:46.353526 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260880 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 05 13:18:46.353526 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260883 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 05 13:18:46.353526 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260885 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 05 13:18:46.353526 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260888 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 05 13:18:46.353526 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260890 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 05 13:18:46.353526 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260893 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 05 13:18:46.353526 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260895 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 05 13:18:46.353526 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260899 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 05 13:18:46.353526 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260902 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 05 13:18:46.353526 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260905 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 05 13:18:46.353526 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260908 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 05 13:18:46.354167 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260911 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 05 13:18:46.354167 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260913 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 05 13:18:46.354167 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260935 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 05 13:18:46.354167 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260939 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 05 13:18:46.354167 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260942 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 05 13:18:46.354167 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260945 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 05 13:18:46.354167 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260948 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 05 13:18:46.354167 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260951 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 05 13:18:46.354167 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260954 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 05 13:18:46.354167 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260956 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 05 13:18:46.354167 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260961 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 05 13:18:46.354167 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260964 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 05 13:18:46.354167 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260968 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 05 13:18:46.354167 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260971 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 05 13:18:46.354167 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260974 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 05 13:18:46.354167 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260977 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 05 13:18:46.354167 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260980 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 05 13:18:46.354167 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260983 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 05 13:18:46.354167 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260986 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 05 13:18:46.354167 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260989 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 05 13:18:46.354795 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260995 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 05 13:18:46.354795 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.260997 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 05 13:18:46.354795 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261000 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 05 13:18:46.354795 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261003 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 05 13:18:46.354795 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261005 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 05 13:18:46.354795 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261008 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 05 13:18:46.354795 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261010 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 05 13:18:46.354795 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261013 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 05 13:18:46.354795 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261016 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 05 13:18:46.354795 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261025 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 05 13:18:46.354795 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261028 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 05 13:18:46.354795 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261031 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 05 13:18:46.354795 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261034 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 05 13:18:46.354795 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261037 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 05 13:18:46.354795 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261039 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 05 13:18:46.354795 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261042 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 05 13:18:46.354795 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261046 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 05 13:18:46.354795 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261049 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 05 13:18:46.354795 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261052 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 05 13:18:46.355427 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261054 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 05 13:18:46.355427 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261058 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 05 13:18:46.355427 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261062 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 05 13:18:46.355427 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261470 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 05 13:18:46.355427 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261475 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 05 13:18:46.355427 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261479 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 05 13:18:46.355427 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261482 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 05 13:18:46.355427 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261485 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 05 13:18:46.355427 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261488 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 05 13:18:46.355427 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261490 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 05 13:18:46.355427 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261493 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 05 13:18:46.355427 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261496 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 05 13:18:46.355427 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261498 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 05 13:18:46.355427 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261503 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 05 13:18:46.355427 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261505 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 05 13:18:46.355427 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261508 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 05 13:18:46.355427 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261511 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 05 13:18:46.355427 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261513 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 05 13:18:46.355427 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261516 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 05 13:18:46.356261 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261519 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 05 13:18:46.356261 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261521 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 05 13:18:46.356261 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261524 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 05 13:18:46.356261 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261526 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 05 13:18:46.356261 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261529 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 05 13:18:46.356261 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261532 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 05 13:18:46.356261 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261534 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 05 13:18:46.356261 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261536 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 05 13:18:46.356261 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261539 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 05 13:18:46.356261 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261541 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 05 13:18:46.356261 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261550 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 05 13:18:46.356261 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261552 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 05 13:18:46.356261 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261555 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 05 13:18:46.356261 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261558 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 05 13:18:46.356261 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261560 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 05 13:18:46.356261 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261563 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 05 13:18:46.356261 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261567 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 05 13:18:46.356261 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261570 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 05 13:18:46.356261 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261573 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 05 13:18:46.356261 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261575 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 05 13:18:46.356870 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261578 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 05 13:18:46.356870 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261580 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 05 13:18:46.356870 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261583 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 05 13:18:46.356870 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261585 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 05 13:18:46.356870 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261588 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 05 13:18:46.356870 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261590 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 05 13:18:46.356870 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261595 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 05 13:18:46.356870 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261598 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 05 13:18:46.356870 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261601 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 05 13:18:46.356870 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261604 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 05 13:18:46.356870 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261606 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 05 13:18:46.356870 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261609 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 05 13:18:46.356870 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261611 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 05 13:18:46.356870 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261614 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 05 13:18:46.356870 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261617 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 05 13:18:46.356870 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261619 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 05 13:18:46.356870 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261621 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 05 13:18:46.356870 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261624 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 05 13:18:46.356870 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261626 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 05 13:18:46.356870 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261629 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 05 13:18:46.357491 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261631 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 05 13:18:46.357491 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261634 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 05 13:18:46.357491 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261636 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 05 13:18:46.357491 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261639 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 05 13:18:46.357491 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261642 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 05 13:18:46.357491 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261645 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 05 13:18:46.357491 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261647 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 05 13:18:46.357491 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261649 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 05 13:18:46.357491 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261653 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 05 13:18:46.357491 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261656 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 05 13:18:46.357491 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261659 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 05 13:18:46.357491 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261663 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 05 13:18:46.357491 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261666 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 05 13:18:46.357491 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261669 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 05 13:18:46.357491 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261672 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 05 13:18:46.357491 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261675 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 05 13:18:46.357491 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261677 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 05 13:18:46.357491 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261680 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 05 13:18:46.357491 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261683 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 05 13:18:46.358565 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261687 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 05 13:18:46.358565 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261689 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 05 13:18:46.358565 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261692 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 05 13:18:46.358565 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261694 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 05 13:18:46.358565 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261697 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 05 13:18:46.358565 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261700 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 05 13:18:46.358565 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261702 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 05 13:18:46.358565 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261704 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 05 13:18:46.358565 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261707 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 05 13:18:46.358565 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261709 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 05 13:18:46.358565 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261712 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 05 13:18:46.358565 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261714 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 05 13:18:46.358565 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261717 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 05 13:18:46.358565 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261719 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 05 13:18:46.358565 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261723 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 05 13:18:46.358565 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261726 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 05 13:18:46.358565 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261729 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 05 13:18:46.358565 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261733 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 05 13:18:46.358565 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261736 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 05 13:18:46.359186 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261738 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 05 13:18:46.359186 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261741 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 05 13:18:46.359186 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261745 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 05 13:18:46.359186 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261748 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 05 13:18:46.359186 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.261751 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 05 13:18:46.359186 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.261828 2573 flags.go:64] FLAG: --address="0.0.0.0" Jul 05 13:18:46.359186 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.261838 2573 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 05 13:18:46.359186 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.261853 2573 flags.go:64] FLAG: --anonymous-auth="true" Jul 05 13:18:46.359186 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.261860 2573 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 05 13:18:46.359186 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.261866 2573 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 05 13:18:46.359186 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.261871 2573 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 05 13:18:46.359186 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.261877 2573 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 05 13:18:46.359186 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.261885 2573 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 05 13:18:46.359186 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.261888 2573 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 05 13:18:46.359186 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.261891 2573 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 05 13:18:46.359186 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.261894 2573 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 05 13:18:46.359186 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.261898 2573 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 05 13:18:46.359186 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.261901 2573 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 05 13:18:46.359186 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.261904 2573 flags.go:64] FLAG: --cgroup-root="" Jul 05 13:18:46.359186 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.261907 2573 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 05 13:18:46.359186 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.261910 2573 flags.go:64] FLAG: --client-ca-file="" Jul 05 13:18:46.359186 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.261913 2573 flags.go:64] FLAG: --cloud-provider="external" Jul 05 13:18:46.359186 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.261927 2573 flags.go:64] FLAG: --cluster-dns="[]" Jul 05 13:18:46.359186 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.261933 2573 flags.go:64] FLAG: --cluster-domain="" Jul 05 13:18:46.360117 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.261936 2573 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 05 13:18:46.360117 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.261939 2573 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 05 13:18:46.360117 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.261942 2573 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 05 13:18:46.360117 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.261945 2573 flags.go:64] FLAG: --container-log-max-files="5" Jul 05 13:18:46.360117 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.261949 2573 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 05 13:18:46.360117 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.261951 2573 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 05 13:18:46.360117 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.261954 2573 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 05 13:18:46.360117 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.261959 2573 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 05 13:18:46.360117 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.261962 2573 flags.go:64] FLAG: --contention-profiling="false" Jul 05 13:18:46.360117 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.261965 2573 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 05 13:18:46.360117 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.261969 2573 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 05 13:18:46.360117 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.261972 2573 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 05 13:18:46.360117 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.261975 2573 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 05 13:18:46.360117 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.261979 2573 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 05 13:18:46.360117 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.261982 2573 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 05 13:18:46.360117 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.261985 2573 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 05 13:18:46.360117 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.261988 2573 flags.go:64] FLAG: --enable-load-reader="false" Jul 05 13:18:46.360117 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.261991 2573 flags.go:64] FLAG: --enable-server="true" Jul 05 13:18:46.360117 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.261994 2573 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 05 13:18:46.360117 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.261999 2573 flags.go:64] FLAG: --event-burst="100" Jul 05 13:18:46.360117 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262003 2573 flags.go:64] FLAG: --event-qps="50" Jul 05 13:18:46.360117 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262006 2573 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 05 13:18:46.360117 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262009 2573 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 05 13:18:46.360117 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262012 2573 flags.go:64] FLAG: --eviction-hard="" Jul 05 13:18:46.361124 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262016 2573 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 05 13:18:46.361124 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262018 2573 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 05 13:18:46.361124 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262021 2573 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 05 13:18:46.361124 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262025 2573 flags.go:64] FLAG: --eviction-soft="" Jul 05 13:18:46.361124 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262028 2573 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 05 13:18:46.361124 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262031 2573 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 05 13:18:46.361124 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262033 2573 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 05 13:18:46.361124 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262036 2573 flags.go:64] FLAG: --experimental-mounter-path="" Jul 05 13:18:46.361124 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262039 2573 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 05 13:18:46.361124 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262042 2573 flags.go:64] FLAG: --fail-swap-on="true" Jul 05 13:18:46.361124 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262044 2573 flags.go:64] FLAG: --feature-gates="" Jul 05 13:18:46.361124 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262048 2573 flags.go:64] FLAG: --file-check-frequency="20s" Jul 05 13:18:46.361124 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262051 2573 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 05 13:18:46.361124 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262054 2573 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 05 13:18:46.361124 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262057 2573 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 05 13:18:46.361124 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262060 2573 flags.go:64] FLAG: --healthz-port="10248" Jul 05 13:18:46.361124 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262064 2573 flags.go:64] FLAG: --help="false" Jul 05 13:18:46.361124 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262066 2573 flags.go:64] FLAG: --hostname-override="ip-10-0-139-155.ec2.internal" Jul 05 13:18:46.361124 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262071 2573 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 05 13:18:46.361124 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262074 2573 flags.go:64] FLAG: --http-check-frequency="20s" Jul 05 13:18:46.361124 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262077 2573 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 05 13:18:46.361124 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262080 2573 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 05 13:18:46.361124 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262083 2573 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 05 13:18:46.362307 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262086 2573 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 05 13:18:46.362307 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262089 2573 flags.go:64] FLAG: --image-service-endpoint="" Jul 05 13:18:46.362307 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262092 2573 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 05 13:18:46.362307 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262095 2573 flags.go:64] FLAG: --kube-api-burst="100" Jul 05 13:18:46.362307 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262097 2573 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 05 13:18:46.362307 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262102 2573 flags.go:64] FLAG: --kube-api-qps="50" Jul 05 13:18:46.362307 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262105 2573 flags.go:64] FLAG: --kube-reserved="" Jul 05 13:18:46.362307 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262108 2573 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 05 13:18:46.362307 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262110 2573 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 05 13:18:46.362307 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262113 2573 flags.go:64] FLAG: --kubelet-cgroups="" Jul 05 13:18:46.362307 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262116 2573 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 05 13:18:46.362307 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262119 2573 flags.go:64] FLAG: --lock-file="" Jul 05 13:18:46.362307 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262121 2573 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 05 13:18:46.362307 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262124 2573 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 05 13:18:46.362307 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262127 2573 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 05 13:18:46.362307 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262132 2573 flags.go:64] FLAG: --log-json-split-stream="false" Jul 05 13:18:46.362307 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262135 2573 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 05 13:18:46.362307 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262138 2573 flags.go:64] FLAG: --log-text-split-stream="false" Jul 05 13:18:46.362307 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262141 2573 flags.go:64] FLAG: --logging-format="text" Jul 05 13:18:46.362307 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262144 2573 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 05 13:18:46.362307 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262147 2573 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 05 13:18:46.362307 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262149 2573 flags.go:64] FLAG: --manifest-url="" Jul 05 13:18:46.362307 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262152 2573 flags.go:64] FLAG: --manifest-url-header="" Jul 05 13:18:46.362307 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262157 2573 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 05 13:18:46.362307 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262159 2573 flags.go:64] FLAG: --max-open-files="1000000" Jul 05 13:18:46.363973 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262164 2573 flags.go:64] FLAG: --max-pods="110" Jul 05 13:18:46.363973 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262167 2573 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 05 13:18:46.363973 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262172 2573 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 05 13:18:46.363973 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262175 2573 flags.go:64] FLAG: --memory-manager-policy="None" Jul 05 13:18:46.363973 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262178 2573 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 05 13:18:46.363973 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262181 2573 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 05 13:18:46.363973 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262184 2573 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 05 13:18:46.363973 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262187 2573 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 05 13:18:46.363973 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262194 2573 flags.go:64] FLAG: --node-status-max-images="50" Jul 05 13:18:46.363973 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262196 2573 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 05 13:18:46.363973 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262199 2573 flags.go:64] FLAG: --oom-score-adj="-999" Jul 05 13:18:46.363973 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262202 2573 flags.go:64] FLAG: --pod-cidr="" Jul 05 13:18:46.363973 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262206 2573 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 05 13:18:46.363973 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262212 2573 flags.go:64] FLAG: --pod-manifest-path="" Jul 05 13:18:46.363973 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262215 2573 flags.go:64] FLAG: --pod-max-pids="-1" Jul 05 13:18:46.363973 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262218 2573 flags.go:64] FLAG: --pods-per-core="0" Jul 05 13:18:46.363973 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262221 2573 flags.go:64] FLAG: --port="10250" Jul 05 13:18:46.363973 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262224 2573 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 05 13:18:46.363973 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262227 2573 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-086c7aca703c0c005" Jul 05 13:18:46.363973 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262230 2573 flags.go:64] FLAG: --qos-reserved="" Jul 05 13:18:46.363973 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262233 2573 flags.go:64] FLAG: --read-only-port="10255" Jul 05 13:18:46.363973 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262235 2573 flags.go:64] FLAG: --register-node="true" Jul 05 13:18:46.363973 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262238 2573 flags.go:64] FLAG: --register-with-taints="" Jul 05 13:18:46.363973 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262242 2573 flags.go:64] FLAG: --registry-burst="10" Jul 05 13:18:46.365390 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262245 2573 flags.go:64] FLAG: --registry-qps="5" Jul 05 13:18:46.365390 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262247 2573 flags.go:64] FLAG: --reserved-cpus="" Jul 05 13:18:46.365390 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262250 2573 flags.go:64] FLAG: --reserved-memory="" Jul 05 13:18:46.365390 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262254 2573 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 05 13:18:46.365390 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262257 2573 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 05 13:18:46.365390 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262260 2573 flags.go:64] FLAG: --rotate-certificates="false" Jul 05 13:18:46.365390 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262263 2573 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 05 13:18:46.365390 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262265 2573 flags.go:64] FLAG: --runonce="false" Jul 05 13:18:46.365390 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262268 2573 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 05 13:18:46.365390 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262271 2573 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 05 13:18:46.365390 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262276 2573 flags.go:64] FLAG: --seccomp-default="false" Jul 05 13:18:46.365390 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262278 2573 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 05 13:18:46.365390 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262281 2573 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 05 13:18:46.365390 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262284 2573 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 05 13:18:46.365390 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262287 2573 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 05 13:18:46.365390 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262290 2573 flags.go:64] FLAG: --storage-driver-password="root" Jul 05 13:18:46.365390 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262293 2573 flags.go:64] FLAG: --storage-driver-secure="false" Jul 05 13:18:46.365390 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262296 2573 flags.go:64] FLAG: --storage-driver-table="stats" Jul 05 13:18:46.365390 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262299 2573 flags.go:64] FLAG: --storage-driver-user="root" Jul 05 13:18:46.365390 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262301 2573 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 05 13:18:46.365390 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262304 2573 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 05 13:18:46.365390 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262309 2573 flags.go:64] FLAG: --system-cgroups="" Jul 05 13:18:46.365390 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262312 2573 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 05 13:18:46.365390 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262317 2573 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 05 13:18:46.365390 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262320 2573 flags.go:64] FLAG: --tls-cert-file="" Jul 05 13:18:46.471902 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262323 2573 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 05 13:18:46.471902 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262327 2573 flags.go:64] FLAG: --tls-min-version="" Jul 05 13:18:46.471902 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262330 2573 flags.go:64] FLAG: --tls-private-key-file="" Jul 05 13:18:46.471902 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262332 2573 flags.go:64] FLAG: --topology-manager-policy="none" Jul 05 13:18:46.471902 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262335 2573 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 05 13:18:46.471902 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262338 2573 flags.go:64] FLAG: --topology-manager-scope="container" Jul 05 13:18:46.471902 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262341 2573 flags.go:64] FLAG: --v="2" Jul 05 13:18:46.471902 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262345 2573 flags.go:64] FLAG: --version="false" Jul 05 13:18:46.471902 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262348 2573 flags.go:64] FLAG: --vmodule="" Jul 05 13:18:46.471902 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262352 2573 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 05 13:18:46.471902 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.262355 2573 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 05 13:18:46.471902 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262474 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 05 13:18:46.471902 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262479 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 05 13:18:46.471902 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262482 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 05 13:18:46.471902 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262485 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 05 13:18:46.471902 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262488 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 05 13:18:46.471902 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262491 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 05 13:18:46.471902 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262495 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 05 13:18:46.471902 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262499 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 05 13:18:46.471902 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262501 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 05 13:18:46.471902 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262504 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 05 13:18:46.471902 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262507 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 05 13:18:47.694705 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262509 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 05 13:18:47.694705 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262512 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 05 13:18:47.694705 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262515 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 05 13:18:47.694705 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262517 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 05 13:18:47.694705 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262520 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 05 13:18:47.694705 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262523 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 05 13:18:47.694705 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262526 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 05 13:18:47.694705 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262529 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 05 13:18:47.694705 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262531 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 05 13:18:47.694705 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262534 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 05 13:18:47.694705 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262537 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 05 13:18:47.694705 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262539 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 05 13:18:47.694705 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262542 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 05 13:18:47.694705 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262544 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 05 13:18:47.694705 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262547 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 05 13:18:47.694705 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262549 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 05 13:18:47.694705 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262552 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 05 13:18:47.694705 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262554 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 05 13:18:47.694705 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262557 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 05 13:18:47.694705 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262559 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 05 13:18:47.695634 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262562 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 05 13:18:47.695634 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262564 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 05 13:18:47.695634 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262567 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 05 13:18:47.695634 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262569 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 05 13:18:47.695634 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262572 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 05 13:18:47.695634 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262574 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 05 13:18:47.695634 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262577 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 05 13:18:47.695634 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262582 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 05 13:18:47.695634 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262586 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 05 13:18:47.695634 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262590 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 05 13:18:47.695634 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262593 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 05 13:18:47.695634 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262596 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 05 13:18:47.695634 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262599 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 05 13:18:47.695634 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262603 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 05 13:18:47.695634 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262606 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 05 13:18:47.695634 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262608 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 05 13:18:47.695634 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262611 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 05 13:18:47.695634 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262614 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 05 13:18:47.695634 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262618 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 05 13:18:47.696330 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262621 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 05 13:18:47.696330 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262623 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 05 13:18:47.696330 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262626 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 05 13:18:47.696330 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262629 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 05 13:18:47.696330 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262631 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 05 13:18:47.696330 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262634 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 05 13:18:47.696330 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262637 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 05 13:18:47.696330 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262639 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 05 13:18:47.696330 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262642 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 05 13:18:47.696330 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262644 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 05 13:18:47.696330 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262647 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 05 13:18:47.696330 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262650 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 05 13:18:47.696330 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262653 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 05 13:18:47.696330 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262655 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 05 13:18:47.696330 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262658 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 05 13:18:47.696330 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262661 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 05 13:18:47.696330 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262663 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 05 13:18:47.696330 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262666 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 05 13:18:47.696330 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262668 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 05 13:18:47.696330 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262671 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 05 13:18:47.696982 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262674 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 05 13:18:47.696982 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262677 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 05 13:18:47.696982 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262679 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 05 13:18:47.696982 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262683 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 05 13:18:47.696982 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262685 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 05 13:18:47.696982 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262688 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 05 13:18:47.696982 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262691 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 05 13:18:47.696982 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262693 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 05 13:18:47.696982 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262696 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 05 13:18:47.696982 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262698 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 05 13:18:47.696982 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262701 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 05 13:18:47.696982 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262706 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 05 13:18:47.696982 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262710 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 05 13:18:47.696982 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262713 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 05 13:18:47.696982 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262716 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 05 13:18:47.696982 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262718 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 05 13:18:47.696982 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262722 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 05 13:18:47.696982 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262724 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 05 13:18:47.696982 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262727 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 05 13:18:47.696982 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262729 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 05 13:18:47.697636 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262732 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 05 13:18:47.697636 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262734 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 05 13:18:47.697636 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262737 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 05 13:18:47.697636 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262739 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 05 13:18:47.697636 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262742 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 05 13:18:47.697636 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262745 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 05 13:18:47.697636 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262748 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 05 13:18:47.697636 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262750 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 05 13:18:47.697636 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.262753 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 05 13:18:47.697636 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.263280 2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 05 13:18:47.697636 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.270228 2573 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 05 13:18:47.697636 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.270244 2573 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 05 13:18:47.697636 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.270785 2573 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 05 13:18:47.697636 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.270793 2573 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 05 13:18:47.697636 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.270854 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 05 13:18:47.698349 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.270859 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 05 13:18:47.698349 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.270863 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 05 13:18:47.698349 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.270866 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 05 13:18:47.698349 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.270870 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 05 13:18:47.698349 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.270873 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 05 13:18:47.698349 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.270875 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 05 13:18:47.698349 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.270878 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 05 13:18:47.698349 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.270881 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 05 13:18:47.698349 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.270884 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 05 13:18:47.698349 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.270887 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 05 13:18:47.698349 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.270889 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 05 13:18:47.698349 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.270892 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 05 13:18:47.698349 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.270895 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 05 13:18:47.698349 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.270900 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 05 13:18:47.698349 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.270903 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 05 13:18:47.698349 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.270906 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 05 13:18:47.698349 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.270908 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 05 13:18:47.698349 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.270911 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 05 13:18:47.698349 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.270913 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 05 13:18:47.698349 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.270931 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 05 13:18:47.699203 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.270934 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 05 13:18:47.699203 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.270936 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 05 13:18:47.699203 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.270939 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 05 13:18:47.699203 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.270941 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 05 13:18:47.699203 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.270944 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 05 13:18:47.699203 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.270947 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 05 13:18:47.699203 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.270950 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 05 13:18:47.699203 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.270954 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 05 13:18:47.699203 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.270958 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 05 13:18:47.699203 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.270961 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 05 13:18:47.699203 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.270964 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 05 13:18:47.699203 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.270966 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 05 13:18:47.699203 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.270969 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 05 13:18:47.699203 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.270972 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 05 13:18:47.699203 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.270975 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 05 13:18:47.699203 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.270978 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 05 13:18:47.699203 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.270981 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 05 13:18:47.699203 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.270985 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 05 13:18:47.699203 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.270988 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 05 13:18:47.699842 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.270991 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 05 13:18:47.699842 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.270994 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 05 13:18:47.699842 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.270997 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 05 13:18:47.699842 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271000 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 05 13:18:47.699842 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271003 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 05 13:18:47.699842 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271006 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 05 13:18:47.699842 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271010 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 05 13:18:47.699842 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271013 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 05 13:18:47.699842 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271016 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 05 13:18:47.699842 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271018 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 05 13:18:47.699842 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271021 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 05 13:18:47.699842 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271023 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 05 13:18:47.699842 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271026 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 05 13:18:47.699842 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271029 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 05 13:18:47.699842 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271031 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 05 13:18:47.699842 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271034 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 05 13:18:47.699842 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271036 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 05 13:18:47.699842 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271039 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 05 13:18:47.699842 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271042 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 05 13:18:47.699842 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271044 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 05 13:18:47.700482 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271047 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 05 13:18:47.700482 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271050 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 05 13:18:47.700482 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271056 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 05 13:18:47.700482 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271059 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 05 13:18:47.700482 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271062 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 05 13:18:47.700482 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271065 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 05 13:18:47.700482 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271068 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 05 13:18:47.700482 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271070 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 05 13:18:47.700482 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271073 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 05 13:18:47.700482 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271077 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 05 13:18:47.700482 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271080 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 05 13:18:47.700482 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271083 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 05 13:18:47.700482 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271086 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 05 13:18:47.700482 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271089 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 05 13:18:47.700482 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271092 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 05 13:18:47.700482 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271095 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 05 13:18:47.700482 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271097 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 05 13:18:47.700482 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271100 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 05 13:18:47.700482 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271104 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 05 13:18:47.701097 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271107 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 05 13:18:47.701097 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271110 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 05 13:18:47.701097 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271112 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 05 13:18:47.701097 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271115 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 05 13:18:47.701097 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271118 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 05 13:18:47.701097 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271120 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 05 13:18:47.701097 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271123 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 05 13:18:47.701097 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271125 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 05 13:18:47.701097 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271128 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 05 13:18:47.701097 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271130 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 05 13:18:47.701097 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271133 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 05 13:18:47.701097 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271136 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 05 13:18:47.701097 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271138 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 05 13:18:47.701097 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271141 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 05 13:18:47.701097 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271143 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 05 13:18:47.701097 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271151 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 05 13:18:47.701097 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271155 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 05 13:18:47.701097 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271157 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 05 13:18:47.701097 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271160 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 05 13:18:47.701097 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271163 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 05 13:18:47.701941 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.271168 2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 05 13:18:47.701941 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271280 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 05 13:18:47.701941 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271285 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 05 13:18:47.701941 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271288 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 05 13:18:47.701941 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271291 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 05 13:18:47.701941 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271294 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 05 13:18:47.701941 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271297 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 05 13:18:47.701941 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271300 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 05 13:18:47.701941 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271303 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 05 13:18:47.701941 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271306 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 05 13:18:47.701941 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271309 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 05 13:18:47.701941 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271312 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 05 13:18:47.701941 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271316 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 05 13:18:47.701941 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271319 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 05 13:18:47.701941 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271322 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 05 13:18:47.702565 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271324 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 05 13:18:47.702565 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271329 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 05 13:18:47.702565 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271333 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 05 13:18:47.702565 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271336 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 05 13:18:47.702565 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271339 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 05 13:18:47.702565 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271341 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 05 13:18:47.702565 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271344 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 05 13:18:47.702565 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271347 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 05 13:18:47.702565 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271350 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 05 13:18:47.702565 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271352 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 05 13:18:47.702565 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271355 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 05 13:18:47.702565 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271357 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 05 13:18:47.702565 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271361 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 05 13:18:47.702565 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271364 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 05 13:18:47.702565 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271367 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 05 13:18:47.702565 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271369 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 05 13:18:47.702565 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271372 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 05 13:18:47.702565 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271374 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 05 13:18:47.702565 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271377 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 05 13:18:47.702565 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271379 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 05 13:18:47.703184 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271382 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 05 13:18:47.703184 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271384 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 05 13:18:47.703184 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271387 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 05 13:18:47.703184 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271389 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 05 13:18:47.703184 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271392 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 05 13:18:47.703184 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271394 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 05 13:18:47.703184 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271397 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 05 13:18:47.703184 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271399 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 05 13:18:47.703184 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271402 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 05 13:18:47.703184 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271406 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 05 13:18:47.703184 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271409 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 05 13:18:47.703184 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271411 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 05 13:18:47.703184 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271414 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 05 13:18:47.703184 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271416 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 05 13:18:47.703184 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271419 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 05 13:18:47.703184 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271421 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 05 13:18:47.703184 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271424 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 05 13:18:47.703184 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271426 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 05 13:18:47.703184 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271429 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 05 13:18:47.703184 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271432 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 05 13:18:47.703795 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271434 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 05 13:18:47.703795 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271437 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 05 13:18:47.703795 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271439 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 05 13:18:47.703795 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271441 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 05 13:18:47.703795 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271445 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 05 13:18:47.703795 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271448 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 05 13:18:47.703795 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271451 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 05 13:18:47.703795 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271454 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 05 13:18:47.703795 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271456 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 05 13:18:47.703795 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271459 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 05 13:18:47.703795 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271461 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 05 13:18:47.703795 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271464 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 05 13:18:47.703795 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271466 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 05 13:18:47.703795 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271469 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 05 13:18:47.703795 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271473 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 05 13:18:47.703795 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271476 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 05 13:18:47.703795 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271479 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 05 13:18:47.703795 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271482 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 05 13:18:47.703795 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271485 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 05 13:18:47.704416 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271487 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 05 13:18:47.704416 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271490 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 05 13:18:47.704416 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271494 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 05 13:18:47.704416 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271496 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 05 13:18:47.704416 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271499 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 05 13:18:47.704416 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271502 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 05 13:18:47.704416 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271504 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 05 13:18:47.704416 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271507 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 05 13:18:47.704416 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271509 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 05 13:18:47.704416 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271512 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 05 13:18:47.704416 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271514 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 05 13:18:47.704416 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271517 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 05 13:18:47.704416 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271519 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 05 13:18:47.704416 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271522 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 05 13:18:47.704416 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271524 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 05 13:18:47.704416 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271527 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 05 13:18:47.704416 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271529 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 05 13:18:47.704416 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271533 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 05 13:18:47.704416 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271536 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 05 13:18:47.705232 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271539 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 05 13:18:47.705232 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271542 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 05 13:18:47.705232 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271546 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 05 13:18:47.705232 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271548 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 05 13:18:47.705232 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271551 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 05 13:18:47.705232 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271553 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 05 13:18:47.705232 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:46.271556 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 05 13:18:47.705232 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.271560 2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 05 13:18:47.705232 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.271717 2573 server.go:962] "Client rotation is on, will bootstrap in background" Jul 05 13:18:47.705232 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.273784 2573 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 05 13:18:47.705232 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.275205 2573 server.go:1019] "Starting client certificate rotation" Jul 05 13:18:47.705232 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.275301 2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 05 13:18:47.705232 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.275335 2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 05 13:18:47.705663 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.277940 2573 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 05 13:18:47.705663 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.283210 2573 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 05 13:18:47.705663 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.293491 2573 log.go:25] "Validated CRI v1 runtime API" Jul 05 13:18:47.705663 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.295578 2573 log.go:25] "Validated CRI v1 image API" Jul 05 13:18:47.705663 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.297097 2573 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 05 13:18:47.705663 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.299518 2573 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 f91afcba-3884-4f43-81c0-5846aefd5094:/dev/nvme0n1p4 fdd6e808-c191-4356-998d-fd48c8957831:/dev/nvme0n1p3] Jul 05 13:18:47.705663 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.299535 2573 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 05 13:18:47.706098 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.303714 2573 manager.go:216] Machine: {Timestamp:2026-07-05 13:18:46.303207843 +0000 UTC m=+0.399898215 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100971 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2c45a8d54e632fd42382e4e6b9c8ed SystemUUID:ec2c45a8-d54e-632f-d423-82e4e6b9c8ed BootID:924a9b1d-aea9-4e49-9fc2-cd5ef412b4c8 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:b3:45:e9:85:7b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:b3:45:e9:85:7b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:7e:5f:ed:52:e8:ea Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 05 13:18:47.706098 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.303810 2573 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 05 13:18:47.706098 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.303911 2573 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 05 13:18:47.706098 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.305304 2573 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 05 13:18:47.706098 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.305338 2573 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-155.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 05 13:18:47.706098 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.306597 2573 topology_manager.go:138] "Creating topology manager with none policy" Jul 05 13:18:47.706098 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.306611 2573 container_manager_linux.go:309] "Creating device plugin manager" Jul 05 13:18:47.706098 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.306621 2573 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 05 13:18:47.706098 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.307500 2573 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 05 13:18:47.706098 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.308654 2573 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 05 13:18:47.706098 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.311675 2573 state_mem.go:36] "Initialized new in-memory state store" Jul 05 13:18:47.706098 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.311777 2573 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 05 13:18:47.706098 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.312183 2573 kubelet.go:486] "Attempting to sync node with API server" Jul 05 13:18:47.706098 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.312193 2573 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 05 13:18:47.706098 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.312208 2573 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 05 13:18:47.706098 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.312217 2573 kubelet.go:398] "Adding apiserver pod source" Jul 05 13:18:47.706098 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.312227 2573 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 05 13:18:47.706098 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.312511 2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 05 13:18:47.707045 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.312525 2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 05 13:18:47.707045 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.314258 2573 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 05 13:18:47.707045 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.314604 2573 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 05 13:18:47.707045 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.316442 2573 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 05 13:18:47.707045 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.316470 2573 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 05 13:18:47.707045 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.317495 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 05 13:18:47.707045 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.317518 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 05 13:18:47.707045 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.317526 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 05 13:18:47.707045 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.317531 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 05 13:18:47.707045 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.317540 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 05 13:18:47.707045 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.317545 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 05 13:18:47.707045 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.317554 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 05 13:18:47.707045 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.317560 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 05 13:18:47.707045 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.317567 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 05 13:18:47.707045 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.317576 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 05 13:18:47.707045 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.317588 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 05 13:18:47.707045 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.317593 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 05 13:18:47.707045 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.318321 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 05 13:18:47.707045 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.318331 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 05 13:18:47.707045 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.321853 2573 server.go:1268] "Started kubelet" Jul 05 13:18:47.707045 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:46.321990 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-155.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 05 13:18:47.707045 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.322041 2573 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-155.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 05 13:18:47.707045 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:46.322225 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 05 13:18:47.707045 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.322894 2573 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 05 13:18:47.707045 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.323649 2573 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 05 13:18:47.707045 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.323719 2573 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 05 13:18:47.707045 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.324648 2573 server.go:310] "Adding debug handlers to kubelet server" Jul 05 13:18:47.707045 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.325057 2573 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 05 13:18:47.707045 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:46.333855 2573 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 05 13:18:47.707045 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:46.335173 2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-155.ec2.internal.18bf6745a77462a5 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-155.ec2.internal,UID:ip-10-0-139-155.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-155.ec2.internal,},FirstTimestamp:2026-07-05 13:18:46.321832613 +0000 UTC m=+0.418522986,LastTimestamp:2026-07-05 13:18:46.321832613 +0000 UTC m=+0.418522986,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-155.ec2.internal,}" Jul 05 13:18:47.707045 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.337362 2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cx6nf" Jul 05 13:18:47.707045 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.337590 2573 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 05 13:18:47.707045 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.337939 2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 05 13:18:47.707045 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.338820 2573 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 05 13:18:47.707045 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.338829 2573 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 05 13:18:47.707045 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:46.339028 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-155.ec2.internal\" not found" Jul 05 13:18:47.707045 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.339303 2573 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 05 13:18:47.707045 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.339372 2573 reconstruct.go:97] "Volume reconstruction finished" Jul 05 13:18:47.707045 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.339379 2573 reconciler.go:29] "Reconciler: start to sync state" Jul 05 13:18:47.708194 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.339976 2573 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 05 13:18:47.708194 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.339988 2573 factory.go:55] Registering systemd factory Jul 05 13:18:47.708194 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.339995 2573 factory.go:223] Registration of the systemd container factory successfully Jul 05 13:18:47.708194 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.340261 2573 factory.go:153] Registering CRI-O factory Jul 05 13:18:47.708194 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.340275 2573 factory.go:223] Registration of the crio container factory successfully Jul 05 13:18:47.708194 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.340298 2573 factory.go:103] Registering Raw factory Jul 05 13:18:47.708194 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.340312 2573 manager.go:1196] Started watching for new ooms in manager Jul 05 13:18:47.708194 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.340730 2573 manager.go:349] Starting recovery of all containers Jul 05 13:18:47.708194 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.347142 2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cx6nf" Jul 05 13:18:47.708194 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.349176 2573 manager.go:354] Recovery completed Jul 05 13:18:47.708194 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:46.349698 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 05 13:18:47.708194 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:46.349790 2573 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-139-155.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 05 13:18:47.708194 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.355985 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 13:18:47.708194 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.357711 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-155.ec2.internal" event="NodeHasSufficientMemory" Jul 05 13:18:47.708194 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.357736 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-155.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 13:18:47.708194 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.357751 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-155.ec2.internal" event="NodeHasSufficientPID" Jul 05 13:18:47.708194 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.358538 2573 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 05 13:18:47.708194 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.358547 2573 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 05 13:18:47.708194 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.358562 2573 state_mem.go:36] "Initialized new in-memory state store" Jul 05 13:18:47.708194 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.359728 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 13:18:47.708194 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.359801 2573 policy_none.go:49] "None policy: Start" Jul 05 13:18:47.708194 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.359828 2573 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 05 13:18:47.708194 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.359850 2573 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 05 13:18:47.708194 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.360808 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-155.ec2.internal" event="NodeHasSufficientMemory" Jul 05 13:18:47.708194 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.360845 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-155.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 13:18:47.708194 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.360860 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-155.ec2.internal" event="NodeHasSufficientPID" Jul 05 13:18:47.708194 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.361332 2573 policy_none.go:47] "Start" Jul 05 13:18:47.708194 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.399315 2573 manager.go:337] "Starting Device Plugin manager" Jul 05 13:18:47.708194 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:46.399374 2573 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 05 13:18:47.708194 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.399388 2573 server.go:87] "Starting device plugin registration server" Jul 05 13:18:47.708194 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.399605 2573 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 05 13:18:47.708194 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.399617 2573 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 05 13:18:47.708194 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.399870 2573 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 05 13:18:47.708194 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.399968 2573 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 05 13:18:47.708194 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:46.399977 2573 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 05 13:18:47.708194 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:46.400359 2573 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 05 13:18:47.708194 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:46.400397 2573 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-155.ec2.internal\" not found" Jul 05 13:18:47.708194 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:47.701159 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 13:18:47.708194 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:47.705976 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-155.ec2.internal" event="NodeHasSufficientMemory" Jul 05 13:18:47.708194 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:47.706011 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-155.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 13:18:47.708194 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:47.706022 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-155.ec2.internal" event="NodeHasSufficientPID" Jul 05 13:18:47.708194 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:47.706056 2573 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-155.ec2.internal" Jul 05 13:18:47.809234 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:47.715833 2573 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-155.ec2.internal" Jul 05 13:18:47.809234 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:47.716262 2573 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-155.ec2.internal\": node \"ip-10-0-139-155.ec2.internal\" not found" Jul 05 13:18:47.809234 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:47.728139 2573 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 05 13:18:47.809234 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:47.729358 2573 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 05 13:18:47.809234 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:47.729389 2573 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 05 13:18:47.809234 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:47.729413 2573 kubelet.go:2448] "Starting kubelet main sync loop" Jul 05 13:18:47.809234 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:47.729442 2573 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 05 13:18:47.809234 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:47.737425 2573 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 05 13:18:47.809234 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:47.737483 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-155.ec2.internal\" not found" Jul 05 13:18:47.925604 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:47.830089 2573 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-155.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-155.ec2.internal"] Jul 05 13:18:47.925604 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:47.830831 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 13:18:47.925604 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:47.831776 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-155.ec2.internal" event="NodeHasSufficientMemory" Jul 05 13:18:47.925604 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:47.831804 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-155.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 13:18:47.925604 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:47.831814 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-155.ec2.internal" event="NodeHasSufficientPID" Jul 05 13:18:47.925604 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:47.834009 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 13:18:47.925604 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:47.834139 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-155.ec2.internal" Jul 05 13:18:47.925604 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:47.834175 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 13:18:47.925604 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:47.834793 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-155.ec2.internal" event="NodeHasSufficientMemory" Jul 05 13:18:47.925604 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:47.834815 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-155.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 13:18:47.925604 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:47.834825 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-155.ec2.internal" event="NodeHasSufficientPID" Jul 05 13:18:47.925604 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:47.834793 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-155.ec2.internal" event="NodeHasSufficientMemory" Jul 05 13:18:47.925604 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:47.835041 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-155.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 13:18:47.925604 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:47.835056 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-155.ec2.internal" event="NodeHasSufficientPID" Jul 05 13:18:47.925604 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:47.837119 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-155.ec2.internal" Jul 05 13:18:47.925604 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:47.837142 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 13:18:47.925604 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:47.837575 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-155.ec2.internal\" not found" Jul 05 13:18:47.925604 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:47.837731 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-155.ec2.internal" event="NodeHasSufficientMemory" Jul 05 13:18:47.925604 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:47.837755 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-155.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 13:18:47.925604 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:47.837768 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-155.ec2.internal" event="NodeHasSufficientPID" Jul 05 13:18:47.925604 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:47.842299 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2159b56be8d2ffd1bbbc05521c1be8b4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-155.ec2.internal\" (UID: \"2159b56be8d2ffd1bbbc05521c1be8b4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-155.ec2.internal" Jul 05 13:18:47.925604 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:47.842325 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2159b56be8d2ffd1bbbc05521c1be8b4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-155.ec2.internal\" (UID: \"2159b56be8d2ffd1bbbc05521c1be8b4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-155.ec2.internal" Jul 05 13:18:47.925604 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:47.842350 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f7be285f4bc4ee8d9639260737851004-config\") pod \"kube-apiserver-proxy-ip-10-0-139-155.ec2.internal\" (UID: \"f7be285f4bc4ee8d9639260737851004\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-155.ec2.internal" Jul 05 13:18:47.925604 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:47.871755 2573 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-155.ec2.internal\" not found" node="ip-10-0-139-155.ec2.internal" Jul 05 13:18:47.925604 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:47.877831 2573 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-155.ec2.internal\" not found" node="ip-10-0-139-155.ec2.internal" Jul 05 13:18:48.042327 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:47.937661 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-155.ec2.internal\" not found" Jul 05 13:18:48.042327 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:47.943010 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2159b56be8d2ffd1bbbc05521c1be8b4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-155.ec2.internal\" (UID: \"2159b56be8d2ffd1bbbc05521c1be8b4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-155.ec2.internal" Jul 05 13:18:48.042327 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:47.943037 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f7be285f4bc4ee8d9639260737851004-config\") pod \"kube-apiserver-proxy-ip-10-0-139-155.ec2.internal\" (UID: \"f7be285f4bc4ee8d9639260737851004\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-155.ec2.internal" Jul 05 13:18:48.042327 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:47.943053 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2159b56be8d2ffd1bbbc05521c1be8b4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-155.ec2.internal\" (UID: \"2159b56be8d2ffd1bbbc05521c1be8b4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-155.ec2.internal" Jul 05 13:18:48.042327 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:47.943090 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2159b56be8d2ffd1bbbc05521c1be8b4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-155.ec2.internal\" (UID: \"2159b56be8d2ffd1bbbc05521c1be8b4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-155.ec2.internal" Jul 05 13:18:48.042327 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:47.943115 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2159b56be8d2ffd1bbbc05521c1be8b4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-155.ec2.internal\" (UID: \"2159b56be8d2ffd1bbbc05521c1be8b4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-155.ec2.internal" Jul 05 13:18:48.042327 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:47.943118 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f7be285f4bc4ee8d9639260737851004-config\") pod \"kube-apiserver-proxy-ip-10-0-139-155.ec2.internal\" (UID: \"f7be285f4bc4ee8d9639260737851004\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-155.ec2.internal" Jul 05 13:18:48.042327 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:48.037838 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-155.ec2.internal\" not found" Jul 05 13:18:48.168245 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:48.138318 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-155.ec2.internal\" not found" Jul 05 13:18:48.282866 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:48.175764 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-155.ec2.internal" Jul 05 13:18:48.282866 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:48.181623 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-155.ec2.internal" Jul 05 13:18:48.282866 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:48.238610 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-155.ec2.internal\" not found" Jul 05 13:18:48.400863 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:48.339119 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-155.ec2.internal\" not found" Jul 05 13:18:48.510760 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:48.439605 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-155.ec2.internal\" not found" Jul 05 13:18:48.510760 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:48.475989 2573 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 05 13:18:48.510760 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:48.476146 2573 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 05 13:18:48.633548 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:48.540303 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-155.ec2.internal\" not found" Jul 05 13:18:48.633548 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:48.551075 2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-04 13:13:47 +0000 UTC" deadline="2028-02-04 06:11:36.939500093 +0000 UTC" Jul 05 13:18:48.633548 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:48.551093 2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13888h52m48.388410383s" Jul 05 13:18:48.722099 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:48.645474 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-155.ec2.internal\" not found" Jul 05 13:18:48.821174 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:48.728285 2573 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 05 13:18:48.821174 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:48.745855 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-155.ec2.internal\" not found" Jul 05 13:18:48.821174 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:48.754230 2573 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 05 13:18:48.821174 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:48.787845 2573 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 05 13:18:48.914754 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:48.840604 2573 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-155.ec2.internal" Jul 05 13:18:48.914754 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:48.853140 2573 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 05 13:18:48.914754 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:48.854037 2573 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-155.ec2.internal" Jul 05 13:18:48.914754 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:48.862814 2573 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 05 13:18:48.914754 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:48.894819 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2159b56be8d2ffd1bbbc05521c1be8b4.slice/crio-3865ed202465b3cc474280c61c9d59c108a9a06fc480b082496c3e118d9da478 WatchSource:0}: Error finding container 3865ed202465b3cc474280c61c9d59c108a9a06fc480b082496c3e118d9da478: Status 404 returned error can't find the container with id 3865ed202465b3cc474280c61c9d59c108a9a06fc480b082496c3e118d9da478 Jul 05 13:18:48.914754 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:48.896118 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7be285f4bc4ee8d9639260737851004.slice/crio-d01bae3151ae9d914f606a10d4b25b5ef513081b72c9747d9b79373a15a565ae WatchSource:0}: Error finding container d01bae3151ae9d914f606a10d4b25b5ef513081b72c9747d9b79373a15a565ae: Status 404 returned error can't find the container with id d01bae3151ae9d914f606a10d4b25b5ef513081b72c9747d9b79373a15a565ae Jul 05 13:18:48.914754 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:48.904854 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 05 13:18:49.514883 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.514847 2573 apiserver.go:52] "Watching apiserver" Jul 05 13:18:49.522650 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.522625 2573 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 05 13:18:49.523924 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.523895 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-hlvjf","openshift-multus/multus-4p2vl","openshift-multus/multus-additional-cni-plugins-fw42d","kube-system/global-pull-secret-syncer-l6cc6","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-155.ec2.internal","openshift-multus/network-metrics-daemon-g9tcx","openshift-network-diagnostics/network-check-target-2nr4r","openshift-network-operator/iptables-alerter-tpn2j","openshift-ovn-kubernetes/ovnkube-node-bzhlz","kube-system/konnectivity-agent-dr5b4","kube-system/kube-apiserver-proxy-ip-10-0-139-155.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjd4z","openshift-cluster-node-tuning-operator/tuned-bh9gk","openshift-dns/node-resolver-j5pwd"] Jul 05 13:18:49.526525 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.526496 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2nr4r" Jul 05 13:18:49.526661 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:49.526586 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2nr4r" podUID="09e0d252-5d66-4697-b469-24d2cdf32679" Jul 05 13:18:49.528735 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.528709 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tpn2j" Jul 05 13:18:49.532484 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.531495 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fw42d" Jul 05 13:18:49.532484 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.531995 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 05 13:18:49.535033 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.535010 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 05 13:18:49.535499 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.535480 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 05 13:18:49.535594 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.535573 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 05 13:18:49.535657 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.535582 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-cb786\"" Jul 05 13:18:49.535717 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.535686 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 05 13:18:49.535773 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.535755 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 05 13:18:49.535873 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.535851 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-ckmtn\"" Jul 05 13:18:49.535934 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.535878 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 05 13:18:49.536006 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.535975 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l6cc6" Jul 05 13:18:49.536058 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:49.536043 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l6cc6" podUID="33610912-09f3-406f-be0f-a6ffbff4c9ba" Jul 05 13:18:49.536132 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.536119 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 05 13:18:49.538218 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.538179 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-hlvjf" Jul 05 13:18:49.540303 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.540285 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g9tcx" Jul 05 13:18:49.540397 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:49.540350 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g9tcx" podUID="2010fc9b-2713-4036-8131-071a9349d36e" Jul 05 13:18:49.541261 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.541227 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 05 13:18:49.541457 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.541402 2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 05 13:18:49.544011 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.542869 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 05 13:18:49.544011 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.542898 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-rngph\"" Jul 05 13:18:49.544011 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.543096 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.544011 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.543121 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 05 13:18:49.547978 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.547044 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bh9gk" Jul 05 13:18:49.548331 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.548309 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 05 13:18:49.548675 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.548659 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 05 13:18:49.548943 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.548916 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 05 13:18:49.549224 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.549209 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-hgxsb\"" Jul 05 13:18:49.549526 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.549512 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 05 13:18:49.549669 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.549651 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 05 13:18:49.549800 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.549270 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 05 13:18:49.550147 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.550128 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 05 13:18:49.550285 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.550265 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 05 13:18:49.550353 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.550287 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-m8hgn\"" Jul 05 13:18:49.551863 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.550790 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4p2vl" Jul 05 13:18:49.553427 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.553284 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-gkhbr\"" Jul 05 13:18:49.553427 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.553315 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 05 13:18:49.554525 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.554184 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dr5b4" Jul 05 13:18:49.554759 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.554741 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fafc6427-6250-484c-88b0-d7b08365d04b-node-log\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.554832 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.554772 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fafc6427-6250-484c-88b0-d7b08365d04b-env-overrides\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.554832 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.554796 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/33610912-09f3-406f-be0f-a6ffbff4c9ba-dbus\") pod \"global-pull-secret-syncer-l6cc6\" (UID: \"33610912-09f3-406f-be0f-a6ffbff4c9ba\") " pod="kube-system/global-pull-secret-syncer-l6cc6" Jul 05 13:18:49.554832 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.554817 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fafc6427-6250-484c-88b0-d7b08365d04b-host-slash\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.554994 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.554840 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fafc6427-6250-484c-88b0-d7b08365d04b-etc-openvswitch\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.554994 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.554878 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/91b3802b-db71-4a40-ad33-0f2ced5ef31d-cnibin\") pod \"multus-additional-cni-plugins-fw42d\" (UID: \"91b3802b-db71-4a40-ad33-0f2ced5ef31d\") " pod="openshift-multus/multus-additional-cni-plugins-fw42d" Jul 05 13:18:49.554994 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.554922 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/33610912-09f3-406f-be0f-a6ffbff4c9ba-kubelet-config\") pod \"global-pull-secret-syncer-l6cc6\" (UID: \"33610912-09f3-406f-be0f-a6ffbff4c9ba\") " pod="kube-system/global-pull-secret-syncer-l6cc6" Jul 05 13:18:49.554994 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.554967 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2010fc9b-2713-4036-8131-071a9349d36e-metrics-certs\") pod \"network-metrics-daemon-g9tcx\" (UID: \"2010fc9b-2713-4036-8131-071a9349d36e\") " pod="openshift-multus/network-metrics-daemon-g9tcx" Jul 05 13:18:49.555188 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.555003 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fafc6427-6250-484c-88b0-d7b08365d04b-systemd-units\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.555188 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.555035 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fafc6427-6250-484c-88b0-d7b08365d04b-host-run-netns\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.555188 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.555060 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fafc6427-6250-484c-88b0-d7b08365d04b-ovnkube-config\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.555188 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.555090 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwsk5\" (UniqueName: \"kubernetes.io/projected/fafc6427-6250-484c-88b0-d7b08365d04b-kube-api-access-cwsk5\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.555188 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.555114 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/93c7227a-9d34-4565-abe7-ff14cddfe6b2-host-slash\") pod \"iptables-alerter-tpn2j\" (UID: \"93c7227a-9d34-4565-abe7-ff14cddfe6b2\") " pod="openshift-network-operator/iptables-alerter-tpn2j" Jul 05 13:18:49.555188 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.555151 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2a975403-6fed-4035-bf2d-c161d33baf8e-serviceca\") pod \"node-ca-hlvjf\" (UID: \"2a975403-6fed-4035-bf2d-c161d33baf8e\") " pod="openshift-image-registry/node-ca-hlvjf" Jul 05 13:18:49.555188 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.555174 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fafc6427-6250-484c-88b0-d7b08365d04b-run-openvswitch\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.555498 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.555197 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fafc6427-6250-484c-88b0-d7b08365d04b-host-run-ovn-kubernetes\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.555498 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.555226 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fafc6427-6250-484c-88b0-d7b08365d04b-host-cni-bin\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.555498 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.555253 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/91b3802b-db71-4a40-ad33-0f2ced5ef31d-cni-binary-copy\") pod \"multus-additional-cni-plugins-fw42d\" (UID: \"91b3802b-db71-4a40-ad33-0f2ced5ef31d\") " pod="openshift-multus/multus-additional-cni-plugins-fw42d" Jul 05 13:18:49.555498 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.555275 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/91b3802b-db71-4a40-ad33-0f2ced5ef31d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fw42d\" (UID: \"91b3802b-db71-4a40-ad33-0f2ced5ef31d\") " pod="openshift-multus/multus-additional-cni-plugins-fw42d" Jul 05 13:18:49.555498 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.555297 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2a975403-6fed-4035-bf2d-c161d33baf8e-host\") pod \"node-ca-hlvjf\" (UID: \"2a975403-6fed-4035-bf2d-c161d33baf8e\") " pod="openshift-image-registry/node-ca-hlvjf" Jul 05 13:18:49.555498 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.555319 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fafc6427-6250-484c-88b0-d7b08365d04b-var-lib-openvswitch\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.555498 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.555340 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fafc6427-6250-484c-88b0-d7b08365d04b-ovn-node-metrics-cert\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.555498 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.555364 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krvhl\" (UniqueName: \"kubernetes.io/projected/09e0d252-5d66-4697-b469-24d2cdf32679-kube-api-access-krvhl\") pod \"network-check-target-2nr4r\" (UID: \"09e0d252-5d66-4697-b469-24d2cdf32679\") " pod="openshift-network-diagnostics/network-check-target-2nr4r" Jul 05 13:18:49.555498 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.555396 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/91b3802b-db71-4a40-ad33-0f2ced5ef31d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fw42d\" (UID: \"91b3802b-db71-4a40-ad33-0f2ced5ef31d\") " pod="openshift-multus/multus-additional-cni-plugins-fw42d" Jul 05 13:18:49.555498 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.555458 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bkzq\" (UniqueName: \"kubernetes.io/projected/91b3802b-db71-4a40-ad33-0f2ced5ef31d-kube-api-access-8bkzq\") pod \"multus-additional-cni-plugins-fw42d\" (UID: \"91b3802b-db71-4a40-ad33-0f2ced5ef31d\") " pod="openshift-multus/multus-additional-cni-plugins-fw42d" Jul 05 13:18:49.555498 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.555484 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v58qx\" (UniqueName: \"kubernetes.io/projected/2a975403-6fed-4035-bf2d-c161d33baf8e-kube-api-access-v58qx\") pod \"node-ca-hlvjf\" (UID: \"2a975403-6fed-4035-bf2d-c161d33baf8e\") " pod="openshift-image-registry/node-ca-hlvjf" Jul 05 13:18:49.556078 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.555506 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fafc6427-6250-484c-88b0-d7b08365d04b-run-systemd\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.556078 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.555529 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fafc6427-6250-484c-88b0-d7b08365d04b-ovnkube-script-lib\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.556078 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.555560 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s65wl\" (UniqueName: \"kubernetes.io/projected/93c7227a-9d34-4565-abe7-ff14cddfe6b2-kube-api-access-s65wl\") pod \"iptables-alerter-tpn2j\" (UID: \"93c7227a-9d34-4565-abe7-ff14cddfe6b2\") " pod="openshift-network-operator/iptables-alerter-tpn2j" Jul 05 13:18:49.556078 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.555594 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5lpx\" (UniqueName: \"kubernetes.io/projected/2010fc9b-2713-4036-8131-071a9349d36e-kube-api-access-f5lpx\") pod \"network-metrics-daemon-g9tcx\" (UID: \"2010fc9b-2713-4036-8131-071a9349d36e\") " pod="openshift-multus/network-metrics-daemon-g9tcx" Jul 05 13:18:49.556078 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.555617 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fafc6427-6250-484c-88b0-d7b08365d04b-host-kubelet\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.556078 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.555655 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fafc6427-6250-484c-88b0-d7b08365d04b-log-socket\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.556078 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.555689 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fafc6427-6250-484c-88b0-d7b08365d04b-host-cni-netd\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.556078 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.555716 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fafc6427-6250-484c-88b0-d7b08365d04b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.556078 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.555742 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/93c7227a-9d34-4565-abe7-ff14cddfe6b2-iptables-alerter-script\") pod \"iptables-alerter-tpn2j\" (UID: \"93c7227a-9d34-4565-abe7-ff14cddfe6b2\") " pod="openshift-network-operator/iptables-alerter-tpn2j" Jul 05 13:18:49.556078 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.555767 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/91b3802b-db71-4a40-ad33-0f2ced5ef31d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fw42d\" (UID: \"91b3802b-db71-4a40-ad33-0f2ced5ef31d\") " pod="openshift-multus/multus-additional-cni-plugins-fw42d" Jul 05 13:18:49.556078 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.555790 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/33610912-09f3-406f-be0f-a6ffbff4c9ba-original-pull-secret\") pod \"global-pull-secret-syncer-l6cc6\" (UID: \"33610912-09f3-406f-be0f-a6ffbff4c9ba\") " pod="kube-system/global-pull-secret-syncer-l6cc6" Jul 05 13:18:49.556078 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.555814 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fafc6427-6250-484c-88b0-d7b08365d04b-run-ovn\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.556078 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.555835 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/91b3802b-db71-4a40-ad33-0f2ced5ef31d-system-cni-dir\") pod \"multus-additional-cni-plugins-fw42d\" (UID: \"91b3802b-db71-4a40-ad33-0f2ced5ef31d\") " pod="openshift-multus/multus-additional-cni-plugins-fw42d" Jul 05 13:18:49.556078 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.555862 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/91b3802b-db71-4a40-ad33-0f2ced5ef31d-os-release\") pod \"multus-additional-cni-plugins-fw42d\" (UID: \"91b3802b-db71-4a40-ad33-0f2ced5ef31d\") " pod="openshift-multus/multus-additional-cni-plugins-fw42d" Jul 05 13:18:49.556078 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.556004 2573 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 05 13:18:49.556758 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.556316 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 05 13:18:49.556758 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.556400 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-g9p7x\"" Jul 05 13:18:49.556758 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.556596 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 05 13:18:49.558132 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.558113 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-j5pwd" Jul 05 13:18:49.560230 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.560212 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 05 13:18:49.560310 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.560212 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 05 13:18:49.560630 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.560609 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-smh84\"" Jul 05 13:18:49.560702 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.560672 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjd4z" Jul 05 13:18:49.562926 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.562903 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 05 13:18:49.563112 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.563079 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-rw2g6\"" Jul 05 13:18:49.563213 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.563128 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 05 13:18:49.563315 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.563300 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 05 13:18:49.591017 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.590992 2573 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 05 13:18:49.601444 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.601418 2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-67dm2" Jul 05 13:18:49.611188 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.611172 2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-67dm2" Jul 05 13:18:49.642277 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.642258 2573 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 05 13:18:49.657495 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.656908 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6b401f24-d573-41fc-8668-0fd9d69e2cd3-sys\") pod \"tuned-bh9gk\" (UID: \"6b401f24-d573-41fc-8668-0fd9d69e2cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-bh9gk" Jul 05 13:18:49.657495 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.656946 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fafc6427-6250-484c-88b0-d7b08365d04b-node-log\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.657495 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.657028 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fafc6427-6250-484c-88b0-d7b08365d04b-node-log\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.657495 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.657033 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6b401f24-d573-41fc-8668-0fd9d69e2cd3-tmp\") pod \"tuned-bh9gk\" (UID: \"6b401f24-d573-41fc-8668-0fd9d69e2cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-bh9gk" Jul 05 13:18:49.657495 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.657069 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8ec840e9-44f1-4daa-963a-96632c87bb8a-system-cni-dir\") pod \"multus-4p2vl\" (UID: \"8ec840e9-44f1-4daa-963a-96632c87bb8a\") " pod="openshift-multus/multus-4p2vl" Jul 05 13:18:49.657495 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.657093 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8ec840e9-44f1-4daa-963a-96632c87bb8a-multus-cni-dir\") pod \"multus-4p2vl\" (UID: \"8ec840e9-44f1-4daa-963a-96632c87bb8a\") " pod="openshift-multus/multus-4p2vl" Jul 05 13:18:49.657495 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.657115 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/63ac7e2e-3c53-4a93-96e3-c038ed07a986-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xjd4z\" (UID: \"63ac7e2e-3c53-4a93-96e3-c038ed07a986\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjd4z" Jul 05 13:18:49.657495 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.657142 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/33610912-09f3-406f-be0f-a6ffbff4c9ba-dbus\") pod \"global-pull-secret-syncer-l6cc6\" (UID: \"33610912-09f3-406f-be0f-a6ffbff4c9ba\") " pod="kube-system/global-pull-secret-syncer-l6cc6" Jul 05 13:18:49.657495 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.657166 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fafc6427-6250-484c-88b0-d7b08365d04b-etc-openvswitch\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.657495 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.657190 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8ec840e9-44f1-4daa-963a-96632c87bb8a-cni-binary-copy\") pod \"multus-4p2vl\" (UID: \"8ec840e9-44f1-4daa-963a-96632c87bb8a\") " pod="openshift-multus/multus-4p2vl" Jul 05 13:18:49.657495 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.657213 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8ec840e9-44f1-4daa-963a-96632c87bb8a-multus-socket-dir-parent\") pod \"multus-4p2vl\" (UID: \"8ec840e9-44f1-4daa-963a-96632c87bb8a\") " pod="openshift-multus/multus-4p2vl" Jul 05 13:18:49.657495 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.657239 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/63ac7e2e-3c53-4a93-96e3-c038ed07a986-registration-dir\") pod \"aws-ebs-csi-driver-node-xjd4z\" (UID: \"63ac7e2e-3c53-4a93-96e3-c038ed07a986\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjd4z" Jul 05 13:18:49.657495 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.657265 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94h27\" (UniqueName: \"kubernetes.io/projected/63ac7e2e-3c53-4a93-96e3-c038ed07a986-kube-api-access-94h27\") pod \"aws-ebs-csi-driver-node-xjd4z\" (UID: \"63ac7e2e-3c53-4a93-96e3-c038ed07a986\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjd4z" Jul 05 13:18:49.657495 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.657290 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2010fc9b-2713-4036-8131-071a9349d36e-metrics-certs\") pod \"network-metrics-daemon-g9tcx\" (UID: \"2010fc9b-2713-4036-8131-071a9349d36e\") " pod="openshift-multus/network-metrics-daemon-g9tcx" Jul 05 13:18:49.657495 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.657314 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fafc6427-6250-484c-88b0-d7b08365d04b-systemd-units\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.657495 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.657336 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fafc6427-6250-484c-88b0-d7b08365d04b-host-run-netns\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.657495 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.657361 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fafc6427-6250-484c-88b0-d7b08365d04b-ovnkube-config\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.658363 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.657384 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8zcd\" (UniqueName: \"kubernetes.io/projected/8ec840e9-44f1-4daa-963a-96632c87bb8a-kube-api-access-r8zcd\") pod \"multus-4p2vl\" (UID: \"8ec840e9-44f1-4daa-963a-96632c87bb8a\") " pod="openshift-multus/multus-4p2vl" Jul 05 13:18:49.658363 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.657407 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2lh9\" (UniqueName: \"kubernetes.io/projected/7d069d57-4ff7-4edd-94b6-10a12329f310-kube-api-access-n2lh9\") pod \"node-resolver-j5pwd\" (UID: \"7d069d57-4ff7-4edd-94b6-10a12329f310\") " pod="openshift-dns/node-resolver-j5pwd" Jul 05 13:18:49.658363 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.657429 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/63ac7e2e-3c53-4a93-96e3-c038ed07a986-sys-fs\") pod \"aws-ebs-csi-driver-node-xjd4z\" (UID: \"63ac7e2e-3c53-4a93-96e3-c038ed07a986\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjd4z" Jul 05 13:18:49.658363 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.657452 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6b401f24-d573-41fc-8668-0fd9d69e2cd3-etc-tuned\") pod \"tuned-bh9gk\" (UID: \"6b401f24-d573-41fc-8668-0fd9d69e2cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-bh9gk" Jul 05 13:18:49.658363 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.657470 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/33610912-09f3-406f-be0f-a6ffbff4c9ba-dbus\") pod \"global-pull-secret-syncer-l6cc6\" (UID: \"33610912-09f3-406f-be0f-a6ffbff4c9ba\") " pod="kube-system/global-pull-secret-syncer-l6cc6" Jul 05 13:18:49.658363 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.657477 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fafc6427-6250-484c-88b0-d7b08365d04b-run-openvswitch\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.658363 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.657495 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fafc6427-6250-484c-88b0-d7b08365d04b-etc-openvswitch\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.658363 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.657514 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fafc6427-6250-484c-88b0-d7b08365d04b-host-run-ovn-kubernetes\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.658363 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.657524 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fafc6427-6250-484c-88b0-d7b08365d04b-run-openvswitch\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.658363 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.657552 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fafc6427-6250-484c-88b0-d7b08365d04b-host-run-netns\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.658363 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.657556 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/91b3802b-db71-4a40-ad33-0f2ced5ef31d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fw42d\" (UID: \"91b3802b-db71-4a40-ad33-0f2ced5ef31d\") " pod="openshift-multus/multus-additional-cni-plugins-fw42d" Jul 05 13:18:49.658363 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.657561 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fafc6427-6250-484c-88b0-d7b08365d04b-host-run-ovn-kubernetes\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.658363 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.657593 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bkzq\" (UniqueName: \"kubernetes.io/projected/91b3802b-db71-4a40-ad33-0f2ced5ef31d-kube-api-access-8bkzq\") pod \"multus-additional-cni-plugins-fw42d\" (UID: \"91b3802b-db71-4a40-ad33-0f2ced5ef31d\") " pod="openshift-multus/multus-additional-cni-plugins-fw42d" Jul 05 13:18:49.658363 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.657618 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2a975403-6fed-4035-bf2d-c161d33baf8e-host\") pod \"node-ca-hlvjf\" (UID: \"2a975403-6fed-4035-bf2d-c161d33baf8e\") " pod="openshift-image-registry/node-ca-hlvjf" Jul 05 13:18:49.658363 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.657642 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fafc6427-6250-484c-88b0-d7b08365d04b-var-lib-openvswitch\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.658363 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.657666 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6b401f24-d573-41fc-8668-0fd9d69e2cd3-etc-sysctl-d\") pod \"tuned-bh9gk\" (UID: \"6b401f24-d573-41fc-8668-0fd9d69e2cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-bh9gk" Jul 05 13:18:49.658363 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.657689 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6b401f24-d573-41fc-8668-0fd9d69e2cd3-lib-modules\") pod \"tuned-bh9gk\" (UID: \"6b401f24-d573-41fc-8668-0fd9d69e2cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-bh9gk" Jul 05 13:18:49.659253 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.657711 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6b401f24-d573-41fc-8668-0fd9d69e2cd3-var-lib-kubelet\") pod \"tuned-bh9gk\" (UID: \"6b401f24-d573-41fc-8668-0fd9d69e2cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-bh9gk" Jul 05 13:18:49.659253 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.657737 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fafc6427-6250-484c-88b0-d7b08365d04b-run-systemd\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.659253 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.657764 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fafc6427-6250-484c-88b0-d7b08365d04b-ovnkube-script-lib\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.659253 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.657789 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s65wl\" (UniqueName: \"kubernetes.io/projected/93c7227a-9d34-4565-abe7-ff14cddfe6b2-kube-api-access-s65wl\") pod \"iptables-alerter-tpn2j\" (UID: \"93c7227a-9d34-4565-abe7-ff14cddfe6b2\") " pod="openshift-network-operator/iptables-alerter-tpn2j" Jul 05 13:18:49.659253 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.657813 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6b401f24-d573-41fc-8668-0fd9d69e2cd3-etc-systemd\") pod \"tuned-bh9gk\" (UID: \"6b401f24-d573-41fc-8668-0fd9d69e2cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-bh9gk" Jul 05 13:18:49.659253 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.657859 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2a975403-6fed-4035-bf2d-c161d33baf8e-host\") pod \"node-ca-hlvjf\" (UID: \"2a975403-6fed-4035-bf2d-c161d33baf8e\") " pod="openshift-image-registry/node-ca-hlvjf" Jul 05 13:18:49.659253 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.657868 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fafc6427-6250-484c-88b0-d7b08365d04b-var-lib-openvswitch\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.659253 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.657896 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d069d57-4ff7-4edd-94b6-10a12329f310-tmp-dir\") pod \"node-resolver-j5pwd\" (UID: \"7d069d57-4ff7-4edd-94b6-10a12329f310\") " pod="openshift-dns/node-resolver-j5pwd" Jul 05 13:18:49.659253 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.657921 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fafc6427-6250-484c-88b0-d7b08365d04b-run-systemd\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.659253 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.657923 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/63ac7e2e-3c53-4a93-96e3-c038ed07a986-socket-dir\") pod \"aws-ebs-csi-driver-node-xjd4z\" (UID: \"63ac7e2e-3c53-4a93-96e3-c038ed07a986\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjd4z" Jul 05 13:18:49.659253 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.657977 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fafc6427-6250-484c-88b0-d7b08365d04b-host-kubelet\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.659253 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.658026 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fafc6427-6250-484c-88b0-d7b08365d04b-log-socket\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.659253 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.658055 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fafc6427-6250-484c-88b0-d7b08365d04b-host-kubelet\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.659253 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.658088 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fafc6427-6250-484c-88b0-d7b08365d04b-log-socket\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.659253 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:49.658144 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:18:49.659253 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.658153 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fafc6427-6250-484c-88b0-d7b08365d04b-ovnkube-config\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.659253 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.658184 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/91b3802b-db71-4a40-ad33-0f2ced5ef31d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fw42d\" (UID: \"91b3802b-db71-4a40-ad33-0f2ced5ef31d\") " pod="openshift-multus/multus-additional-cni-plugins-fw42d" Jul 05 13:18:49.660029 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:49.658247 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2010fc9b-2713-4036-8131-071a9349d36e-metrics-certs podName:2010fc9b-2713-4036-8131-071a9349d36e nodeName:}" failed. No retries permitted until 2026-07-05 13:18:50.158189174 +0000 UTC m=+3.053847259 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2010fc9b-2713-4036-8131-071a9349d36e-metrics-certs") pod "network-metrics-daemon-g9tcx" (UID: "2010fc9b-2713-4036-8131-071a9349d36e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:18:49.660029 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.658273 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fafc6427-6250-484c-88b0-d7b08365d04b-host-cni-netd\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.660029 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.658280 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fafc6427-6250-484c-88b0-d7b08365d04b-systemd-units\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.660029 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.658321 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fafc6427-6250-484c-88b0-d7b08365d04b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.660029 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.658348 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6b401f24-d573-41fc-8668-0fd9d69e2cd3-etc-kubernetes\") pod \"tuned-bh9gk\" (UID: \"6b401f24-d573-41fc-8668-0fd9d69e2cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-bh9gk" Jul 05 13:18:49.660029 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.658381 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6b401f24-d573-41fc-8668-0fd9d69e2cd3-host\") pod \"tuned-bh9gk\" (UID: \"6b401f24-d573-41fc-8668-0fd9d69e2cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-bh9gk" Jul 05 13:18:49.660029 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.658383 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fafc6427-6250-484c-88b0-d7b08365d04b-host-cni-netd\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.660029 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.658413 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fafc6427-6250-484c-88b0-d7b08365d04b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.660029 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.658427 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8ec840e9-44f1-4daa-963a-96632c87bb8a-cnibin\") pod \"multus-4p2vl\" (UID: \"8ec840e9-44f1-4daa-963a-96632c87bb8a\") " pod="openshift-multus/multus-4p2vl" Jul 05 13:18:49.660029 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.658490 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8ec840e9-44f1-4daa-963a-96632c87bb8a-host-run-k8s-cni-cncf-io\") pod \"multus-4p2vl\" (UID: \"8ec840e9-44f1-4daa-963a-96632c87bb8a\") " pod="openshift-multus/multus-4p2vl" Jul 05 13:18:49.660029 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.658536 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/33610912-09f3-406f-be0f-a6ffbff4c9ba-original-pull-secret\") pod \"global-pull-secret-syncer-l6cc6\" (UID: \"33610912-09f3-406f-be0f-a6ffbff4c9ba\") " pod="kube-system/global-pull-secret-syncer-l6cc6" Jul 05 13:18:49.660029 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.658566 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fafc6427-6250-484c-88b0-d7b08365d04b-run-ovn\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.660029 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.658591 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bd7w2\" (UniqueName: \"kubernetes.io/projected/6b401f24-d573-41fc-8668-0fd9d69e2cd3-kube-api-access-bd7w2\") pod \"tuned-bh9gk\" (UID: \"6b401f24-d573-41fc-8668-0fd9d69e2cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-bh9gk" Jul 05 13:18:49.660029 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.658618 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8ec840e9-44f1-4daa-963a-96632c87bb8a-host-var-lib-cni-multus\") pod \"multus-4p2vl\" (UID: \"8ec840e9-44f1-4daa-963a-96632c87bb8a\") " pod="openshift-multus/multus-4p2vl" Jul 05 13:18:49.660029 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.658641 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/514779be-55a5-41cb-9bce-42dd1c7398e5-agent-certs\") pod \"konnectivity-agent-dr5b4\" (UID: \"514779be-55a5-41cb-9bce-42dd1c7398e5\") " pod="kube-system/konnectivity-agent-dr5b4" Jul 05 13:18:49.660029 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.658646 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fafc6427-6250-484c-88b0-d7b08365d04b-run-ovn\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.660856 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.658664 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/63ac7e2e-3c53-4a93-96e3-c038ed07a986-device-dir\") pod \"aws-ebs-csi-driver-node-xjd4z\" (UID: \"63ac7e2e-3c53-4a93-96e3-c038ed07a986\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjd4z" Jul 05 13:18:49.660856 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.658698 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fafc6427-6250-484c-88b0-d7b08365d04b-env-overrides\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.660856 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.658726 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6b401f24-d573-41fc-8668-0fd9d69e2cd3-etc-modprobe-d\") pod \"tuned-bh9gk\" (UID: \"6b401f24-d573-41fc-8668-0fd9d69e2cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-bh9gk" Jul 05 13:18:49.660856 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.658750 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8ec840e9-44f1-4daa-963a-96632c87bb8a-host-run-multus-certs\") pod \"multus-4p2vl\" (UID: \"8ec840e9-44f1-4daa-963a-96632c87bb8a\") " pod="openshift-multus/multus-4p2vl" Jul 05 13:18:49.660856 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.658773 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8ec840e9-44f1-4daa-963a-96632c87bb8a-etc-kubernetes\") pod \"multus-4p2vl\" (UID: \"8ec840e9-44f1-4daa-963a-96632c87bb8a\") " pod="openshift-multus/multus-4p2vl" Jul 05 13:18:49.660856 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.658796 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fafc6427-6250-484c-88b0-d7b08365d04b-host-slash\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.660856 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.658818 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/91b3802b-db71-4a40-ad33-0f2ced5ef31d-cnibin\") pod \"multus-additional-cni-plugins-fw42d\" (UID: \"91b3802b-db71-4a40-ad33-0f2ced5ef31d\") " pod="openshift-multus/multus-additional-cni-plugins-fw42d" Jul 05 13:18:49.660856 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.658840 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6b401f24-d573-41fc-8668-0fd9d69e2cd3-etc-sysctl-conf\") pod \"tuned-bh9gk\" (UID: \"6b401f24-d573-41fc-8668-0fd9d69e2cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-bh9gk" Jul 05 13:18:49.660856 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.658864 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8ec840e9-44f1-4daa-963a-96632c87bb8a-host-var-lib-kubelet\") pod \"multus-4p2vl\" (UID: \"8ec840e9-44f1-4daa-963a-96632c87bb8a\") " pod="openshift-multus/multus-4p2vl" Jul 05 13:18:49.660856 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.658886 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8ec840e9-44f1-4daa-963a-96632c87bb8a-hostroot\") pod \"multus-4p2vl\" (UID: \"8ec840e9-44f1-4daa-963a-96632c87bb8a\") " pod="openshift-multus/multus-4p2vl" Jul 05 13:18:49.660856 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.658909 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/33610912-09f3-406f-be0f-a6ffbff4c9ba-kubelet-config\") pod \"global-pull-secret-syncer-l6cc6\" (UID: \"33610912-09f3-406f-be0f-a6ffbff4c9ba\") " pod="kube-system/global-pull-secret-syncer-l6cc6" Jul 05 13:18:49.660856 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.658935 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwsk5\" (UniqueName: \"kubernetes.io/projected/fafc6427-6250-484c-88b0-d7b08365d04b-kube-api-access-cwsk5\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.660856 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.658974 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/93c7227a-9d34-4565-abe7-ff14cddfe6b2-host-slash\") pod \"iptables-alerter-tpn2j\" (UID: \"93c7227a-9d34-4565-abe7-ff14cddfe6b2\") " pod="openshift-network-operator/iptables-alerter-tpn2j" Jul 05 13:18:49.660856 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.658998 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8ec840e9-44f1-4daa-963a-96632c87bb8a-host-var-lib-cni-bin\") pod \"multus-4p2vl\" (UID: \"8ec840e9-44f1-4daa-963a-96632c87bb8a\") " pod="openshift-multus/multus-4p2vl" Jul 05 13:18:49.660856 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.659020 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8ec840e9-44f1-4daa-963a-96632c87bb8a-multus-conf-dir\") pod \"multus-4p2vl\" (UID: \"8ec840e9-44f1-4daa-963a-96632c87bb8a\") " pod="openshift-multus/multus-4p2vl" Jul 05 13:18:49.660856 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.659070 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7d069d57-4ff7-4edd-94b6-10a12329f310-hosts-file\") pod \"node-resolver-j5pwd\" (UID: \"7d069d57-4ff7-4edd-94b6-10a12329f310\") " pod="openshift-dns/node-resolver-j5pwd" Jul 05 13:18:49.660856 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.659091 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fafc6427-6250-484c-88b0-d7b08365d04b-ovnkube-script-lib\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.661657 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.659090 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fafc6427-6250-484c-88b0-d7b08365d04b-env-overrides\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.661657 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.659097 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2a975403-6fed-4035-bf2d-c161d33baf8e-serviceca\") pod \"node-ca-hlvjf\" (UID: \"2a975403-6fed-4035-bf2d-c161d33baf8e\") " pod="openshift-image-registry/node-ca-hlvjf" Jul 05 13:18:49.661657 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.659147 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fafc6427-6250-484c-88b0-d7b08365d04b-host-cni-bin\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.661657 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.659157 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fafc6427-6250-484c-88b0-d7b08365d04b-host-slash\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.661657 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.659210 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/93c7227a-9d34-4565-abe7-ff14cddfe6b2-host-slash\") pod \"iptables-alerter-tpn2j\" (UID: \"93c7227a-9d34-4565-abe7-ff14cddfe6b2\") " pod="openshift-network-operator/iptables-alerter-tpn2j" Jul 05 13:18:49.661657 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.659212 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/33610912-09f3-406f-be0f-a6ffbff4c9ba-kubelet-config\") pod \"global-pull-secret-syncer-l6cc6\" (UID: \"33610912-09f3-406f-be0f-a6ffbff4c9ba\") " pod="kube-system/global-pull-secret-syncer-l6cc6" Jul 05 13:18:49.661657 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.659249 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/91b3802b-db71-4a40-ad33-0f2ced5ef31d-cni-binary-copy\") pod \"multus-additional-cni-plugins-fw42d\" (UID: \"91b3802b-db71-4a40-ad33-0f2ced5ef31d\") " pod="openshift-multus/multus-additional-cni-plugins-fw42d" Jul 05 13:18:49.661657 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.659279 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/91b3802b-db71-4a40-ad33-0f2ced5ef31d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fw42d\" (UID: \"91b3802b-db71-4a40-ad33-0f2ced5ef31d\") " pod="openshift-multus/multus-additional-cni-plugins-fw42d" Jul 05 13:18:49.661657 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.659306 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/63ac7e2e-3c53-4a93-96e3-c038ed07a986-etc-selinux\") pod \"aws-ebs-csi-driver-node-xjd4z\" (UID: \"63ac7e2e-3c53-4a93-96e3-c038ed07a986\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjd4z" Jul 05 13:18:49.661657 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.659332 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fafc6427-6250-484c-88b0-d7b08365d04b-ovn-node-metrics-cert\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.661657 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.659358 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krvhl\" (UniqueName: \"kubernetes.io/projected/09e0d252-5d66-4697-b469-24d2cdf32679-kube-api-access-krvhl\") pod \"network-check-target-2nr4r\" (UID: \"09e0d252-5d66-4697-b469-24d2cdf32679\") " pod="openshift-network-diagnostics/network-check-target-2nr4r" Jul 05 13:18:49.661657 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.659409 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fafc6427-6250-484c-88b0-d7b08365d04b-host-cni-bin\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.661657 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.659442 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/91b3802b-db71-4a40-ad33-0f2ced5ef31d-cnibin\") pod \"multus-additional-cni-plugins-fw42d\" (UID: \"91b3802b-db71-4a40-ad33-0f2ced5ef31d\") " pod="openshift-multus/multus-additional-cni-plugins-fw42d" Jul 05 13:18:49.661657 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.659520 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6b401f24-d573-41fc-8668-0fd9d69e2cd3-run\") pod \"tuned-bh9gk\" (UID: \"6b401f24-d573-41fc-8668-0fd9d69e2cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-bh9gk" Jul 05 13:18:49.661657 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.659548 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8ec840e9-44f1-4daa-963a-96632c87bb8a-host-run-netns\") pod \"multus-4p2vl\" (UID: \"8ec840e9-44f1-4daa-963a-96632c87bb8a\") " pod="openshift-multus/multus-4p2vl" Jul 05 13:18:49.661657 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.659578 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v58qx\" (UniqueName: \"kubernetes.io/projected/2a975403-6fed-4035-bf2d-c161d33baf8e-kube-api-access-v58qx\") pod \"node-ca-hlvjf\" (UID: \"2a975403-6fed-4035-bf2d-c161d33baf8e\") " pod="openshift-image-registry/node-ca-hlvjf" Jul 05 13:18:49.661657 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.659592 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2a975403-6fed-4035-bf2d-c161d33baf8e-serviceca\") pod \"node-ca-hlvjf\" (UID: \"2a975403-6fed-4035-bf2d-c161d33baf8e\") " pod="openshift-image-registry/node-ca-hlvjf" Jul 05 13:18:49.662503 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.659625 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8ec840e9-44f1-4daa-963a-96632c87bb8a-os-release\") pod \"multus-4p2vl\" (UID: \"8ec840e9-44f1-4daa-963a-96632c87bb8a\") " pod="openshift-multus/multus-4p2vl" Jul 05 13:18:49.662503 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.659653 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5lpx\" (UniqueName: \"kubernetes.io/projected/2010fc9b-2713-4036-8131-071a9349d36e-kube-api-access-f5lpx\") pod \"network-metrics-daemon-g9tcx\" (UID: \"2010fc9b-2713-4036-8131-071a9349d36e\") " pod="openshift-multus/network-metrics-daemon-g9tcx" Jul 05 13:18:49.662503 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.659681 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/93c7227a-9d34-4565-abe7-ff14cddfe6b2-iptables-alerter-script\") pod \"iptables-alerter-tpn2j\" (UID: \"93c7227a-9d34-4565-abe7-ff14cddfe6b2\") " pod="openshift-network-operator/iptables-alerter-tpn2j" Jul 05 13:18:49.662503 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.659648 2573 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 05 13:18:49.662503 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.659805 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/91b3802b-db71-4a40-ad33-0f2ced5ef31d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fw42d\" (UID: \"91b3802b-db71-4a40-ad33-0f2ced5ef31d\") " pod="openshift-multus/multus-additional-cni-plugins-fw42d" Jul 05 13:18:49.662503 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.659838 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6b401f24-d573-41fc-8668-0fd9d69e2cd3-etc-sysconfig\") pod \"tuned-bh9gk\" (UID: \"6b401f24-d573-41fc-8668-0fd9d69e2cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-bh9gk" Jul 05 13:18:49.662503 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.659861 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/91b3802b-db71-4a40-ad33-0f2ced5ef31d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fw42d\" (UID: \"91b3802b-db71-4a40-ad33-0f2ced5ef31d\") " pod="openshift-multus/multus-additional-cni-plugins-fw42d" Jul 05 13:18:49.662503 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.659864 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8ec840e9-44f1-4daa-963a-96632c87bb8a-multus-daemon-config\") pod \"multus-4p2vl\" (UID: \"8ec840e9-44f1-4daa-963a-96632c87bb8a\") " pod="openshift-multus/multus-4p2vl" Jul 05 13:18:49.662503 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.659982 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/514779be-55a5-41cb-9bce-42dd1c7398e5-konnectivity-ca\") pod \"konnectivity-agent-dr5b4\" (UID: \"514779be-55a5-41cb-9bce-42dd1c7398e5\") " pod="kube-system/konnectivity-agent-dr5b4" Jul 05 13:18:49.662503 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.660089 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/91b3802b-db71-4a40-ad33-0f2ced5ef31d-system-cni-dir\") pod \"multus-additional-cni-plugins-fw42d\" (UID: \"91b3802b-db71-4a40-ad33-0f2ced5ef31d\") " pod="openshift-multus/multus-additional-cni-plugins-fw42d" Jul 05 13:18:49.662503 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.660120 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/91b3802b-db71-4a40-ad33-0f2ced5ef31d-os-release\") pod \"multus-additional-cni-plugins-fw42d\" (UID: \"91b3802b-db71-4a40-ad33-0f2ced5ef31d\") " pod="openshift-multus/multus-additional-cni-plugins-fw42d" Jul 05 13:18:49.662503 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.660151 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/91b3802b-db71-4a40-ad33-0f2ced5ef31d-cni-binary-copy\") pod \"multus-additional-cni-plugins-fw42d\" (UID: \"91b3802b-db71-4a40-ad33-0f2ced5ef31d\") " pod="openshift-multus/multus-additional-cni-plugins-fw42d" Jul 05 13:18:49.662503 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.660173 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/91b3802b-db71-4a40-ad33-0f2ced5ef31d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fw42d\" (UID: \"91b3802b-db71-4a40-ad33-0f2ced5ef31d\") " pod="openshift-multus/multus-additional-cni-plugins-fw42d" Jul 05 13:18:49.662503 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.660216 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/91b3802b-db71-4a40-ad33-0f2ced5ef31d-system-cni-dir\") pod \"multus-additional-cni-plugins-fw42d\" (UID: \"91b3802b-db71-4a40-ad33-0f2ced5ef31d\") " pod="openshift-multus/multus-additional-cni-plugins-fw42d" Jul 05 13:18:49.662503 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.660225 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/91b3802b-db71-4a40-ad33-0f2ced5ef31d-os-release\") pod \"multus-additional-cni-plugins-fw42d\" (UID: \"91b3802b-db71-4a40-ad33-0f2ced5ef31d\") " pod="openshift-multus/multus-additional-cni-plugins-fw42d" Jul 05 13:18:49.662503 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.660356 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/93c7227a-9d34-4565-abe7-ff14cddfe6b2-iptables-alerter-script\") pod \"iptables-alerter-tpn2j\" (UID: \"93c7227a-9d34-4565-abe7-ff14cddfe6b2\") " pod="openshift-network-operator/iptables-alerter-tpn2j" Jul 05 13:18:49.662503 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:49.660422 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 05 13:18:49.663338 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:49.660472 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33610912-09f3-406f-be0f-a6ffbff4c9ba-original-pull-secret podName:33610912-09f3-406f-be0f-a6ffbff4c9ba nodeName:}" failed. No retries permitted until 2026-07-05 13:18:50.160456657 +0000 UTC m=+3.056114728 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/33610912-09f3-406f-be0f-a6ffbff4c9ba-original-pull-secret") pod "global-pull-secret-syncer-l6cc6" (UID: "33610912-09f3-406f-be0f-a6ffbff4c9ba") : object "kube-system"/"original-pull-secret" not registered Jul 05 13:18:49.666763 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.664105 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fafc6427-6250-484c-88b0-d7b08365d04b-ovn-node-metrics-cert\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.672978 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:49.670822 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 05 13:18:49.672978 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:49.670853 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 05 13:18:49.672978 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:49.670867 2573 projected.go:196] Error preparing data for projected volume kube-api-access-krvhl for pod openshift-network-diagnostics/network-check-target-2nr4r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:18:49.672978 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:49.670927 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/09e0d252-5d66-4697-b469-24d2cdf32679-kube-api-access-krvhl podName:09e0d252-5d66-4697-b469-24d2cdf32679 nodeName:}" failed. No retries permitted until 2026-07-05 13:18:50.170910065 +0000 UTC m=+3.066568154 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-krvhl" (UniqueName: "kubernetes.io/projected/09e0d252-5d66-4697-b469-24d2cdf32679-kube-api-access-krvhl") pod "network-check-target-2nr4r" (UID: "09e0d252-5d66-4697-b469-24d2cdf32679") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:18:49.676116 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.674354 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s65wl\" (UniqueName: \"kubernetes.io/projected/93c7227a-9d34-4565-abe7-ff14cddfe6b2-kube-api-access-s65wl\") pod \"iptables-alerter-tpn2j\" (UID: \"93c7227a-9d34-4565-abe7-ff14cddfe6b2\") " pod="openshift-network-operator/iptables-alerter-tpn2j" Jul 05 13:18:49.676116 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.674398 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwsk5\" (UniqueName: \"kubernetes.io/projected/fafc6427-6250-484c-88b0-d7b08365d04b-kube-api-access-cwsk5\") pod \"ovnkube-node-bzhlz\" (UID: \"fafc6427-6250-484c-88b0-d7b08365d04b\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.676116 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.675871 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5lpx\" (UniqueName: \"kubernetes.io/projected/2010fc9b-2713-4036-8131-071a9349d36e-kube-api-access-f5lpx\") pod \"network-metrics-daemon-g9tcx\" (UID: \"2010fc9b-2713-4036-8131-071a9349d36e\") " pod="openshift-multus/network-metrics-daemon-g9tcx" Jul 05 13:18:49.676330 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.676280 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bkzq\" (UniqueName: \"kubernetes.io/projected/91b3802b-db71-4a40-ad33-0f2ced5ef31d-kube-api-access-8bkzq\") pod \"multus-additional-cni-plugins-fw42d\" (UID: \"91b3802b-db71-4a40-ad33-0f2ced5ef31d\") " pod="openshift-multus/multus-additional-cni-plugins-fw42d" Jul 05 13:18:49.679606 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.679581 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v58qx\" (UniqueName: \"kubernetes.io/projected/2a975403-6fed-4035-bf2d-c161d33baf8e-kube-api-access-v58qx\") pod \"node-ca-hlvjf\" (UID: \"2a975403-6fed-4035-bf2d-c161d33baf8e\") " pod="openshift-image-registry/node-ca-hlvjf" Jul 05 13:18:49.741874 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.741752 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-155.ec2.internal" event={"ID":"f7be285f4bc4ee8d9639260737851004","Type":"ContainerStarted","Data":"d01bae3151ae9d914f606a10d4b25b5ef513081b72c9747d9b79373a15a565ae"} Jul 05 13:18:49.743646 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.743617 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-155.ec2.internal" event={"ID":"2159b56be8d2ffd1bbbc05521c1be8b4","Type":"ContainerStarted","Data":"3865ed202465b3cc474280c61c9d59c108a9a06fc480b082496c3e118d9da478"} Jul 05 13:18:49.760902 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.760874 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6b401f24-d573-41fc-8668-0fd9d69e2cd3-etc-sysctl-d\") pod \"tuned-bh9gk\" (UID: \"6b401f24-d573-41fc-8668-0fd9d69e2cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-bh9gk" Jul 05 13:18:49.761026 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.760917 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6b401f24-d573-41fc-8668-0fd9d69e2cd3-lib-modules\") pod \"tuned-bh9gk\" (UID: \"6b401f24-d573-41fc-8668-0fd9d69e2cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-bh9gk" Jul 05 13:18:49.761026 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.760944 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6b401f24-d573-41fc-8668-0fd9d69e2cd3-var-lib-kubelet\") pod \"tuned-bh9gk\" (UID: \"6b401f24-d573-41fc-8668-0fd9d69e2cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-bh9gk" Jul 05 13:18:49.761026 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.760984 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6b401f24-d573-41fc-8668-0fd9d69e2cd3-etc-systemd\") pod \"tuned-bh9gk\" (UID: \"6b401f24-d573-41fc-8668-0fd9d69e2cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-bh9gk" Jul 05 13:18:49.761026 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.761009 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d069d57-4ff7-4edd-94b6-10a12329f310-tmp-dir\") pod \"node-resolver-j5pwd\" (UID: \"7d069d57-4ff7-4edd-94b6-10a12329f310\") " pod="openshift-dns/node-resolver-j5pwd" Jul 05 13:18:49.761235 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.761031 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/63ac7e2e-3c53-4a93-96e3-c038ed07a986-socket-dir\") pod \"aws-ebs-csi-driver-node-xjd4z\" (UID: \"63ac7e2e-3c53-4a93-96e3-c038ed07a986\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjd4z" Jul 05 13:18:49.761235 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.761059 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6b401f24-d573-41fc-8668-0fd9d69e2cd3-etc-kubernetes\") pod \"tuned-bh9gk\" (UID: \"6b401f24-d573-41fc-8668-0fd9d69e2cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-bh9gk" Jul 05 13:18:49.761235 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.761082 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6b401f24-d573-41fc-8668-0fd9d69e2cd3-host\") pod \"tuned-bh9gk\" (UID: \"6b401f24-d573-41fc-8668-0fd9d69e2cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-bh9gk" Jul 05 13:18:49.761235 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.761104 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8ec840e9-44f1-4daa-963a-96632c87bb8a-cnibin\") pod \"multus-4p2vl\" (UID: \"8ec840e9-44f1-4daa-963a-96632c87bb8a\") " pod="openshift-multus/multus-4p2vl" Jul 05 13:18:49.761235 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.761125 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8ec840e9-44f1-4daa-963a-96632c87bb8a-host-run-k8s-cni-cncf-io\") pod \"multus-4p2vl\" (UID: \"8ec840e9-44f1-4daa-963a-96632c87bb8a\") " pod="openshift-multus/multus-4p2vl" Jul 05 13:18:49.761235 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.761165 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bd7w2\" (UniqueName: \"kubernetes.io/projected/6b401f24-d573-41fc-8668-0fd9d69e2cd3-kube-api-access-bd7w2\") pod \"tuned-bh9gk\" (UID: \"6b401f24-d573-41fc-8668-0fd9d69e2cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-bh9gk" Jul 05 13:18:49.761235 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.761189 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8ec840e9-44f1-4daa-963a-96632c87bb8a-host-var-lib-cni-multus\") pod \"multus-4p2vl\" (UID: \"8ec840e9-44f1-4daa-963a-96632c87bb8a\") " pod="openshift-multus/multus-4p2vl" Jul 05 13:18:49.761235 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.761214 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/514779be-55a5-41cb-9bce-42dd1c7398e5-agent-certs\") pod \"konnectivity-agent-dr5b4\" (UID: \"514779be-55a5-41cb-9bce-42dd1c7398e5\") " pod="kube-system/konnectivity-agent-dr5b4" Jul 05 13:18:49.761235 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.761237 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/63ac7e2e-3c53-4a93-96e3-c038ed07a986-device-dir\") pod \"aws-ebs-csi-driver-node-xjd4z\" (UID: \"63ac7e2e-3c53-4a93-96e3-c038ed07a986\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjd4z" Jul 05 13:18:49.761657 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.761261 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6b401f24-d573-41fc-8668-0fd9d69e2cd3-etc-modprobe-d\") pod \"tuned-bh9gk\" (UID: \"6b401f24-d573-41fc-8668-0fd9d69e2cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-bh9gk" Jul 05 13:18:49.761657 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.761284 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8ec840e9-44f1-4daa-963a-96632c87bb8a-host-run-multus-certs\") pod \"multus-4p2vl\" (UID: \"8ec840e9-44f1-4daa-963a-96632c87bb8a\") " pod="openshift-multus/multus-4p2vl" Jul 05 13:18:49.761657 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.761307 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8ec840e9-44f1-4daa-963a-96632c87bb8a-etc-kubernetes\") pod \"multus-4p2vl\" (UID: \"8ec840e9-44f1-4daa-963a-96632c87bb8a\") " pod="openshift-multus/multus-4p2vl" Jul 05 13:18:49.761657 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.761330 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6b401f24-d573-41fc-8668-0fd9d69e2cd3-etc-sysctl-conf\") pod \"tuned-bh9gk\" (UID: \"6b401f24-d573-41fc-8668-0fd9d69e2cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-bh9gk" Jul 05 13:18:49.761657 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.761351 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/63ac7e2e-3c53-4a93-96e3-c038ed07a986-socket-dir\") pod \"aws-ebs-csi-driver-node-xjd4z\" (UID: \"63ac7e2e-3c53-4a93-96e3-c038ed07a986\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjd4z" Jul 05 13:18:49.761657 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.761400 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8ec840e9-44f1-4daa-963a-96632c87bb8a-host-var-lib-kubelet\") pod \"multus-4p2vl\" (UID: \"8ec840e9-44f1-4daa-963a-96632c87bb8a\") " pod="openshift-multus/multus-4p2vl" Jul 05 13:18:49.761657 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.761413 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6b401f24-d573-41fc-8668-0fd9d69e2cd3-host\") pod \"tuned-bh9gk\" (UID: \"6b401f24-d573-41fc-8668-0fd9d69e2cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-bh9gk" Jul 05 13:18:49.761657 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.761441 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6b401f24-d573-41fc-8668-0fd9d69e2cd3-etc-kubernetes\") pod \"tuned-bh9gk\" (UID: \"6b401f24-d573-41fc-8668-0fd9d69e2cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-bh9gk" Jul 05 13:18:49.761657 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.761472 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8ec840e9-44f1-4daa-963a-96632c87bb8a-cnibin\") pod \"multus-4p2vl\" (UID: \"8ec840e9-44f1-4daa-963a-96632c87bb8a\") " pod="openshift-multus/multus-4p2vl" Jul 05 13:18:49.761657 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.761505 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8ec840e9-44f1-4daa-963a-96632c87bb8a-host-run-k8s-cni-cncf-io\") pod \"multus-4p2vl\" (UID: \"8ec840e9-44f1-4daa-963a-96632c87bb8a\") " pod="openshift-multus/multus-4p2vl" Jul 05 13:18:49.761657 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.761551 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6b401f24-d573-41fc-8668-0fd9d69e2cd3-etc-sysctl-d\") pod \"tuned-bh9gk\" (UID: \"6b401f24-d573-41fc-8668-0fd9d69e2cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-bh9gk" Jul 05 13:18:49.761657 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.761636 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6b401f24-d573-41fc-8668-0fd9d69e2cd3-lib-modules\") pod \"tuned-bh9gk\" (UID: \"6b401f24-d573-41fc-8668-0fd9d69e2cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-bh9gk" Jul 05 13:18:49.762223 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.761680 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6b401f24-d573-41fc-8668-0fd9d69e2cd3-var-lib-kubelet\") pod \"tuned-bh9gk\" (UID: \"6b401f24-d573-41fc-8668-0fd9d69e2cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-bh9gk" Jul 05 13:18:49.762223 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.761716 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6b401f24-d573-41fc-8668-0fd9d69e2cd3-etc-systemd\") pod \"tuned-bh9gk\" (UID: \"6b401f24-d573-41fc-8668-0fd9d69e2cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-bh9gk" Jul 05 13:18:49.762223 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.761946 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d069d57-4ff7-4edd-94b6-10a12329f310-tmp-dir\") pod \"node-resolver-j5pwd\" (UID: \"7d069d57-4ff7-4edd-94b6-10a12329f310\") " pod="openshift-dns/node-resolver-j5pwd" Jul 05 13:18:49.762223 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.761352 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8ec840e9-44f1-4daa-963a-96632c87bb8a-host-var-lib-kubelet\") pod \"multus-4p2vl\" (UID: \"8ec840e9-44f1-4daa-963a-96632c87bb8a\") " pod="openshift-multus/multus-4p2vl" Jul 05 13:18:49.762223 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.762019 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8ec840e9-44f1-4daa-963a-96632c87bb8a-host-var-lib-cni-multus\") pod \"multus-4p2vl\" (UID: \"8ec840e9-44f1-4daa-963a-96632c87bb8a\") " pod="openshift-multus/multus-4p2vl" Jul 05 13:18:49.762223 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.762029 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8ec840e9-44f1-4daa-963a-96632c87bb8a-hostroot\") pod \"multus-4p2vl\" (UID: \"8ec840e9-44f1-4daa-963a-96632c87bb8a\") " pod="openshift-multus/multus-4p2vl" Jul 05 13:18:49.762223 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.762064 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8ec840e9-44f1-4daa-963a-96632c87bb8a-host-var-lib-cni-bin\") pod \"multus-4p2vl\" (UID: \"8ec840e9-44f1-4daa-963a-96632c87bb8a\") " pod="openshift-multus/multus-4p2vl" Jul 05 13:18:49.762223 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.762087 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8ec840e9-44f1-4daa-963a-96632c87bb8a-multus-conf-dir\") pod \"multus-4p2vl\" (UID: \"8ec840e9-44f1-4daa-963a-96632c87bb8a\") " pod="openshift-multus/multus-4p2vl" Jul 05 13:18:49.762223 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.762108 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7d069d57-4ff7-4edd-94b6-10a12329f310-hosts-file\") pod \"node-resolver-j5pwd\" (UID: \"7d069d57-4ff7-4edd-94b6-10a12329f310\") " pod="openshift-dns/node-resolver-j5pwd" Jul 05 13:18:49.762223 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.762136 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/63ac7e2e-3c53-4a93-96e3-c038ed07a986-etc-selinux\") pod \"aws-ebs-csi-driver-node-xjd4z\" (UID: \"63ac7e2e-3c53-4a93-96e3-c038ed07a986\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjd4z" Jul 05 13:18:49.762223 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.762169 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6b401f24-d573-41fc-8668-0fd9d69e2cd3-run\") pod \"tuned-bh9gk\" (UID: \"6b401f24-d573-41fc-8668-0fd9d69e2cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-bh9gk" Jul 05 13:18:49.762223 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.762191 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8ec840e9-44f1-4daa-963a-96632c87bb8a-host-run-netns\") pod \"multus-4p2vl\" (UID: \"8ec840e9-44f1-4daa-963a-96632c87bb8a\") " pod="openshift-multus/multus-4p2vl" Jul 05 13:18:49.762223 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.762215 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8ec840e9-44f1-4daa-963a-96632c87bb8a-os-release\") pod \"multus-4p2vl\" (UID: \"8ec840e9-44f1-4daa-963a-96632c87bb8a\") " pod="openshift-multus/multus-4p2vl" Jul 05 13:18:49.762822 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.762257 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6b401f24-d573-41fc-8668-0fd9d69e2cd3-etc-sysconfig\") pod \"tuned-bh9gk\" (UID: \"6b401f24-d573-41fc-8668-0fd9d69e2cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-bh9gk" Jul 05 13:18:49.762822 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.762285 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8ec840e9-44f1-4daa-963a-96632c87bb8a-multus-daemon-config\") pod \"multus-4p2vl\" (UID: \"8ec840e9-44f1-4daa-963a-96632c87bb8a\") " pod="openshift-multus/multus-4p2vl" Jul 05 13:18:49.762822 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.762308 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/514779be-55a5-41cb-9bce-42dd1c7398e5-konnectivity-ca\") pod \"konnectivity-agent-dr5b4\" (UID: \"514779be-55a5-41cb-9bce-42dd1c7398e5\") " pod="kube-system/konnectivity-agent-dr5b4" Jul 05 13:18:49.762822 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.762333 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6b401f24-d573-41fc-8668-0fd9d69e2cd3-sys\") pod \"tuned-bh9gk\" (UID: \"6b401f24-d573-41fc-8668-0fd9d69e2cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-bh9gk" Jul 05 13:18:49.762822 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.762356 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6b401f24-d573-41fc-8668-0fd9d69e2cd3-tmp\") pod \"tuned-bh9gk\" (UID: \"6b401f24-d573-41fc-8668-0fd9d69e2cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-bh9gk" Jul 05 13:18:49.762822 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.762376 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8ec840e9-44f1-4daa-963a-96632c87bb8a-system-cni-dir\") pod \"multus-4p2vl\" (UID: \"8ec840e9-44f1-4daa-963a-96632c87bb8a\") " pod="openshift-multus/multus-4p2vl" Jul 05 13:18:49.762822 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.762424 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8ec840e9-44f1-4daa-963a-96632c87bb8a-multus-cni-dir\") pod \"multus-4p2vl\" (UID: \"8ec840e9-44f1-4daa-963a-96632c87bb8a\") " pod="openshift-multus/multus-4p2vl" Jul 05 13:18:49.762822 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.762448 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/63ac7e2e-3c53-4a93-96e3-c038ed07a986-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xjd4z\" (UID: \"63ac7e2e-3c53-4a93-96e3-c038ed07a986\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjd4z" Jul 05 13:18:49.762822 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.762473 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8ec840e9-44f1-4daa-963a-96632c87bb8a-cni-binary-copy\") pod \"multus-4p2vl\" (UID: \"8ec840e9-44f1-4daa-963a-96632c87bb8a\") " pod="openshift-multus/multus-4p2vl" Jul 05 13:18:49.762822 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.762495 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8ec840e9-44f1-4daa-963a-96632c87bb8a-multus-socket-dir-parent\") pod \"multus-4p2vl\" (UID: \"8ec840e9-44f1-4daa-963a-96632c87bb8a\") " pod="openshift-multus/multus-4p2vl" Jul 05 13:18:49.762822 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.762517 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/63ac7e2e-3c53-4a93-96e3-c038ed07a986-registration-dir\") pod \"aws-ebs-csi-driver-node-xjd4z\" (UID: \"63ac7e2e-3c53-4a93-96e3-c038ed07a986\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjd4z" Jul 05 13:18:49.762822 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.762545 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94h27\" (UniqueName: \"kubernetes.io/projected/63ac7e2e-3c53-4a93-96e3-c038ed07a986-kube-api-access-94h27\") pod \"aws-ebs-csi-driver-node-xjd4z\" (UID: \"63ac7e2e-3c53-4a93-96e3-c038ed07a986\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjd4z" Jul 05 13:18:49.762822 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.762584 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8zcd\" (UniqueName: \"kubernetes.io/projected/8ec840e9-44f1-4daa-963a-96632c87bb8a-kube-api-access-r8zcd\") pod \"multus-4p2vl\" (UID: \"8ec840e9-44f1-4daa-963a-96632c87bb8a\") " pod="openshift-multus/multus-4p2vl" Jul 05 13:18:49.762822 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.762610 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2lh9\" (UniqueName: \"kubernetes.io/projected/7d069d57-4ff7-4edd-94b6-10a12329f310-kube-api-access-n2lh9\") pod \"node-resolver-j5pwd\" (UID: \"7d069d57-4ff7-4edd-94b6-10a12329f310\") " pod="openshift-dns/node-resolver-j5pwd" Jul 05 13:18:49.762822 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.762635 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/63ac7e2e-3c53-4a93-96e3-c038ed07a986-sys-fs\") pod \"aws-ebs-csi-driver-node-xjd4z\" (UID: \"63ac7e2e-3c53-4a93-96e3-c038ed07a986\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjd4z" Jul 05 13:18:49.762822 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.762668 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6b401f24-d573-41fc-8668-0fd9d69e2cd3-etc-tuned\") pod \"tuned-bh9gk\" (UID: \"6b401f24-d573-41fc-8668-0fd9d69e2cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-bh9gk" Jul 05 13:18:49.763566 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.763190 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/63ac7e2e-3c53-4a93-96e3-c038ed07a986-device-dir\") pod \"aws-ebs-csi-driver-node-xjd4z\" (UID: \"63ac7e2e-3c53-4a93-96e3-c038ed07a986\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjd4z" Jul 05 13:18:49.763566 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.763310 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6b401f24-d573-41fc-8668-0fd9d69e2cd3-etc-modprobe-d\") pod \"tuned-bh9gk\" (UID: \"6b401f24-d573-41fc-8668-0fd9d69e2cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-bh9gk" Jul 05 13:18:49.763566 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.763357 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8ec840e9-44f1-4daa-963a-96632c87bb8a-host-run-multus-certs\") pod \"multus-4p2vl\" (UID: \"8ec840e9-44f1-4daa-963a-96632c87bb8a\") " pod="openshift-multus/multus-4p2vl" Jul 05 13:18:49.763566 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.763371 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8ec840e9-44f1-4daa-963a-96632c87bb8a-hostroot\") pod \"multus-4p2vl\" (UID: \"8ec840e9-44f1-4daa-963a-96632c87bb8a\") " pod="openshift-multus/multus-4p2vl" Jul 05 13:18:49.763566 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.763397 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8ec840e9-44f1-4daa-963a-96632c87bb8a-etc-kubernetes\") pod \"multus-4p2vl\" (UID: \"8ec840e9-44f1-4daa-963a-96632c87bb8a\") " pod="openshift-multus/multus-4p2vl" Jul 05 13:18:49.763566 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.763414 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8ec840e9-44f1-4daa-963a-96632c87bb8a-host-var-lib-cni-bin\") pod \"multus-4p2vl\" (UID: \"8ec840e9-44f1-4daa-963a-96632c87bb8a\") " pod="openshift-multus/multus-4p2vl" Jul 05 13:18:49.763566 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.763446 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8ec840e9-44f1-4daa-963a-96632c87bb8a-multus-conf-dir\") pod \"multus-4p2vl\" (UID: \"8ec840e9-44f1-4daa-963a-96632c87bb8a\") " pod="openshift-multus/multus-4p2vl" Jul 05 13:18:49.763566 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.763481 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7d069d57-4ff7-4edd-94b6-10a12329f310-hosts-file\") pod \"node-resolver-j5pwd\" (UID: \"7d069d57-4ff7-4edd-94b6-10a12329f310\") " pod="openshift-dns/node-resolver-j5pwd" Jul 05 13:18:49.763566 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.763507 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6b401f24-d573-41fc-8668-0fd9d69e2cd3-etc-sysctl-conf\") pod \"tuned-bh9gk\" (UID: \"6b401f24-d573-41fc-8668-0fd9d69e2cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-bh9gk" Jul 05 13:18:49.763566 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.763550 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8ec840e9-44f1-4daa-963a-96632c87bb8a-system-cni-dir\") pod \"multus-4p2vl\" (UID: \"8ec840e9-44f1-4daa-963a-96632c87bb8a\") " pod="openshift-multus/multus-4p2vl" Jul 05 13:18:49.764576 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.763578 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/63ac7e2e-3c53-4a93-96e3-c038ed07a986-etc-selinux\") pod \"aws-ebs-csi-driver-node-xjd4z\" (UID: \"63ac7e2e-3c53-4a93-96e3-c038ed07a986\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjd4z" Jul 05 13:18:49.764576 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.763619 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6b401f24-d573-41fc-8668-0fd9d69e2cd3-run\") pod \"tuned-bh9gk\" (UID: \"6b401f24-d573-41fc-8668-0fd9d69e2cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-bh9gk" Jul 05 13:18:49.764576 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.763635 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8ec840e9-44f1-4daa-963a-96632c87bb8a-multus-cni-dir\") pod \"multus-4p2vl\" (UID: \"8ec840e9-44f1-4daa-963a-96632c87bb8a\") " pod="openshift-multus/multus-4p2vl" Jul 05 13:18:49.764576 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.763657 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8ec840e9-44f1-4daa-963a-96632c87bb8a-host-run-netns\") pod \"multus-4p2vl\" (UID: \"8ec840e9-44f1-4daa-963a-96632c87bb8a\") " pod="openshift-multus/multus-4p2vl" Jul 05 13:18:49.764576 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.763711 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8ec840e9-44f1-4daa-963a-96632c87bb8a-os-release\") pod \"multus-4p2vl\" (UID: \"8ec840e9-44f1-4daa-963a-96632c87bb8a\") " pod="openshift-multus/multus-4p2vl" Jul 05 13:18:49.764576 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.763753 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/63ac7e2e-3c53-4a93-96e3-c038ed07a986-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xjd4z\" (UID: \"63ac7e2e-3c53-4a93-96e3-c038ed07a986\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjd4z" Jul 05 13:18:49.764576 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.763793 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6b401f24-d573-41fc-8668-0fd9d69e2cd3-etc-sysconfig\") pod \"tuned-bh9gk\" (UID: \"6b401f24-d573-41fc-8668-0fd9d69e2cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-bh9gk" Jul 05 13:18:49.764576 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.764039 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6b401f24-d573-41fc-8668-0fd9d69e2cd3-sys\") pod \"tuned-bh9gk\" (UID: \"6b401f24-d573-41fc-8668-0fd9d69e2cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-bh9gk" Jul 05 13:18:49.764576 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.764205 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8ec840e9-44f1-4daa-963a-96632c87bb8a-cni-binary-copy\") pod \"multus-4p2vl\" (UID: \"8ec840e9-44f1-4daa-963a-96632c87bb8a\") " pod="openshift-multus/multus-4p2vl" Jul 05 13:18:49.764576 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.764282 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8ec840e9-44f1-4daa-963a-96632c87bb8a-multus-socket-dir-parent\") pod \"multus-4p2vl\" (UID: \"8ec840e9-44f1-4daa-963a-96632c87bb8a\") " pod="openshift-multus/multus-4p2vl" Jul 05 13:18:49.764576 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.764290 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8ec840e9-44f1-4daa-963a-96632c87bb8a-multus-daemon-config\") pod \"multus-4p2vl\" (UID: \"8ec840e9-44f1-4daa-963a-96632c87bb8a\") " pod="openshift-multus/multus-4p2vl" Jul 05 13:18:49.764576 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.764338 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/63ac7e2e-3c53-4a93-96e3-c038ed07a986-registration-dir\") pod \"aws-ebs-csi-driver-node-xjd4z\" (UID: \"63ac7e2e-3c53-4a93-96e3-c038ed07a986\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjd4z" Jul 05 13:18:49.764576 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.764558 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/63ac7e2e-3c53-4a93-96e3-c038ed07a986-sys-fs\") pod \"aws-ebs-csi-driver-node-xjd4z\" (UID: \"63ac7e2e-3c53-4a93-96e3-c038ed07a986\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjd4z" Jul 05 13:18:49.764576 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.764575 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/514779be-55a5-41cb-9bce-42dd1c7398e5-konnectivity-ca\") pod \"konnectivity-agent-dr5b4\" (UID: \"514779be-55a5-41cb-9bce-42dd1c7398e5\") " pod="kube-system/konnectivity-agent-dr5b4" Jul 05 13:18:49.765306 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.764906 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/514779be-55a5-41cb-9bce-42dd1c7398e5-agent-certs\") pod \"konnectivity-agent-dr5b4\" (UID: \"514779be-55a5-41cb-9bce-42dd1c7398e5\") " pod="kube-system/konnectivity-agent-dr5b4" Jul 05 13:18:49.765306 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.764909 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6b401f24-d573-41fc-8668-0fd9d69e2cd3-etc-tuned\") pod \"tuned-bh9gk\" (UID: \"6b401f24-d573-41fc-8668-0fd9d69e2cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-bh9gk" Jul 05 13:18:49.766651 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.766399 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6b401f24-d573-41fc-8668-0fd9d69e2cd3-tmp\") pod \"tuned-bh9gk\" (UID: \"6b401f24-d573-41fc-8668-0fd9d69e2cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-bh9gk" Jul 05 13:18:49.771902 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.771568 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bd7w2\" (UniqueName: \"kubernetes.io/projected/6b401f24-d573-41fc-8668-0fd9d69e2cd3-kube-api-access-bd7w2\") pod \"tuned-bh9gk\" (UID: \"6b401f24-d573-41fc-8668-0fd9d69e2cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-bh9gk" Jul 05 13:18:49.771902 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.771862 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2lh9\" (UniqueName: \"kubernetes.io/projected/7d069d57-4ff7-4edd-94b6-10a12329f310-kube-api-access-n2lh9\") pod \"node-resolver-j5pwd\" (UID: \"7d069d57-4ff7-4edd-94b6-10a12329f310\") " pod="openshift-dns/node-resolver-j5pwd" Jul 05 13:18:49.772670 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.772625 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8zcd\" (UniqueName: \"kubernetes.io/projected/8ec840e9-44f1-4daa-963a-96632c87bb8a-kube-api-access-r8zcd\") pod \"multus-4p2vl\" (UID: \"8ec840e9-44f1-4daa-963a-96632c87bb8a\") " pod="openshift-multus/multus-4p2vl" Jul 05 13:18:49.773462 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.773435 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-94h27\" (UniqueName: \"kubernetes.io/projected/63ac7e2e-3c53-4a93-96e3-c038ed07a986-kube-api-access-94h27\") pod \"aws-ebs-csi-driver-node-xjd4z\" (UID: \"63ac7e2e-3c53-4a93-96e3-c038ed07a986\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjd4z" Jul 05 13:18:49.843130 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.843092 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tpn2j" Jul 05 13:18:49.858572 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.858451 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fw42d" Jul 05 13:18:49.868631 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.868610 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-hlvjf" Jul 05 13:18:49.876484 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.876464 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:18:49.885147 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.885125 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bh9gk" Jul 05 13:18:49.894829 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.894811 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4p2vl" Jul 05 13:18:49.902454 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.902428 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dr5b4" Jul 05 13:18:49.910021 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.910004 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-j5pwd" Jul 05 13:18:49.916623 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:49.916604 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjd4z" Jul 05 13:18:50.165562 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:50.165482 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/33610912-09f3-406f-be0f-a6ffbff4c9ba-original-pull-secret\") pod \"global-pull-secret-syncer-l6cc6\" (UID: \"33610912-09f3-406f-be0f-a6ffbff4c9ba\") " pod="kube-system/global-pull-secret-syncer-l6cc6" Jul 05 13:18:50.165562 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:50.165558 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2010fc9b-2713-4036-8131-071a9349d36e-metrics-certs\") pod \"network-metrics-daemon-g9tcx\" (UID: \"2010fc9b-2713-4036-8131-071a9349d36e\") " pod="openshift-multus/network-metrics-daemon-g9tcx" Jul 05 13:18:50.165789 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:50.165648 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 05 13:18:50.165789 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:50.165728 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33610912-09f3-406f-be0f-a6ffbff4c9ba-original-pull-secret podName:33610912-09f3-406f-be0f-a6ffbff4c9ba nodeName:}" failed. No retries permitted until 2026-07-05 13:18:51.165706773 +0000 UTC m=+4.061364855 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/33610912-09f3-406f-be0f-a6ffbff4c9ba-original-pull-secret") pod "global-pull-secret-syncer-l6cc6" (UID: "33610912-09f3-406f-be0f-a6ffbff4c9ba") : object "kube-system"/"original-pull-secret" not registered Jul 05 13:18:50.165789 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:50.165653 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:18:50.165918 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:50.165805 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2010fc9b-2713-4036-8131-071a9349d36e-metrics-certs podName:2010fc9b-2713-4036-8131-071a9349d36e nodeName:}" failed. No retries permitted until 2026-07-05 13:18:51.165786742 +0000 UTC m=+4.061444827 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2010fc9b-2713-4036-8131-071a9349d36e-metrics-certs") pod "network-metrics-daemon-g9tcx" (UID: "2010fc9b-2713-4036-8131-071a9349d36e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:18:50.266258 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:50.266213 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krvhl\" (UniqueName: \"kubernetes.io/projected/09e0d252-5d66-4697-b469-24d2cdf32679-kube-api-access-krvhl\") pod \"network-check-target-2nr4r\" (UID: \"09e0d252-5d66-4697-b469-24d2cdf32679\") " pod="openshift-network-diagnostics/network-check-target-2nr4r" Jul 05 13:18:50.266441 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:50.266378 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 05 13:18:50.266441 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:50.266403 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 05 13:18:50.266441 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:50.266415 2573 projected.go:196] Error preparing data for projected volume kube-api-access-krvhl for pod openshift-network-diagnostics/network-check-target-2nr4r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:18:50.266580 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:50.266472 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/09e0d252-5d66-4697-b469-24d2cdf32679-kube-api-access-krvhl podName:09e0d252-5d66-4697-b469-24d2cdf32679 nodeName:}" failed. No retries permitted until 2026-07-05 13:18:51.266455309 +0000 UTC m=+4.162113402 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-krvhl" (UniqueName: "kubernetes.io/projected/09e0d252-5d66-4697-b469-24d2cdf32679-kube-api-access-krvhl") pod "network-check-target-2nr4r" (UID: "09e0d252-5d66-4697-b469-24d2cdf32679") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:18:50.614914 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:50.614866 2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-04 13:13:49 +0000 UTC" deadline="2028-02-17 05:15:01.680721273 +0000 UTC" Jul 05 13:18:50.614914 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:50.614907 2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14199h56m11.065818628s" Jul 05 13:18:50.714924 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:50.714767 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63ac7e2e_3c53_4a93_96e3_c038ed07a986.slice/crio-9f050f5752b57bddfabc9858e32b122fa2ca73660ccd13b5884a59e08aca5832 WatchSource:0}: Error finding container 9f050f5752b57bddfabc9858e32b122fa2ca73660ccd13b5884a59e08aca5832: Status 404 returned error can't find the container with id 9f050f5752b57bddfabc9858e32b122fa2ca73660ccd13b5884a59e08aca5832 Jul 05 13:18:50.729930 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:50.729907 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l6cc6" Jul 05 13:18:50.730065 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:50.730038 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l6cc6" podUID="33610912-09f3-406f-be0f-a6ffbff4c9ba" Jul 05 13:18:50.739451 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:50.739429 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d069d57_4ff7_4edd_94b6_10a12329f310.slice/crio-74a7ef4bfe2dbbdf244a9c464fb875825222b27bb508d95ea542783b3976c977 WatchSource:0}: Error finding container 74a7ef4bfe2dbbdf244a9c464fb875825222b27bb508d95ea542783b3976c977: Status 404 returned error can't find the container with id 74a7ef4bfe2dbbdf244a9c464fb875825222b27bb508d95ea542783b3976c977 Jul 05 13:18:50.740798 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:50.740778 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfafc6427_6250_484c_88b0_d7b08365d04b.slice/crio-93c5b9a0c6fbd66a43ec649b286e13cb7556f2cd057ca6da1d5f24744f8713b9 WatchSource:0}: Error finding container 93c5b9a0c6fbd66a43ec649b286e13cb7556f2cd057ca6da1d5f24744f8713b9: Status 404 returned error can't find the container with id 93c5b9a0c6fbd66a43ec649b286e13cb7556f2cd057ca6da1d5f24744f8713b9 Jul 05 13:18:50.742208 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:50.742189 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93c7227a_9d34_4565_abe7_ff14cddfe6b2.slice/crio-a568f6826b1e91796066799c87258981db3640fac1227039af64c59234dc2adc WatchSource:0}: Error finding container a568f6826b1e91796066799c87258981db3640fac1227039af64c59234dc2adc: Status 404 returned error can't find the container with id a568f6826b1e91796066799c87258981db3640fac1227039af64c59234dc2adc Jul 05 13:18:50.742890 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:50.742865 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b401f24_d573_41fc_8668_0fd9d69e2cd3.slice/crio-19cd064dc1b888359e344dc512cfd1fcebd707d46cd2e129310d7a3ca8286188 WatchSource:0}: Error finding container 19cd064dc1b888359e344dc512cfd1fcebd707d46cd2e129310d7a3ca8286188: Status 404 returned error can't find the container with id 19cd064dc1b888359e344dc512cfd1fcebd707d46cd2e129310d7a3ca8286188 Jul 05 13:18:50.744341 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:50.744316 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91b3802b_db71_4a40_ad33_0f2ced5ef31d.slice/crio-1b7f1c4b446facd00dfd94e7553eccc0fbd9bba30b207246eab6e7f4fce48da0 WatchSource:0}: Error finding container 1b7f1c4b446facd00dfd94e7553eccc0fbd9bba30b207246eab6e7f4fce48da0: Status 404 returned error can't find the container with id 1b7f1c4b446facd00dfd94e7553eccc0fbd9bba30b207246eab6e7f4fce48da0 Jul 05 13:18:50.745056 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:50.744974 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod514779be_55a5_41cb_9bce_42dd1c7398e5.slice/crio-7723eb4bd409873899eef47abb5de90d82d3770cb53cba59ef1df18fc3880cfe WatchSource:0}: Error finding container 7723eb4bd409873899eef47abb5de90d82d3770cb53cba59ef1df18fc3880cfe: Status 404 returned error can't find the container with id 7723eb4bd409873899eef47abb5de90d82d3770cb53cba59ef1df18fc3880cfe Jul 05 13:18:50.745805 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:50.745713 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bh9gk" event={"ID":"6b401f24-d573-41fc-8668-0fd9d69e2cd3","Type":"ContainerStarted","Data":"19cd064dc1b888359e344dc512cfd1fcebd707d46cd2e129310d7a3ca8286188"} Jul 05 13:18:50.745862 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:18:50.745814 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a975403_6fed_4035_bf2d_c161d33baf8e.slice/crio-2da262e2525c95ca745401e4f6a95fd051f26146d4798a401c2d8121be7ca530 WatchSource:0}: Error finding container 2da262e2525c95ca745401e4f6a95fd051f26146d4798a401c2d8121be7ca530: Status 404 returned error can't find the container with id 2da262e2525c95ca745401e4f6a95fd051f26146d4798a401c2d8121be7ca530 Jul 05 13:18:50.747611 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:50.747556 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tpn2j" event={"ID":"93c7227a-9d34-4565-abe7-ff14cddfe6b2","Type":"ContainerStarted","Data":"a568f6826b1e91796066799c87258981db3640fac1227039af64c59234dc2adc"} Jul 05 13:18:50.749733 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:50.749703 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-j5pwd" event={"ID":"7d069d57-4ff7-4edd-94b6-10a12329f310","Type":"ContainerStarted","Data":"74a7ef4bfe2dbbdf244a9c464fb875825222b27bb508d95ea542783b3976c977"} Jul 05 13:18:50.750987 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:50.750869 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4p2vl" event={"ID":"8ec840e9-44f1-4daa-963a-96632c87bb8a","Type":"ContainerStarted","Data":"188e141dae967d1ac92358ccb776f2e1880e6a130808657cc998f74e3fdb8c05"} Jul 05 13:18:50.754345 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:50.754316 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjd4z" event={"ID":"63ac7e2e-3c53-4a93-96e3-c038ed07a986","Type":"ContainerStarted","Data":"9f050f5752b57bddfabc9858e32b122fa2ca73660ccd13b5884a59e08aca5832"} Jul 05 13:18:50.756172 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:50.756146 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" event={"ID":"fafc6427-6250-484c-88b0-d7b08365d04b","Type":"ContainerStarted","Data":"93c5b9a0c6fbd66a43ec649b286e13cb7556f2cd057ca6da1d5f24744f8713b9"} Jul 05 13:18:51.174064 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:51.173794 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/33610912-09f3-406f-be0f-a6ffbff4c9ba-original-pull-secret\") pod \"global-pull-secret-syncer-l6cc6\" (UID: \"33610912-09f3-406f-be0f-a6ffbff4c9ba\") " pod="kube-system/global-pull-secret-syncer-l6cc6" Jul 05 13:18:51.174064 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:51.174031 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 05 13:18:51.174306 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:51.174212 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33610912-09f3-406f-be0f-a6ffbff4c9ba-original-pull-secret podName:33610912-09f3-406f-be0f-a6ffbff4c9ba nodeName:}" failed. No retries permitted until 2026-07-05 13:18:53.174118138 +0000 UTC m=+6.069776207 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/33610912-09f3-406f-be0f-a6ffbff4c9ba-original-pull-secret") pod "global-pull-secret-syncer-l6cc6" (UID: "33610912-09f3-406f-be0f-a6ffbff4c9ba") : object "kube-system"/"original-pull-secret" not registered Jul 05 13:18:51.174368 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:51.174327 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2010fc9b-2713-4036-8131-071a9349d36e-metrics-certs\") pod \"network-metrics-daemon-g9tcx\" (UID: \"2010fc9b-2713-4036-8131-071a9349d36e\") " pod="openshift-multus/network-metrics-daemon-g9tcx" Jul 05 13:18:51.174464 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:51.174450 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:18:51.174528 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:51.174511 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2010fc9b-2713-4036-8131-071a9349d36e-metrics-certs podName:2010fc9b-2713-4036-8131-071a9349d36e nodeName:}" failed. No retries permitted until 2026-07-05 13:18:53.17449513 +0000 UTC m=+6.070153200 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2010fc9b-2713-4036-8131-071a9349d36e-metrics-certs") pod "network-metrics-daemon-g9tcx" (UID: "2010fc9b-2713-4036-8131-071a9349d36e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:18:51.275536 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:51.275432 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krvhl\" (UniqueName: \"kubernetes.io/projected/09e0d252-5d66-4697-b469-24d2cdf32679-kube-api-access-krvhl\") pod \"network-check-target-2nr4r\" (UID: \"09e0d252-5d66-4697-b469-24d2cdf32679\") " pod="openshift-network-diagnostics/network-check-target-2nr4r" Jul 05 13:18:51.275687 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:51.275614 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 05 13:18:51.275687 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:51.275639 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 05 13:18:51.275687 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:51.275652 2573 projected.go:196] Error preparing data for projected volume kube-api-access-krvhl for pod openshift-network-diagnostics/network-check-target-2nr4r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:18:51.275842 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:51.275709 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/09e0d252-5d66-4697-b469-24d2cdf32679-kube-api-access-krvhl podName:09e0d252-5d66-4697-b469-24d2cdf32679 nodeName:}" failed. No retries permitted until 2026-07-05 13:18:53.2756913 +0000 UTC m=+6.171349384 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-krvhl" (UniqueName: "kubernetes.io/projected/09e0d252-5d66-4697-b469-24d2cdf32679-kube-api-access-krvhl") pod "network-check-target-2nr4r" (UID: "09e0d252-5d66-4697-b469-24d2cdf32679") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:18:51.615786 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:51.615652 2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-04 13:13:49 +0000 UTC" deadline="2028-03-06 15:21:50.713797031 +0000 UTC" Jul 05 13:18:51.615786 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:51.615707 2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14642h2m59.098094118s" Jul 05 13:18:51.732353 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:51.732318 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g9tcx" Jul 05 13:18:51.732598 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:51.732461 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g9tcx" podUID="2010fc9b-2713-4036-8131-071a9349d36e" Jul 05 13:18:51.732849 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:51.732828 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2nr4r" Jul 05 13:18:51.732964 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:51.732926 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2nr4r" podUID="09e0d252-5d66-4697-b469-24d2cdf32679" Jul 05 13:18:51.792422 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:51.792387 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-155.ec2.internal" event={"ID":"f7be285f4bc4ee8d9639260737851004","Type":"ContainerStarted","Data":"a79c9d54c461419aaaace298dfc9d1ef538ee62720dc05662c07a3568e02b956"} Jul 05 13:18:51.806028 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:51.805996 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-hlvjf" event={"ID":"2a975403-6fed-4035-bf2d-c161d33baf8e","Type":"ContainerStarted","Data":"2da262e2525c95ca745401e4f6a95fd051f26146d4798a401c2d8121be7ca530"} Jul 05 13:18:51.816760 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:51.816708 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-155.ec2.internal" podStartSLOduration=3.816688791 podStartE2EDuration="3.816688791s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:18:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:18:51.815770536 +0000 UTC m=+4.711428627" watchObservedRunningTime="2026-07-05 13:18:51.816688791 +0000 UTC m=+4.712346882" Jul 05 13:18:51.827606 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:51.827520 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dr5b4" event={"ID":"514779be-55a5-41cb-9bce-42dd1c7398e5","Type":"ContainerStarted","Data":"7723eb4bd409873899eef47abb5de90d82d3770cb53cba59ef1df18fc3880cfe"} Jul 05 13:18:51.838529 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:51.838487 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fw42d" event={"ID":"91b3802b-db71-4a40-ad33-0f2ced5ef31d","Type":"ContainerStarted","Data":"1b7f1c4b446facd00dfd94e7553eccc0fbd9bba30b207246eab6e7f4fce48da0"} Jul 05 13:18:51.856587 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:51.856558 2573 generic.go:356] "Generic (PLEG): container finished" podID="2159b56be8d2ffd1bbbc05521c1be8b4" containerID="872dc86826af74b60d5adb4a8f1c93a7313eee1a2b4f58dc7d1155b685d16593" exitCode=0 Jul 05 13:18:51.856687 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:51.856604 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-155.ec2.internal" event={"ID":"2159b56be8d2ffd1bbbc05521c1be8b4","Type":"ContainerDied","Data":"872dc86826af74b60d5adb4a8f1c93a7313eee1a2b4f58dc7d1155b685d16593"} Jul 05 13:18:52.730935 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:52.730447 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l6cc6" Jul 05 13:18:52.730935 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:52.730586 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l6cc6" podUID="33610912-09f3-406f-be0f-a6ffbff4c9ba" Jul 05 13:18:52.893984 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:52.893933 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-155.ec2.internal" event={"ID":"2159b56be8d2ffd1bbbc05521c1be8b4","Type":"ContainerStarted","Data":"99a3da064bb803f62f3fb789cfee372d5e432e8c260a397da97751ee65519280"} Jul 05 13:18:52.921129 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:52.919756 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-155.ec2.internal" podStartSLOduration=4.9197356800000005 podStartE2EDuration="4.91973568s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:18:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:18:52.919707325 +0000 UTC m=+5.815365415" watchObservedRunningTime="2026-07-05 13:18:52.91973568 +0000 UTC m=+5.815393768" Jul 05 13:18:53.200332 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:53.199567 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/33610912-09f3-406f-be0f-a6ffbff4c9ba-original-pull-secret\") pod \"global-pull-secret-syncer-l6cc6\" (UID: \"33610912-09f3-406f-be0f-a6ffbff4c9ba\") " pod="kube-system/global-pull-secret-syncer-l6cc6" Jul 05 13:18:53.200332 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:53.199658 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2010fc9b-2713-4036-8131-071a9349d36e-metrics-certs\") pod \"network-metrics-daemon-g9tcx\" (UID: \"2010fc9b-2713-4036-8131-071a9349d36e\") " pod="openshift-multus/network-metrics-daemon-g9tcx" Jul 05 13:18:53.200332 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:53.199786 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:18:53.200332 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:53.199847 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2010fc9b-2713-4036-8131-071a9349d36e-metrics-certs podName:2010fc9b-2713-4036-8131-071a9349d36e nodeName:}" failed. No retries permitted until 2026-07-05 13:18:57.199828093 +0000 UTC m=+10.095486165 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2010fc9b-2713-4036-8131-071a9349d36e-metrics-certs") pod "network-metrics-daemon-g9tcx" (UID: "2010fc9b-2713-4036-8131-071a9349d36e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:18:53.200332 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:53.200236 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 05 13:18:53.200332 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:53.200287 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33610912-09f3-406f-be0f-a6ffbff4c9ba-original-pull-secret podName:33610912-09f3-406f-be0f-a6ffbff4c9ba nodeName:}" failed. No retries permitted until 2026-07-05 13:18:57.200271876 +0000 UTC m=+10.095929949 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/33610912-09f3-406f-be0f-a6ffbff4c9ba-original-pull-secret") pod "global-pull-secret-syncer-l6cc6" (UID: "33610912-09f3-406f-be0f-a6ffbff4c9ba") : object "kube-system"/"original-pull-secret" not registered Jul 05 13:18:53.300318 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:53.300277 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krvhl\" (UniqueName: \"kubernetes.io/projected/09e0d252-5d66-4697-b469-24d2cdf32679-kube-api-access-krvhl\") pod \"network-check-target-2nr4r\" (UID: \"09e0d252-5d66-4697-b469-24d2cdf32679\") " pod="openshift-network-diagnostics/network-check-target-2nr4r" Jul 05 13:18:53.300505 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:53.300489 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 05 13:18:53.300570 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:53.300511 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 05 13:18:53.300570 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:53.300524 2573 projected.go:196] Error preparing data for projected volume kube-api-access-krvhl for pod openshift-network-diagnostics/network-check-target-2nr4r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:18:53.300685 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:53.300593 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/09e0d252-5d66-4697-b469-24d2cdf32679-kube-api-access-krvhl podName:09e0d252-5d66-4697-b469-24d2cdf32679 nodeName:}" failed. No retries permitted until 2026-07-05 13:18:57.300575152 +0000 UTC m=+10.196233225 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-krvhl" (UniqueName: "kubernetes.io/projected/09e0d252-5d66-4697-b469-24d2cdf32679-kube-api-access-krvhl") pod "network-check-target-2nr4r" (UID: "09e0d252-5d66-4697-b469-24d2cdf32679") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:18:53.729839 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:53.729792 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2nr4r" Jul 05 13:18:53.730014 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:53.729928 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2nr4r" podUID="09e0d252-5d66-4697-b469-24d2cdf32679" Jul 05 13:18:53.730368 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:53.730347 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g9tcx" Jul 05 13:18:53.730478 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:53.730456 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g9tcx" podUID="2010fc9b-2713-4036-8131-071a9349d36e" Jul 05 13:18:54.729911 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:54.729880 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l6cc6" Jul 05 13:18:54.730377 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:54.730035 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l6cc6" podUID="33610912-09f3-406f-be0f-a6ffbff4c9ba" Jul 05 13:18:55.729916 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:55.729796 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2nr4r" Jul 05 13:18:55.730088 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:55.729922 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2nr4r" podUID="09e0d252-5d66-4697-b469-24d2cdf32679" Jul 05 13:18:55.730472 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:55.730289 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g9tcx" Jul 05 13:18:55.730472 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:55.730408 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g9tcx" podUID="2010fc9b-2713-4036-8131-071a9349d36e" Jul 05 13:18:56.729708 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:56.729672 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l6cc6" Jul 05 13:18:56.729882 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:56.729808 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l6cc6" podUID="33610912-09f3-406f-be0f-a6ffbff4c9ba" Jul 05 13:18:57.234306 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:57.234071 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/33610912-09f3-406f-be0f-a6ffbff4c9ba-original-pull-secret\") pod \"global-pull-secret-syncer-l6cc6\" (UID: \"33610912-09f3-406f-be0f-a6ffbff4c9ba\") " pod="kube-system/global-pull-secret-syncer-l6cc6" Jul 05 13:18:57.234306 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:57.234162 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2010fc9b-2713-4036-8131-071a9349d36e-metrics-certs\") pod \"network-metrics-daemon-g9tcx\" (UID: \"2010fc9b-2713-4036-8131-071a9349d36e\") " pod="openshift-multus/network-metrics-daemon-g9tcx" Jul 05 13:18:57.234306 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:57.234190 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 05 13:18:57.234306 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:57.234249 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33610912-09f3-406f-be0f-a6ffbff4c9ba-original-pull-secret podName:33610912-09f3-406f-be0f-a6ffbff4c9ba nodeName:}" failed. No retries permitted until 2026-07-05 13:19:05.234230814 +0000 UTC m=+18.129888902 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/33610912-09f3-406f-be0f-a6ffbff4c9ba-original-pull-secret") pod "global-pull-secret-syncer-l6cc6" (UID: "33610912-09f3-406f-be0f-a6ffbff4c9ba") : object "kube-system"/"original-pull-secret" not registered Jul 05 13:18:57.234306 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:57.234253 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:18:57.234306 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:57.234285 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2010fc9b-2713-4036-8131-071a9349d36e-metrics-certs podName:2010fc9b-2713-4036-8131-071a9349d36e nodeName:}" failed. No retries permitted until 2026-07-05 13:19:05.234275113 +0000 UTC m=+18.129933181 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2010fc9b-2713-4036-8131-071a9349d36e-metrics-certs") pod "network-metrics-daemon-g9tcx" (UID: "2010fc9b-2713-4036-8131-071a9349d36e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:18:57.335500 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:57.335463 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krvhl\" (UniqueName: \"kubernetes.io/projected/09e0d252-5d66-4697-b469-24d2cdf32679-kube-api-access-krvhl\") pod \"network-check-target-2nr4r\" (UID: \"09e0d252-5d66-4697-b469-24d2cdf32679\") " pod="openshift-network-diagnostics/network-check-target-2nr4r" Jul 05 13:18:57.335677 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:57.335653 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 05 13:18:57.335741 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:57.335681 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 05 13:18:57.335741 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:57.335695 2573 projected.go:196] Error preparing data for projected volume kube-api-access-krvhl for pod openshift-network-diagnostics/network-check-target-2nr4r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:18:57.335836 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:57.335753 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/09e0d252-5d66-4697-b469-24d2cdf32679-kube-api-access-krvhl podName:09e0d252-5d66-4697-b469-24d2cdf32679 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:05.335732588 +0000 UTC m=+18.231390658 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-krvhl" (UniqueName: "kubernetes.io/projected/09e0d252-5d66-4697-b469-24d2cdf32679-kube-api-access-krvhl") pod "network-check-target-2nr4r" (UID: "09e0d252-5d66-4697-b469-24d2cdf32679") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:18:57.731342 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:57.730717 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g9tcx" Jul 05 13:18:57.731342 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:57.730837 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g9tcx" podUID="2010fc9b-2713-4036-8131-071a9349d36e" Jul 05 13:18:57.731342 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:57.731200 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2nr4r" Jul 05 13:18:57.731342 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:57.731293 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2nr4r" podUID="09e0d252-5d66-4697-b469-24d2cdf32679" Jul 05 13:18:58.729839 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:58.729794 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l6cc6" Jul 05 13:18:58.730454 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:58.729944 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l6cc6" podUID="33610912-09f3-406f-be0f-a6ffbff4c9ba" Jul 05 13:18:59.733526 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:59.733500 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g9tcx" Jul 05 13:18:59.733967 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:18:59.733561 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2nr4r" Jul 05 13:18:59.733967 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:59.733669 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2nr4r" podUID="09e0d252-5d66-4697-b469-24d2cdf32679" Jul 05 13:18:59.733967 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:18:59.733777 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g9tcx" podUID="2010fc9b-2713-4036-8131-071a9349d36e" Jul 05 13:19:00.729759 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:00.729727 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l6cc6" Jul 05 13:19:00.729929 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:00.729848 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l6cc6" podUID="33610912-09f3-406f-be0f-a6ffbff4c9ba" Jul 05 13:19:01.733040 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:01.733009 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g9tcx" Jul 05 13:19:01.733505 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:01.733014 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2nr4r" Jul 05 13:19:01.733505 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:01.733150 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g9tcx" podUID="2010fc9b-2713-4036-8131-071a9349d36e" Jul 05 13:19:01.733505 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:01.733286 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2nr4r" podUID="09e0d252-5d66-4697-b469-24d2cdf32679" Jul 05 13:19:02.729783 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:02.729750 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l6cc6" Jul 05 13:19:02.729963 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:02.729866 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l6cc6" podUID="33610912-09f3-406f-be0f-a6ffbff4c9ba" Jul 05 13:19:03.730329 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:03.730263 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2nr4r" Jul 05 13:19:03.730329 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:03.730282 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g9tcx" Jul 05 13:19:03.730793 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:03.730384 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2nr4r" podUID="09e0d252-5d66-4697-b469-24d2cdf32679" Jul 05 13:19:03.730793 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:03.730465 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g9tcx" podUID="2010fc9b-2713-4036-8131-071a9349d36e" Jul 05 13:19:04.729696 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:04.729659 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l6cc6" Jul 05 13:19:04.729920 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:04.729801 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l6cc6" podUID="33610912-09f3-406f-be0f-a6ffbff4c9ba" Jul 05 13:19:05.295057 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:05.295022 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2010fc9b-2713-4036-8131-071a9349d36e-metrics-certs\") pod \"network-metrics-daemon-g9tcx\" (UID: \"2010fc9b-2713-4036-8131-071a9349d36e\") " pod="openshift-multus/network-metrics-daemon-g9tcx" Jul 05 13:19:05.295498 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:05.295084 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/33610912-09f3-406f-be0f-a6ffbff4c9ba-original-pull-secret\") pod \"global-pull-secret-syncer-l6cc6\" (UID: \"33610912-09f3-406f-be0f-a6ffbff4c9ba\") " pod="kube-system/global-pull-secret-syncer-l6cc6" Jul 05 13:19:05.295498 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:05.295207 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:19:05.295498 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:05.295220 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 05 13:19:05.295498 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:05.295272 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2010fc9b-2713-4036-8131-071a9349d36e-metrics-certs podName:2010fc9b-2713-4036-8131-071a9349d36e nodeName:}" failed. No retries permitted until 2026-07-05 13:19:21.295253221 +0000 UTC m=+34.190911291 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2010fc9b-2713-4036-8131-071a9349d36e-metrics-certs") pod "network-metrics-daemon-g9tcx" (UID: "2010fc9b-2713-4036-8131-071a9349d36e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:19:05.295498 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:05.295288 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33610912-09f3-406f-be0f-a6ffbff4c9ba-original-pull-secret podName:33610912-09f3-406f-be0f-a6ffbff4c9ba nodeName:}" failed. No retries permitted until 2026-07-05 13:19:21.295281744 +0000 UTC m=+34.190939810 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/33610912-09f3-406f-be0f-a6ffbff4c9ba-original-pull-secret") pod "global-pull-secret-syncer-l6cc6" (UID: "33610912-09f3-406f-be0f-a6ffbff4c9ba") : object "kube-system"/"original-pull-secret" not registered Jul 05 13:19:05.396016 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:05.395977 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krvhl\" (UniqueName: \"kubernetes.io/projected/09e0d252-5d66-4697-b469-24d2cdf32679-kube-api-access-krvhl\") pod \"network-check-target-2nr4r\" (UID: \"09e0d252-5d66-4697-b469-24d2cdf32679\") " pod="openshift-network-diagnostics/network-check-target-2nr4r" Jul 05 13:19:05.396177 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:05.396132 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 05 13:19:05.396177 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:05.396153 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 05 13:19:05.396177 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:05.396163 2573 projected.go:196] Error preparing data for projected volume kube-api-access-krvhl for pod openshift-network-diagnostics/network-check-target-2nr4r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:19:05.396307 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:05.396215 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/09e0d252-5d66-4697-b469-24d2cdf32679-kube-api-access-krvhl podName:09e0d252-5d66-4697-b469-24d2cdf32679 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:21.396197845 +0000 UTC m=+34.291855912 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-krvhl" (UniqueName: "kubernetes.io/projected/09e0d252-5d66-4697-b469-24d2cdf32679-kube-api-access-krvhl") pod "network-check-target-2nr4r" (UID: "09e0d252-5d66-4697-b469-24d2cdf32679") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:19:05.730683 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:05.730644 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2nr4r" Jul 05 13:19:05.730913 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:05.730781 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2nr4r" podUID="09e0d252-5d66-4697-b469-24d2cdf32679" Jul 05 13:19:05.730913 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:05.730826 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g9tcx" Jul 05 13:19:05.731056 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:05.730928 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g9tcx" podUID="2010fc9b-2713-4036-8131-071a9349d36e" Jul 05 13:19:06.729787 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:06.729748 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l6cc6" Jul 05 13:19:06.730278 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:06.729875 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l6cc6" podUID="33610912-09f3-406f-be0f-a6ffbff4c9ba" Jul 05 13:19:07.731186 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:07.731149 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g9tcx" Jul 05 13:19:07.731607 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:07.731273 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g9tcx" podUID="2010fc9b-2713-4036-8131-071a9349d36e" Jul 05 13:19:07.731607 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:07.731355 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2nr4r" Jul 05 13:19:07.731607 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:07.731473 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2nr4r" podUID="09e0d252-5d66-4697-b469-24d2cdf32679" Jul 05 13:19:08.729779 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:08.729748 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l6cc6" Jul 05 13:19:08.729978 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:08.729867 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l6cc6" podUID="33610912-09f3-406f-be0f-a6ffbff4c9ba" Jul 05 13:19:09.730050 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:09.730022 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2nr4r" Jul 05 13:19:09.730404 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:09.730023 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g9tcx" Jul 05 13:19:09.730404 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:09.730156 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2nr4r" podUID="09e0d252-5d66-4697-b469-24d2cdf32679" Jul 05 13:19:09.730404 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:09.730229 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g9tcx" podUID="2010fc9b-2713-4036-8131-071a9349d36e" Jul 05 13:19:09.933167 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:09.932937 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" event={"ID":"fafc6427-6250-484c-88b0-d7b08365d04b","Type":"ContainerStarted","Data":"bb59012fb4e1806b812db68e9e3bf868944f8e22c0e4ab03f536ca97ad8db6d5"} Jul 05 13:19:09.935241 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:09.935215 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjd4z" event={"ID":"63ac7e2e-3c53-4a93-96e3-c038ed07a986","Type":"ContainerStarted","Data":"b1aef32a3a5e2a4457f309986d77944ee4500741cbfa574fe45286d8c57e8905"} Jul 05 13:19:09.936890 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:09.936832 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-hlvjf" event={"ID":"2a975403-6fed-4035-bf2d-c161d33baf8e","Type":"ContainerStarted","Data":"392d010e6a4276e70f38ee967a84f0d3082dd50e702074138c79b9066bcc7582"} Jul 05 13:19:09.939618 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:09.939392 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dr5b4" event={"ID":"514779be-55a5-41cb-9bce-42dd1c7398e5","Type":"ContainerStarted","Data":"5828e92a5deeef9b2605388abf09e70451b8b082200d861bc81b31bdc43f5d39"} Jul 05 13:19:09.941429 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:09.940943 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bh9gk" event={"ID":"6b401f24-d573-41fc-8668-0fd9d69e2cd3","Type":"ContainerStarted","Data":"a81a99b301b8b972bb44e7331f641f528b9896598f5e5e2cb9520ece81a7cf57"} Jul 05 13:19:09.971235 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:09.970916 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-bh9gk" podStartSLOduration=4.022241146 podStartE2EDuration="22.970897462s" totalImagesPullingTime="18.948656316s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:18:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:19:09.969486336 +0000 UTC m=+22.865144426" watchObservedRunningTime="2026-07-05 13:19:09.970897462 +0000 UTC m=+22.866555553" Jul 05 13:19:09.971358 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:09.971252 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-hlvjf" podStartSLOduration=4.02759384 podStartE2EDuration="22.971240118s" totalImagesPullingTime="18.943646278s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:18:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:19:09.953199134 +0000 UTC m=+22.848857226" watchObservedRunningTime="2026-07-05 13:19:09.971240118 +0000 UTC m=+22.866898207" Jul 05 13:19:09.993650 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:09.993462 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-dr5b4" podStartSLOduration=8.08258782 podStartE2EDuration="22.993446845s" totalImagesPullingTime="14.910859025s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:18:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:19:09.992616999 +0000 UTC m=+22.888275093" watchObservedRunningTime="2026-07-05 13:19:09.993446845 +0000 UTC m=+22.889104934" Jul 05 13:19:10.074361 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:10.074324 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:10.135878 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:10.135854 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:10.730272 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:10.730205 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l6cc6" Jul 05 13:19:10.730999 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:10.730329 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l6cc6" podUID="33610912-09f3-406f-be0f-a6ffbff4c9ba" Jul 05 13:19:10.943960 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:10.943923 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-j5pwd" event={"ID":"7d069d57-4ff7-4edd-94b6-10a12329f310","Type":"ContainerStarted","Data":"fea434976ad5404bf4ec9358ca86536994aa54a8134a08554417957075916909"} Jul 05 13:19:10.946740 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:10.946717 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" event={"ID":"fafc6427-6250-484c-88b0-d7b08365d04b","Type":"ContainerStarted","Data":"df644767068932f162bdb18d09a71725d67d11295ba0eedbdab923506551b854"} Jul 05 13:19:10.946888 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:10.946790 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" event={"ID":"fafc6427-6250-484c-88b0-d7b08365d04b","Type":"ContainerStarted","Data":"73bda537e4026c8643775b1a897d47b282f4230efb046fa9af7d7c92e7282c66"} Jul 05 13:19:10.946888 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:10.946842 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" event={"ID":"fafc6427-6250-484c-88b0-d7b08365d04b","Type":"ContainerStarted","Data":"d53f8fda1be25520d40e932e43a4a354afe4dff29c705512f8e2c508f3d4816f"} Jul 05 13:19:10.947014 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:10.946887 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" event={"ID":"fafc6427-6250-484c-88b0-d7b08365d04b","Type":"ContainerStarted","Data":"246191d49451312010703d71f4fef1ff80abd9283148d76a47896aeff7fc3997"} Jul 05 13:19:10.947014 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:10.946934 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" event={"ID":"fafc6427-6250-484c-88b0-d7b08365d04b","Type":"ContainerStarted","Data":"64cdad6d571a1d4ed280a7d93e99ea7b4c563276813ce4d17b18ccb7af42899b"} Jul 05 13:19:10.948107 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:10.948085 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4p2vl" event={"ID":"8ec840e9-44f1-4daa-963a-96632c87bb8a","Type":"ContainerStarted","Data":"d41a76f17e2750d5bc858eda04f02c722f424d2ced3dafce9f8710e1d21d1270"} Jul 05 13:19:10.949481 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:10.949458 2573 generic.go:356] "Generic (PLEG): container finished" podID="91b3802b-db71-4a40-ad33-0f2ced5ef31d" containerID="5f56aa5e980dbe3c76509f1bf303c5d671d3e255ad1f9068ceffc8043d14a8b4" exitCode=0 Jul 05 13:19:10.949589 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:10.949551 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fw42d" event={"ID":"91b3802b-db71-4a40-ad33-0f2ced5ef31d","Type":"ContainerDied","Data":"5f56aa5e980dbe3c76509f1bf303c5d671d3e255ad1f9068ceffc8043d14a8b4"} Jul 05 13:19:10.958491 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:10.958457 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-j5pwd" podStartSLOduration=5.006705834 podStartE2EDuration="23.958445366s" totalImagesPullingTime="18.951739532s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:18:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:19:10.958250074 +0000 UTC m=+23.853908163" watchObservedRunningTime="2026-07-05 13:19:10.958445366 +0000 UTC m=+23.854103455" Jul 05 13:19:10.993127 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:10.993053 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-4p2vl" podStartSLOduration=4.868476602 podStartE2EDuration="23.993041489s" totalImagesPullingTime="19.124564887s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:18:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:19:10.992584178 +0000 UTC m=+23.888242266" watchObservedRunningTime="2026-07-05 13:19:10.993041489 +0000 UTC m=+23.888699579" Jul 05 13:19:11.113483 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:11.113448 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:11.214869 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:11.214845 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:11.280794 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:11.280767 2573 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 05 13:19:11.610188 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:11.610022 2573 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-05T13:19:11.280790448Z","UUID":"077c1e88-621a-4af1-9374-6fa69d3abd2e","Handler":null,"Name":"","Endpoint":""} Jul 05 13:19:11.611592 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:11.611568 2573 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 05 13:19:11.611724 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:11.611615 2573 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 05 13:19:11.730070 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:11.730035 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2nr4r" Jul 05 13:19:11.730070 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:11.730054 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g9tcx" Jul 05 13:19:11.730299 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:11.730162 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2nr4r" podUID="09e0d252-5d66-4697-b469-24d2cdf32679" Jul 05 13:19:11.730299 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:11.730280 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g9tcx" podUID="2010fc9b-2713-4036-8131-071a9349d36e" Jul 05 13:19:11.953130 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:11.953095 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tpn2j" event={"ID":"93c7227a-9d34-4565-abe7-ff14cddfe6b2","Type":"ContainerStarted","Data":"f9f5b7615e94a2857617e3072167cea1cb79b81f12c258ed0f07ab6d530a3cce"} Jul 05 13:19:11.955143 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:11.955108 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjd4z" event={"ID":"63ac7e2e-3c53-4a93-96e3-c038ed07a986","Type":"ContainerStarted","Data":"8b1fe3b4682ce457ffc6aae0d79216829bc454d931af9b7d7a75b5e9a1688549"} Jul 05 13:19:11.968472 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:11.968424 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-tpn2j" podStartSLOduration=6.037086211 podStartE2EDuration="24.968408319s" totalImagesPullingTime="18.931322108s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:18:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:19:11.968022128 +0000 UTC m=+24.863680224" watchObservedRunningTime="2026-07-05 13:19:11.968408319 +0000 UTC m=+24.864066451" Jul 05 13:19:12.011493 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:12.011453 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-dr5b4" Jul 05 13:19:12.012113 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:12.012087 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-dr5b4" Jul 05 13:19:12.089751 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:12.089716 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:12.148551 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:12.148521 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:12.730101 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:12.730072 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l6cc6" Jul 05 13:19:12.730267 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:12.730177 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l6cc6" podUID="33610912-09f3-406f-be0f-a6ffbff4c9ba" Jul 05 13:19:12.962945 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:12.962912 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" event={"ID":"fafc6427-6250-484c-88b0-d7b08365d04b","Type":"ContainerStarted","Data":"01599708f7b362c0b130560e6d693305ef6ef28aae51c77df32874c4d26574d1"} Jul 05 13:19:12.965361 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:12.965317 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjd4z" event={"ID":"63ac7e2e-3c53-4a93-96e3-c038ed07a986","Type":"ContainerStarted","Data":"0000f78db398ba4de57de870e6aa064aeba944c85d794b753d02489c6805f887"} Jul 05 13:19:12.965865 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:12.965840 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-dr5b4" Jul 05 13:19:12.966154 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:12.966135 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-dr5b4" Jul 05 13:19:12.987769 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:12.987694 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjd4z" podStartSLOduration=4.724392796 podStartE2EDuration="25.987676729s" totalImagesPullingTime="21.263283933s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:18:47 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:19:12.987539723 +0000 UTC m=+25.883197815" watchObservedRunningTime="2026-07-05 13:19:12.987676729 +0000 UTC m=+25.883334819" Jul 05 13:19:13.072940 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:13.072914 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:13.524018 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:13.523850 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:13.588427 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:13.588392 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:13.651731 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:13.651690 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:13.730074 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:13.730043 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2nr4r" Jul 05 13:19:13.730310 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:13.730052 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g9tcx" Jul 05 13:19:13.730676 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:13.730644 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2nr4r" podUID="09e0d252-5d66-4697-b469-24d2cdf32679" Jul 05 13:19:13.730869 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:13.730845 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g9tcx" podUID="2010fc9b-2713-4036-8131-071a9349d36e" Jul 05 13:19:13.964162 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:13.964126 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:14.024066 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:14.024037 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:14.730540 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:14.730503 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l6cc6" Jul 05 13:19:14.730730 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:14.730628 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l6cc6" podUID="33610912-09f3-406f-be0f-a6ffbff4c9ba" Jul 05 13:19:14.973434 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:14.973393 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" event={"ID":"fafc6427-6250-484c-88b0-d7b08365d04b","Type":"ContainerStarted","Data":"2dc8afd88cf2a08eaaf294d338d742ece70198831040ed857703328da1577bac"} Jul 05 13:19:14.998392 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:14.998350 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" podStartSLOduration=9.023787906 podStartE2EDuration="27.998335181s" totalImagesPullingTime="18.974547275s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:18:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:19:14.997253945 +0000 UTC m=+27.892912044" watchObservedRunningTime="2026-07-05 13:19:14.998335181 +0000 UTC m=+27.893993261" Jul 05 13:19:15.331062 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:15.330882 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:15.386897 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:15.386861 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:15.730171 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:15.730138 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2nr4r" Jul 05 13:19:15.730331 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:15.730258 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2nr4r" podUID="09e0d252-5d66-4697-b469-24d2cdf32679" Jul 05 13:19:15.730331 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:15.730305 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g9tcx" Jul 05 13:19:15.730422 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:15.730401 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g9tcx" podUID="2010fc9b-2713-4036-8131-071a9349d36e" Jul 05 13:19:15.977285 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:15.977256 2573 generic.go:356] "Generic (PLEG): container finished" podID="91b3802b-db71-4a40-ad33-0f2ced5ef31d" containerID="43f8ae949adc2e00d0fb9aaf33562bc2e790db9b0bd7c19d66b77825b02bd43a" exitCode=0 Jul 05 13:19:15.977749 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:15.977343 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fw42d" event={"ID":"91b3802b-db71-4a40-ad33-0f2ced5ef31d","Type":"ContainerDied","Data":"43f8ae949adc2e00d0fb9aaf33562bc2e790db9b0bd7c19d66b77825b02bd43a"} Jul 05 13:19:15.977749 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:15.977709 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:19:15.977850 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:15.977768 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:19:15.977850 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:15.977816 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:19:15.993207 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:15.993187 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:19:15.993340 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:15.993277 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:19:16.730261 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:16.730141 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l6cc6" Jul 05 13:19:16.730560 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:16.730287 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l6cc6" podUID="33610912-09f3-406f-be0f-a6ffbff4c9ba" Jul 05 13:19:16.965429 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:16.965385 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-l6cc6"] Jul 05 13:19:16.966034 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:16.966008 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2nr4r"] Jul 05 13:19:16.966131 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:16.966119 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2nr4r" Jul 05 13:19:16.966222 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:16.966201 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2nr4r" podUID="09e0d252-5d66-4697-b469-24d2cdf32679" Jul 05 13:19:16.979501 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:16.979478 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-g9tcx"] Jul 05 13:19:16.979897 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:16.979576 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g9tcx" Jul 05 13:19:16.979897 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:16.979686 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g9tcx" podUID="2010fc9b-2713-4036-8131-071a9349d36e" Jul 05 13:19:16.979897 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:16.979856 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l6cc6" Jul 05 13:19:16.980061 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:16.979973 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l6cc6" podUID="33610912-09f3-406f-be0f-a6ffbff4c9ba" Jul 05 13:19:17.984425 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:17.984392 2573 generic.go:356] "Generic (PLEG): container finished" podID="91b3802b-db71-4a40-ad33-0f2ced5ef31d" containerID="3c8fe5b7bf41a58a71eb1fccb5a817c75fe60c546e52977823a4f89c0ed072cb" exitCode=0 Jul 05 13:19:17.984936 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:17.984483 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fw42d" event={"ID":"91b3802b-db71-4a40-ad33-0f2ced5ef31d","Type":"ContainerDied","Data":"3c8fe5b7bf41a58a71eb1fccb5a817c75fe60c546e52977823a4f89c0ed072cb"} Jul 05 13:19:18.730651 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:18.730614 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2nr4r" Jul 05 13:19:18.730651 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:18.730649 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g9tcx" Jul 05 13:19:18.730831 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:18.730718 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l6cc6" Jul 05 13:19:18.730831 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:18.730774 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2nr4r" podUID="09e0d252-5d66-4697-b469-24d2cdf32679" Jul 05 13:19:18.730911 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:18.730869 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l6cc6" podUID="33610912-09f3-406f-be0f-a6ffbff4c9ba" Jul 05 13:19:18.731002 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:18.730980 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g9tcx" podUID="2010fc9b-2713-4036-8131-071a9349d36e" Jul 05 13:19:19.991936 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:19.991770 2573 generic.go:356] "Generic (PLEG): container finished" podID="91b3802b-db71-4a40-ad33-0f2ced5ef31d" containerID="eacde17c3ee0f69b429827e634c757e134610b6e61aa031627faa640d3c58120" exitCode=0 Jul 05 13:19:19.992348 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:19.991864 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fw42d" event={"ID":"91b3802b-db71-4a40-ad33-0f2ced5ef31d","Type":"ContainerDied","Data":"eacde17c3ee0f69b429827e634c757e134610b6e61aa031627faa640d3c58120"} Jul 05 13:19:20.730031 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:20.729787 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2nr4r" Jul 05 13:19:20.730031 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:20.729832 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l6cc6" Jul 05 13:19:20.730031 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:20.729925 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g9tcx" Jul 05 13:19:20.730031 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:20.729940 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2nr4r" podUID="09e0d252-5d66-4697-b469-24d2cdf32679" Jul 05 13:19:20.730348 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:20.730069 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g9tcx" podUID="2010fc9b-2713-4036-8131-071a9349d36e" Jul 05 13:19:20.730348 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:20.730199 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l6cc6" podUID="33610912-09f3-406f-be0f-a6ffbff4c9ba" Jul 05 13:19:20.765156 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:20.765124 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:20.864212 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:20.864171 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:20.964515 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:20.964477 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:21.320197 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:21.320162 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2010fc9b-2713-4036-8131-071a9349d36e-metrics-certs\") pod \"network-metrics-daemon-g9tcx\" (UID: \"2010fc9b-2713-4036-8131-071a9349d36e\") " pod="openshift-multus/network-metrics-daemon-g9tcx" Jul 05 13:19:21.320773 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:21.320231 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/33610912-09f3-406f-be0f-a6ffbff4c9ba-original-pull-secret\") pod \"global-pull-secret-syncer-l6cc6\" (UID: \"33610912-09f3-406f-be0f-a6ffbff4c9ba\") " pod="kube-system/global-pull-secret-syncer-l6cc6" Jul 05 13:19:21.320773 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:21.320285 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:19:21.320773 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:21.320368 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2010fc9b-2713-4036-8131-071a9349d36e-metrics-certs podName:2010fc9b-2713-4036-8131-071a9349d36e nodeName:}" failed. No retries permitted until 2026-07-05 13:19:53.320343277 +0000 UTC m=+66.216001358 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2010fc9b-2713-4036-8131-071a9349d36e-metrics-certs") pod "network-metrics-daemon-g9tcx" (UID: "2010fc9b-2713-4036-8131-071a9349d36e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:19:21.320773 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:21.320371 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 05 13:19:21.320773 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:21.320433 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33610912-09f3-406f-be0f-a6ffbff4c9ba-original-pull-secret podName:33610912-09f3-406f-be0f-a6ffbff4c9ba nodeName:}" failed. No retries permitted until 2026-07-05 13:19:53.320420649 +0000 UTC m=+66.216078721 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/33610912-09f3-406f-be0f-a6ffbff4c9ba-original-pull-secret") pod "global-pull-secret-syncer-l6cc6" (UID: "33610912-09f3-406f-be0f-a6ffbff4c9ba") : object "kube-system"/"original-pull-secret" not registered Jul 05 13:19:21.420806 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:21.420770 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krvhl\" (UniqueName: \"kubernetes.io/projected/09e0d252-5d66-4697-b469-24d2cdf32679-kube-api-access-krvhl\") pod \"network-check-target-2nr4r\" (UID: \"09e0d252-5d66-4697-b469-24d2cdf32679\") " pod="openshift-network-diagnostics/network-check-target-2nr4r" Jul 05 13:19:21.420999 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:21.420979 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 05 13:19:21.421071 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:21.421003 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 05 13:19:21.421071 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:21.421018 2573 projected.go:196] Error preparing data for projected volume kube-api-access-krvhl for pod openshift-network-diagnostics/network-check-target-2nr4r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:19:21.421165 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:21.421099 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/09e0d252-5d66-4697-b469-24d2cdf32679-kube-api-access-krvhl podName:09e0d252-5d66-4697-b469-24d2cdf32679 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:53.42107942 +0000 UTC m=+66.316737492 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-krvhl" (UniqueName: "kubernetes.io/projected/09e0d252-5d66-4697-b469-24d2cdf32679-kube-api-access-krvhl") pod "network-check-target-2nr4r" (UID: "09e0d252-5d66-4697-b469-24d2cdf32679") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:19:21.521431 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:21.521377 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:21.582133 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:21.582046 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:22.405343 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.405315 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-155.ec2.internal" event="NodeReady" Jul 05 13:19:22.405846 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.405479 2573 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 05 13:19:22.481255 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.481221 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-78fc4b5dfd-vrwbb"] Jul 05 13:19:22.485212 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.485186 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-gw7sw"] Jul 05 13:19:22.485356 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.485326 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78fc4b5dfd-vrwbb" Jul 05 13:19:22.488478 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.488457 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-gvrpq"] Jul 05 13:19:22.488615 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.488594 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gw7sw" Jul 05 13:19:22.489178 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.489159 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 05 13:19:22.489268 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.489169 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 05 13:19:22.489550 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.489533 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 05 13:19:22.489674 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.489658 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-jcbpr\"" Jul 05 13:19:22.491825 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.491802 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gvrpq" Jul 05 13:19:22.491913 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.491888 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 05 13:19:22.493131 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.493112 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-gcgw5\"" Jul 05 13:19:22.493225 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.493139 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 05 13:19:22.493548 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.493529 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 05 13:19:22.495592 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.495528 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dww7h\"" Jul 05 13:19:22.495592 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.495584 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 05 13:19:22.496317 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.495860 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 05 13:19:22.496317 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.496298 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 05 13:19:22.503510 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.503483 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gw7sw"] Jul 05 13:19:22.512554 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.512499 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gvrpq"] Jul 05 13:19:22.515000 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.514972 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-78fc4b5dfd-vrwbb"] Jul 05 13:19:22.629323 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.629278 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/80ca46e3-b000-4760-89ad-2ab808957f27-installation-pull-secrets\") pod \"image-registry-78fc4b5dfd-vrwbb\" (UID: \"80ca46e3-b000-4760-89ad-2ab808957f27\") " pod="openshift-image-registry/image-registry-78fc4b5dfd-vrwbb" Jul 05 13:19:22.629323 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.629315 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b385ab14-87cd-4c01-81fe-b2e4e37528f0-cert\") pod \"ingress-canary-gw7sw\" (UID: \"b385ab14-87cd-4c01-81fe-b2e4e37528f0\") " pod="openshift-ingress-canary/ingress-canary-gw7sw" Jul 05 13:19:22.629553 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.629342 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxfjl\" (UniqueName: \"kubernetes.io/projected/d52daf5f-c799-45a9-b187-a9613bfe066c-kube-api-access-fxfjl\") pod \"dns-default-gvrpq\" (UID: \"d52daf5f-c799-45a9-b187-a9613bfe066c\") " pod="openshift-dns/dns-default-gvrpq" Jul 05 13:19:22.629553 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.629370 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/80ca46e3-b000-4760-89ad-2ab808957f27-ca-trust-extracted\") pod \"image-registry-78fc4b5dfd-vrwbb\" (UID: \"80ca46e3-b000-4760-89ad-2ab808957f27\") " pod="openshift-image-registry/image-registry-78fc4b5dfd-vrwbb" Jul 05 13:19:22.629553 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.629389 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/80ca46e3-b000-4760-89ad-2ab808957f27-registry-certificates\") pod \"image-registry-78fc4b5dfd-vrwbb\" (UID: \"80ca46e3-b000-4760-89ad-2ab808957f27\") " pod="openshift-image-registry/image-registry-78fc4b5dfd-vrwbb" Jul 05 13:19:22.629553 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.629411 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/80ca46e3-b000-4760-89ad-2ab808957f27-registry-tls\") pod \"image-registry-78fc4b5dfd-vrwbb\" (UID: \"80ca46e3-b000-4760-89ad-2ab808957f27\") " pod="openshift-image-registry/image-registry-78fc4b5dfd-vrwbb" Jul 05 13:19:22.629749 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.629547 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d52daf5f-c799-45a9-b187-a9613bfe066c-metrics-tls\") pod \"dns-default-gvrpq\" (UID: \"d52daf5f-c799-45a9-b187-a9613bfe066c\") " pod="openshift-dns/dns-default-gvrpq" Jul 05 13:19:22.629749 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.629590 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tz62\" (UniqueName: \"kubernetes.io/projected/b385ab14-87cd-4c01-81fe-b2e4e37528f0-kube-api-access-6tz62\") pod \"ingress-canary-gw7sw\" (UID: \"b385ab14-87cd-4c01-81fe-b2e4e37528f0\") " pod="openshift-ingress-canary/ingress-canary-gw7sw" Jul 05 13:19:22.629749 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.629619 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d52daf5f-c799-45a9-b187-a9613bfe066c-tmp-dir\") pod \"dns-default-gvrpq\" (UID: \"d52daf5f-c799-45a9-b187-a9613bfe066c\") " pod="openshift-dns/dns-default-gvrpq" Jul 05 13:19:22.629749 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.629648 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/80ca46e3-b000-4760-89ad-2ab808957f27-image-registry-private-configuration\") pod \"image-registry-78fc4b5dfd-vrwbb\" (UID: \"80ca46e3-b000-4760-89ad-2ab808957f27\") " pod="openshift-image-registry/image-registry-78fc4b5dfd-vrwbb" Jul 05 13:19:22.629749 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.629682 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d52daf5f-c799-45a9-b187-a9613bfe066c-config-volume\") pod \"dns-default-gvrpq\" (UID: \"d52daf5f-c799-45a9-b187-a9613bfe066c\") " pod="openshift-dns/dns-default-gvrpq" Jul 05 13:19:22.629749 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.629707 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/80ca46e3-b000-4760-89ad-2ab808957f27-trusted-ca\") pod \"image-registry-78fc4b5dfd-vrwbb\" (UID: \"80ca46e3-b000-4760-89ad-2ab808957f27\") " pod="openshift-image-registry/image-registry-78fc4b5dfd-vrwbb" Jul 05 13:19:22.629988 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.629756 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkbqc\" (UniqueName: \"kubernetes.io/projected/80ca46e3-b000-4760-89ad-2ab808957f27-kube-api-access-wkbqc\") pod \"image-registry-78fc4b5dfd-vrwbb\" (UID: \"80ca46e3-b000-4760-89ad-2ab808957f27\") " pod="openshift-image-registry/image-registry-78fc4b5dfd-vrwbb" Jul 05 13:19:22.629988 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.629802 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/80ca46e3-b000-4760-89ad-2ab808957f27-bound-sa-token\") pod \"image-registry-78fc4b5dfd-vrwbb\" (UID: \"80ca46e3-b000-4760-89ad-2ab808957f27\") " pod="openshift-image-registry/image-registry-78fc4b5dfd-vrwbb" Jul 05 13:19:22.657395 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.657358 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:22.730517 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.730480 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2nr4r" Jul 05 13:19:22.730517 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.730513 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d52daf5f-c799-45a9-b187-a9613bfe066c-config-volume\") pod \"dns-default-gvrpq\" (UID: \"d52daf5f-c799-45a9-b187-a9613bfe066c\") " pod="openshift-dns/dns-default-gvrpq" Jul 05 13:19:22.730803 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.730551 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/80ca46e3-b000-4760-89ad-2ab808957f27-trusted-ca\") pod \"image-registry-78fc4b5dfd-vrwbb\" (UID: \"80ca46e3-b000-4760-89ad-2ab808957f27\") " pod="openshift-image-registry/image-registry-78fc4b5dfd-vrwbb" Jul 05 13:19:22.730803 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.730525 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g9tcx" Jul 05 13:19:22.730803 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.730598 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkbqc\" (UniqueName: \"kubernetes.io/projected/80ca46e3-b000-4760-89ad-2ab808957f27-kube-api-access-wkbqc\") pod \"image-registry-78fc4b5dfd-vrwbb\" (UID: \"80ca46e3-b000-4760-89ad-2ab808957f27\") " pod="openshift-image-registry/image-registry-78fc4b5dfd-vrwbb" Jul 05 13:19:22.730803 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.730527 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l6cc6" Jul 05 13:19:22.730803 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.730640 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/80ca46e3-b000-4760-89ad-2ab808957f27-bound-sa-token\") pod \"image-registry-78fc4b5dfd-vrwbb\" (UID: \"80ca46e3-b000-4760-89ad-2ab808957f27\") " pod="openshift-image-registry/image-registry-78fc4b5dfd-vrwbb" Jul 05 13:19:22.730803 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.730675 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/80ca46e3-b000-4760-89ad-2ab808957f27-installation-pull-secrets\") pod \"image-registry-78fc4b5dfd-vrwbb\" (UID: \"80ca46e3-b000-4760-89ad-2ab808957f27\") " pod="openshift-image-registry/image-registry-78fc4b5dfd-vrwbb" Jul 05 13:19:22.730803 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.730704 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b385ab14-87cd-4c01-81fe-b2e4e37528f0-cert\") pod \"ingress-canary-gw7sw\" (UID: \"b385ab14-87cd-4c01-81fe-b2e4e37528f0\") " pod="openshift-ingress-canary/ingress-canary-gw7sw" Jul 05 13:19:22.730803 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.730728 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxfjl\" (UniqueName: \"kubernetes.io/projected/d52daf5f-c799-45a9-b187-a9613bfe066c-kube-api-access-fxfjl\") pod \"dns-default-gvrpq\" (UID: \"d52daf5f-c799-45a9-b187-a9613bfe066c\") " pod="openshift-dns/dns-default-gvrpq" Jul 05 13:19:22.730803 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.730752 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/80ca46e3-b000-4760-89ad-2ab808957f27-ca-trust-extracted\") pod \"image-registry-78fc4b5dfd-vrwbb\" (UID: \"80ca46e3-b000-4760-89ad-2ab808957f27\") " pod="openshift-image-registry/image-registry-78fc4b5dfd-vrwbb" Jul 05 13:19:22.730803 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.730776 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/80ca46e3-b000-4760-89ad-2ab808957f27-registry-certificates\") pod \"image-registry-78fc4b5dfd-vrwbb\" (UID: \"80ca46e3-b000-4760-89ad-2ab808957f27\") " pod="openshift-image-registry/image-registry-78fc4b5dfd-vrwbb" Jul 05 13:19:22.730803 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.730801 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/80ca46e3-b000-4760-89ad-2ab808957f27-registry-tls\") pod \"image-registry-78fc4b5dfd-vrwbb\" (UID: \"80ca46e3-b000-4760-89ad-2ab808957f27\") " pod="openshift-image-registry/image-registry-78fc4b5dfd-vrwbb" Jul 05 13:19:22.731337 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.730846 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d52daf5f-c799-45a9-b187-a9613bfe066c-metrics-tls\") pod \"dns-default-gvrpq\" (UID: \"d52daf5f-c799-45a9-b187-a9613bfe066c\") " pod="openshift-dns/dns-default-gvrpq" Jul 05 13:19:22.731337 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.730873 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tz62\" (UniqueName: \"kubernetes.io/projected/b385ab14-87cd-4c01-81fe-b2e4e37528f0-kube-api-access-6tz62\") pod \"ingress-canary-gw7sw\" (UID: \"b385ab14-87cd-4c01-81fe-b2e4e37528f0\") " pod="openshift-ingress-canary/ingress-canary-gw7sw" Jul 05 13:19:22.731337 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.730897 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d52daf5f-c799-45a9-b187-a9613bfe066c-tmp-dir\") pod \"dns-default-gvrpq\" (UID: \"d52daf5f-c799-45a9-b187-a9613bfe066c\") " pod="openshift-dns/dns-default-gvrpq" Jul 05 13:19:22.731337 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.730921 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/80ca46e3-b000-4760-89ad-2ab808957f27-image-registry-private-configuration\") pod \"image-registry-78fc4b5dfd-vrwbb\" (UID: \"80ca46e3-b000-4760-89ad-2ab808957f27\") " pod="openshift-image-registry/image-registry-78fc4b5dfd-vrwbb" Jul 05 13:19:22.731337 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.731154 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d52daf5f-c799-45a9-b187-a9613bfe066c-config-volume\") pod \"dns-default-gvrpq\" (UID: \"d52daf5f-c799-45a9-b187-a9613bfe066c\") " pod="openshift-dns/dns-default-gvrpq" Jul 05 13:19:22.731337 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:22.731174 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 05 13:19:22.731337 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:22.731249 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d52daf5f-c799-45a9-b187-a9613bfe066c-metrics-tls podName:d52daf5f-c799-45a9-b187-a9613bfe066c nodeName:}" failed. No retries permitted until 2026-07-05 13:19:23.231229836 +0000 UTC m=+36.126887926 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d52daf5f-c799-45a9-b187-a9613bfe066c-metrics-tls") pod "dns-default-gvrpq" (UID: "d52daf5f-c799-45a9-b187-a9613bfe066c") : secret "dns-default-metrics-tls" not found Jul 05 13:19:22.731637 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:22.731407 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 05 13:19:22.731637 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:22.731420 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78fc4b5dfd-vrwbb: secret "image-registry-tls" not found Jul 05 13:19:22.731637 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:22.731461 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/80ca46e3-b000-4760-89ad-2ab808957f27-registry-tls podName:80ca46e3-b000-4760-89ad-2ab808957f27 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:23.231448309 +0000 UTC m=+36.127106392 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/80ca46e3-b000-4760-89ad-2ab808957f27-registry-tls") pod "image-registry-78fc4b5dfd-vrwbb" (UID: "80ca46e3-b000-4760-89ad-2ab808957f27") : secret "image-registry-tls" not found Jul 05 13:19:22.731637 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.731553 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/80ca46e3-b000-4760-89ad-2ab808957f27-trusted-ca\") pod \"image-registry-78fc4b5dfd-vrwbb\" (UID: \"80ca46e3-b000-4760-89ad-2ab808957f27\") " pod="openshift-image-registry/image-registry-78fc4b5dfd-vrwbb" Jul 05 13:19:22.731833 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.731716 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d52daf5f-c799-45a9-b187-a9613bfe066c-tmp-dir\") pod \"dns-default-gvrpq\" (UID: \"d52daf5f-c799-45a9-b187-a9613bfe066c\") " pod="openshift-dns/dns-default-gvrpq" Jul 05 13:19:22.731833 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.731767 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/80ca46e3-b000-4760-89ad-2ab808957f27-registry-certificates\") pod \"image-registry-78fc4b5dfd-vrwbb\" (UID: \"80ca46e3-b000-4760-89ad-2ab808957f27\") " pod="openshift-image-registry/image-registry-78fc4b5dfd-vrwbb" Jul 05 13:19:22.731913 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.731880 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/80ca46e3-b000-4760-89ad-2ab808957f27-ca-trust-extracted\") pod \"image-registry-78fc4b5dfd-vrwbb\" (UID: \"80ca46e3-b000-4760-89ad-2ab808957f27\") " pod="openshift-image-registry/image-registry-78fc4b5dfd-vrwbb" Jul 05 13:19:22.733589 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.733565 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6n5k8\"" Jul 05 13:19:22.733763 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.733710 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 05 13:19:22.734059 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.734041 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 05 13:19:22.734448 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.734290 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 05 13:19:22.734448 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.734380 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 05 13:19:22.735159 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.734981 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-kn7g2\"" Jul 05 13:19:22.735705 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.735683 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/80ca46e3-b000-4760-89ad-2ab808957f27-installation-pull-secrets\") pod \"image-registry-78fc4b5dfd-vrwbb\" (UID: \"80ca46e3-b000-4760-89ad-2ab808957f27\") " pod="openshift-image-registry/image-registry-78fc4b5dfd-vrwbb" Jul 05 13:19:22.735705 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.735703 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/80ca46e3-b000-4760-89ad-2ab808957f27-image-registry-private-configuration\") pod \"image-registry-78fc4b5dfd-vrwbb\" (UID: \"80ca46e3-b000-4760-89ad-2ab808957f27\") " pod="openshift-image-registry/image-registry-78fc4b5dfd-vrwbb" Jul 05 13:19:22.735860 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.735705 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b385ab14-87cd-4c01-81fe-b2e4e37528f0-cert\") pod \"ingress-canary-gw7sw\" (UID: \"b385ab14-87cd-4c01-81fe-b2e4e37528f0\") " pod="openshift-ingress-canary/ingress-canary-gw7sw" Jul 05 13:19:22.745348 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.745321 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/80ca46e3-b000-4760-89ad-2ab808957f27-bound-sa-token\") pod \"image-registry-78fc4b5dfd-vrwbb\" (UID: \"80ca46e3-b000-4760-89ad-2ab808957f27\") " pod="openshift-image-registry/image-registry-78fc4b5dfd-vrwbb" Jul 05 13:19:22.745455 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.745391 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tz62\" (UniqueName: \"kubernetes.io/projected/b385ab14-87cd-4c01-81fe-b2e4e37528f0-kube-api-access-6tz62\") pod \"ingress-canary-gw7sw\" (UID: \"b385ab14-87cd-4c01-81fe-b2e4e37528f0\") " pod="openshift-ingress-canary/ingress-canary-gw7sw" Jul 05 13:19:22.745517 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.745474 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkbqc\" (UniqueName: \"kubernetes.io/projected/80ca46e3-b000-4760-89ad-2ab808957f27-kube-api-access-wkbqc\") pod \"image-registry-78fc4b5dfd-vrwbb\" (UID: \"80ca46e3-b000-4760-89ad-2ab808957f27\") " pod="openshift-image-registry/image-registry-78fc4b5dfd-vrwbb" Jul 05 13:19:22.746018 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.745994 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxfjl\" (UniqueName: \"kubernetes.io/projected/d52daf5f-c799-45a9-b187-a9613bfe066c-kube-api-access-fxfjl\") pod \"dns-default-gvrpq\" (UID: \"d52daf5f-c799-45a9-b187-a9613bfe066c\") " pod="openshift-dns/dns-default-gvrpq" Jul 05 13:19:22.757218 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.757196 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:22.810372 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:22.810341 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gw7sw" Jul 05 13:19:23.233770 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:23.233738 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/80ca46e3-b000-4760-89ad-2ab808957f27-registry-tls\") pod \"image-registry-78fc4b5dfd-vrwbb\" (UID: \"80ca46e3-b000-4760-89ad-2ab808957f27\") " pod="openshift-image-registry/image-registry-78fc4b5dfd-vrwbb" Jul 05 13:19:23.233946 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:23.233798 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d52daf5f-c799-45a9-b187-a9613bfe066c-metrics-tls\") pod \"dns-default-gvrpq\" (UID: \"d52daf5f-c799-45a9-b187-a9613bfe066c\") " pod="openshift-dns/dns-default-gvrpq" Jul 05 13:19:23.233946 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:23.233899 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 05 13:19:23.233946 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:23.233924 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78fc4b5dfd-vrwbb: secret "image-registry-tls" not found Jul 05 13:19:23.234108 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:23.233911 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 05 13:19:23.234108 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:23.234007 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/80ca46e3-b000-4760-89ad-2ab808957f27-registry-tls podName:80ca46e3-b000-4760-89ad-2ab808957f27 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:24.233986684 +0000 UTC m=+37.129644769 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/80ca46e3-b000-4760-89ad-2ab808957f27-registry-tls") pod "image-registry-78fc4b5dfd-vrwbb" (UID: "80ca46e3-b000-4760-89ad-2ab808957f27") : secret "image-registry-tls" not found Jul 05 13:19:23.234108 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:23.234033 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d52daf5f-c799-45a9-b187-a9613bfe066c-metrics-tls podName:d52daf5f-c799-45a9-b187-a9613bfe066c nodeName:}" failed. No retries permitted until 2026-07-05 13:19:24.234024953 +0000 UTC m=+37.129683025 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d52daf5f-c799-45a9-b187-a9613bfe066c-metrics-tls") pod "dns-default-gvrpq" (UID: "d52daf5f-c799-45a9-b187-a9613bfe066c") : secret "dns-default-metrics-tls" not found Jul 05 13:19:23.547015 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:23.546928 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:23.567400 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:23.567367 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-t6rnf"] Jul 05 13:19:23.571631 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:23.571611 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-t6rnf" Jul 05 13:19:23.573920 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:23.573895 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 05 13:19:23.574062 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:23.573993 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vjf7b\"" Jul 05 13:19:23.574847 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:23.574830 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 05 13:19:23.574847 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:23.574837 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 05 13:19:23.574999 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:23.574873 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 05 13:19:23.596566 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:23.596543 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-t6rnf"] Jul 05 13:19:23.636753 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:23.636710 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:23.737406 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:23.737375 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0b88064d-dcff-40b1-af6a-195f267576c6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t6rnf\" (UID: \"0b88064d-dcff-40b1-af6a-195f267576c6\") " pod="openshift-insights/insights-runtime-extractor-t6rnf" Jul 05 13:19:23.737578 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:23.737419 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dzrs\" (UniqueName: \"kubernetes.io/projected/0b88064d-dcff-40b1-af6a-195f267576c6-kube-api-access-7dzrs\") pod \"insights-runtime-extractor-t6rnf\" (UID: \"0b88064d-dcff-40b1-af6a-195f267576c6\") " pod="openshift-insights/insights-runtime-extractor-t6rnf" Jul 05 13:19:23.737578 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:23.737499 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0b88064d-dcff-40b1-af6a-195f267576c6-data-volume\") pod \"insights-runtime-extractor-t6rnf\" (UID: \"0b88064d-dcff-40b1-af6a-195f267576c6\") " pod="openshift-insights/insights-runtime-extractor-t6rnf" Jul 05 13:19:23.737578 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:23.737533 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0b88064d-dcff-40b1-af6a-195f267576c6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t6rnf\" (UID: \"0b88064d-dcff-40b1-af6a-195f267576c6\") " pod="openshift-insights/insights-runtime-extractor-t6rnf" Jul 05 13:19:23.737695 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:23.737579 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0b88064d-dcff-40b1-af6a-195f267576c6-crio-socket\") pod \"insights-runtime-extractor-t6rnf\" (UID: \"0b88064d-dcff-40b1-af6a-195f267576c6\") " pod="openshift-insights/insights-runtime-extractor-t6rnf" Jul 05 13:19:23.777636 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:23.777604 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:23.838304 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:23.838269 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dzrs\" (UniqueName: \"kubernetes.io/projected/0b88064d-dcff-40b1-af6a-195f267576c6-kube-api-access-7dzrs\") pod \"insights-runtime-extractor-t6rnf\" (UID: \"0b88064d-dcff-40b1-af6a-195f267576c6\") " pod="openshift-insights/insights-runtime-extractor-t6rnf" Jul 05 13:19:23.838435 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:23.838361 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0b88064d-dcff-40b1-af6a-195f267576c6-data-volume\") pod \"insights-runtime-extractor-t6rnf\" (UID: \"0b88064d-dcff-40b1-af6a-195f267576c6\") " pod="openshift-insights/insights-runtime-extractor-t6rnf" Jul 05 13:19:23.838435 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:23.838387 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0b88064d-dcff-40b1-af6a-195f267576c6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t6rnf\" (UID: \"0b88064d-dcff-40b1-af6a-195f267576c6\") " pod="openshift-insights/insights-runtime-extractor-t6rnf" Jul 05 13:19:23.838522 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:23.838432 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0b88064d-dcff-40b1-af6a-195f267576c6-crio-socket\") pod \"insights-runtime-extractor-t6rnf\" (UID: \"0b88064d-dcff-40b1-af6a-195f267576c6\") " pod="openshift-insights/insights-runtime-extractor-t6rnf" Jul 05 13:19:23.838629 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:23.838601 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 05 13:19:23.838767 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:23.838642 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0b88064d-dcff-40b1-af6a-195f267576c6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t6rnf\" (UID: \"0b88064d-dcff-40b1-af6a-195f267576c6\") " pod="openshift-insights/insights-runtime-extractor-t6rnf" Jul 05 13:19:23.838767 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:23.838686 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b88064d-dcff-40b1-af6a-195f267576c6-insights-runtime-extractor-tls podName:0b88064d-dcff-40b1-af6a-195f267576c6 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:24.338661121 +0000 UTC m=+37.234319194 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0b88064d-dcff-40b1-af6a-195f267576c6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t6rnf" (UID: "0b88064d-dcff-40b1-af6a-195f267576c6") : secret "insights-runtime-extractor-tls" not found Jul 05 13:19:23.838767 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:23.838661 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0b88064d-dcff-40b1-af6a-195f267576c6-crio-socket\") pod \"insights-runtime-extractor-t6rnf\" (UID: \"0b88064d-dcff-40b1-af6a-195f267576c6\") " pod="openshift-insights/insights-runtime-extractor-t6rnf" Jul 05 13:19:23.838767 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:23.838710 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0b88064d-dcff-40b1-af6a-195f267576c6-data-volume\") pod \"insights-runtime-extractor-t6rnf\" (UID: \"0b88064d-dcff-40b1-af6a-195f267576c6\") " pod="openshift-insights/insights-runtime-extractor-t6rnf" Jul 05 13:19:23.838767 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:23.838725 2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 05 13:19:23.838976 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:23.838789 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0b88064d-dcff-40b1-af6a-195f267576c6-kube-rbac-proxy-cm podName:0b88064d-dcff-40b1-af6a-195f267576c6 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:24.338776881 +0000 UTC m=+37.234434951 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0b88064d-dcff-40b1-af6a-195f267576c6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t6rnf" (UID: "0b88064d-dcff-40b1-af6a-195f267576c6") : configmap "kube-rbac-proxy" not found Jul 05 13:19:23.863412 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:23.863379 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dzrs\" (UniqueName: \"kubernetes.io/projected/0b88064d-dcff-40b1-af6a-195f267576c6-kube-api-access-7dzrs\") pod \"insights-runtime-extractor-t6rnf\" (UID: \"0b88064d-dcff-40b1-af6a-195f267576c6\") " pod="openshift-insights/insights-runtime-extractor-t6rnf" Jul 05 13:19:23.889592 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:23.889406 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:23.964157 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:23.964135 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gw7sw"] Jul 05 13:19:23.967577 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:19:23.967551 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb385ab14_87cd_4c01_81fe_b2e4e37528f0.slice/crio-74fbdd44552b3ac8c391a0c73b48def460c70fc6d44d0f0da6c53e9f756e00e7 WatchSource:0}: Error finding container 74fbdd44552b3ac8c391a0c73b48def460c70fc6d44d0f0da6c53e9f756e00e7: Status 404 returned error can't find the container with id 74fbdd44552b3ac8c391a0c73b48def460c70fc6d44d0f0da6c53e9f756e00e7 Jul 05 13:19:24.004733 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:24.004703 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fw42d" event={"ID":"91b3802b-db71-4a40-ad33-0f2ced5ef31d","Type":"ContainerStarted","Data":"ae45a35a8d8465fb1f6d57949c1df20a81b6d37eaa42df89f2ad4dbd2bacb450"} Jul 05 13:19:24.005907 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:24.005886 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gw7sw" event={"ID":"b385ab14-87cd-4c01-81fe-b2e4e37528f0","Type":"ContainerStarted","Data":"74fbdd44552b3ac8c391a0c73b48def460c70fc6d44d0f0da6c53e9f756e00e7"} Jul 05 13:19:24.176062 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:24.176034 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:24.242230 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:24.242204 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/80ca46e3-b000-4760-89ad-2ab808957f27-registry-tls\") pod \"image-registry-78fc4b5dfd-vrwbb\" (UID: \"80ca46e3-b000-4760-89ad-2ab808957f27\") " pod="openshift-image-registry/image-registry-78fc4b5dfd-vrwbb" Jul 05 13:19:24.242359 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:24.242248 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d52daf5f-c799-45a9-b187-a9613bfe066c-metrics-tls\") pod \"dns-default-gvrpq\" (UID: \"d52daf5f-c799-45a9-b187-a9613bfe066c\") " pod="openshift-dns/dns-default-gvrpq" Jul 05 13:19:24.242393 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:24.242350 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 05 13:19:24.242393 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:24.242360 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 05 13:19:24.242393 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:24.242369 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78fc4b5dfd-vrwbb: secret "image-registry-tls" not found Jul 05 13:19:24.242495 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:24.242428 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d52daf5f-c799-45a9-b187-a9613bfe066c-metrics-tls podName:d52daf5f-c799-45a9-b187-a9613bfe066c nodeName:}" failed. No retries permitted until 2026-07-05 13:19:26.242415732 +0000 UTC m=+39.138073800 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d52daf5f-c799-45a9-b187-a9613bfe066c-metrics-tls") pod "dns-default-gvrpq" (UID: "d52daf5f-c799-45a9-b187-a9613bfe066c") : secret "dns-default-metrics-tls" not found Jul 05 13:19:24.242495 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:24.242441 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/80ca46e3-b000-4760-89ad-2ab808957f27-registry-tls podName:80ca46e3-b000-4760-89ad-2ab808957f27 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:26.242435544 +0000 UTC m=+39.138093610 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/80ca46e3-b000-4760-89ad-2ab808957f27-registry-tls") pod "image-registry-78fc4b5dfd-vrwbb" (UID: "80ca46e3-b000-4760-89ad-2ab808957f27") : secret "image-registry-tls" not found Jul 05 13:19:24.322506 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:24.322466 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:24.334228 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:24.334155 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-qv6qs"] Jul 05 13:19:24.337313 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:24.337291 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-qv6qs" Jul 05 13:19:24.341211 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:24.341183 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 05 13:19:24.341340 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:24.341183 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 05 13:19:24.342505 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:24.342420 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-jp7sf\"" Jul 05 13:19:24.342700 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:24.342675 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0b88064d-dcff-40b1-af6a-195f267576c6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t6rnf\" (UID: \"0b88064d-dcff-40b1-af6a-195f267576c6\") " pod="openshift-insights/insights-runtime-extractor-t6rnf" Jul 05 13:19:24.342843 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:24.342827 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 05 13:19:24.342908 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:24.342894 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b88064d-dcff-40b1-af6a-195f267576c6-insights-runtime-extractor-tls podName:0b88064d-dcff-40b1-af6a-195f267576c6 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:25.342875533 +0000 UTC m=+38.238533614 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0b88064d-dcff-40b1-af6a-195f267576c6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t6rnf" (UID: "0b88064d-dcff-40b1-af6a-195f267576c6") : secret "insights-runtime-extractor-tls" not found Jul 05 13:19:24.343002 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:24.342918 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0b88064d-dcff-40b1-af6a-195f267576c6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t6rnf\" (UID: \"0b88064d-dcff-40b1-af6a-195f267576c6\") " pod="openshift-insights/insights-runtime-extractor-t6rnf" Jul 05 13:19:24.343052 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:24.343038 2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 05 13:19:24.343099 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:24.343084 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0b88064d-dcff-40b1-af6a-195f267576c6-kube-rbac-proxy-cm podName:0b88064d-dcff-40b1-af6a-195f267576c6 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:25.343072116 +0000 UTC m=+38.238730189 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0b88064d-dcff-40b1-af6a-195f267576c6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t6rnf" (UID: "0b88064d-dcff-40b1-af6a-195f267576c6") : configmap "kube-rbac-proxy" not found Jul 05 13:19:24.369901 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:24.369870 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-qv6qs"] Jul 05 13:19:24.436304 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:24.436269 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:24.444439 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:24.444411 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9wzv\" (UniqueName: \"kubernetes.io/projected/320513ee-d812-4b15-87d2-f0bc0a35eaa6-kube-api-access-f9wzv\") pod \"migrator-d57d58cc9-qv6qs\" (UID: \"320513ee-d812-4b15-87d2-f0bc0a35eaa6\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-qv6qs" Jul 05 13:19:24.545387 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:24.545347 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9wzv\" (UniqueName: \"kubernetes.io/projected/320513ee-d812-4b15-87d2-f0bc0a35eaa6-kube-api-access-f9wzv\") pod \"migrator-d57d58cc9-qv6qs\" (UID: \"320513ee-d812-4b15-87d2-f0bc0a35eaa6\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-qv6qs" Jul 05 13:19:24.561006 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:24.560975 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9wzv\" (UniqueName: \"kubernetes.io/projected/320513ee-d812-4b15-87d2-f0bc0a35eaa6-kube-api-access-f9wzv\") pod \"migrator-d57d58cc9-qv6qs\" (UID: \"320513ee-d812-4b15-87d2-f0bc0a35eaa6\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-qv6qs" Jul 05 13:19:24.598423 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:24.598344 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:24.648908 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:24.648878 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-qv6qs" Jul 05 13:19:24.791561 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:24.791531 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-qv6qs"] Jul 05 13:19:24.795490 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:19:24.795463 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod320513ee_d812_4b15_87d2_f0bc0a35eaa6.slice/crio-736727035dbef70f81e5de9a7b4dca9bf02f83a593c22af0986281f06bb29f12 WatchSource:0}: Error finding container 736727035dbef70f81e5de9a7b4dca9bf02f83a593c22af0986281f06bb29f12: Status 404 returned error can't find the container with id 736727035dbef70f81e5de9a7b4dca9bf02f83a593c22af0986281f06bb29f12 Jul 05 13:19:25.009434 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:25.009399 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-qv6qs" event={"ID":"320513ee-d812-4b15-87d2-f0bc0a35eaa6","Type":"ContainerStarted","Data":"736727035dbef70f81e5de9a7b4dca9bf02f83a593c22af0986281f06bb29f12"} Jul 05 13:19:25.012605 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:25.012573 2573 generic.go:356] "Generic (PLEG): container finished" podID="91b3802b-db71-4a40-ad33-0f2ced5ef31d" containerID="ae45a35a8d8465fb1f6d57949c1df20a81b6d37eaa42df89f2ad4dbd2bacb450" exitCode=0 Jul 05 13:19:25.012757 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:25.012614 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fw42d" event={"ID":"91b3802b-db71-4a40-ad33-0f2ced5ef31d","Type":"ContainerDied","Data":"ae45a35a8d8465fb1f6d57949c1df20a81b6d37eaa42df89f2ad4dbd2bacb450"} Jul 05 13:19:25.068507 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:25.068484 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-j5pwd_7d069d57-4ff7-4edd-94b6-10a12329f310/dns-node-resolver/0.log" Jul 05 13:19:25.353398 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:25.353316 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0b88064d-dcff-40b1-af6a-195f267576c6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t6rnf\" (UID: \"0b88064d-dcff-40b1-af6a-195f267576c6\") " pod="openshift-insights/insights-runtime-extractor-t6rnf" Jul 05 13:19:25.353561 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:25.353452 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 05 13:19:25.353561 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:25.353475 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0b88064d-dcff-40b1-af6a-195f267576c6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t6rnf\" (UID: \"0b88064d-dcff-40b1-af6a-195f267576c6\") " pod="openshift-insights/insights-runtime-extractor-t6rnf" Jul 05 13:19:25.353561 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:25.353528 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b88064d-dcff-40b1-af6a-195f267576c6-insights-runtime-extractor-tls podName:0b88064d-dcff-40b1-af6a-195f267576c6 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:27.353505778 +0000 UTC m=+40.249163848 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0b88064d-dcff-40b1-af6a-195f267576c6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t6rnf" (UID: "0b88064d-dcff-40b1-af6a-195f267576c6") : secret "insights-runtime-extractor-tls" not found Jul 05 13:19:25.353731 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:25.353574 2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 05 13:19:25.353731 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:25.353626 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0b88064d-dcff-40b1-af6a-195f267576c6-kube-rbac-proxy-cm podName:0b88064d-dcff-40b1-af6a-195f267576c6 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:27.353610014 +0000 UTC m=+40.249268085 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0b88064d-dcff-40b1-af6a-195f267576c6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t6rnf" (UID: "0b88064d-dcff-40b1-af6a-195f267576c6") : configmap "kube-rbac-proxy" not found Jul 05 13:19:25.915962 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:25.915736 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:26.017374 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:26.017345 2573 generic.go:356] "Generic (PLEG): container finished" podID="91b3802b-db71-4a40-ad33-0f2ced5ef31d" containerID="69aee67a13f9c131030d7e20a25da4a8c492cfff2fac1c9758035ca4cbdfa1f7" exitCode=0 Jul 05 13:19:26.017514 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:26.017404 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fw42d" event={"ID":"91b3802b-db71-4a40-ad33-0f2ced5ef31d","Type":"ContainerDied","Data":"69aee67a13f9c131030d7e20a25da4a8c492cfff2fac1c9758035ca4cbdfa1f7"} Jul 05 13:19:26.026637 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:26.026613 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:26.074450 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:26.074420 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-hlvjf_2a975403-6fed-4035-bf2d-c161d33baf8e/node-ca/0.log" Jul 05 13:19:26.130928 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:26.130896 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:26.239375 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:26.239329 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:26.261431 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:26.261389 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/80ca46e3-b000-4760-89ad-2ab808957f27-registry-tls\") pod \"image-registry-78fc4b5dfd-vrwbb\" (UID: \"80ca46e3-b000-4760-89ad-2ab808957f27\") " pod="openshift-image-registry/image-registry-78fc4b5dfd-vrwbb" Jul 05 13:19:26.261596 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:26.261450 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d52daf5f-c799-45a9-b187-a9613bfe066c-metrics-tls\") pod \"dns-default-gvrpq\" (UID: \"d52daf5f-c799-45a9-b187-a9613bfe066c\") " pod="openshift-dns/dns-default-gvrpq" Jul 05 13:19:26.261596 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:26.261562 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 05 13:19:26.261596 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:26.261583 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78fc4b5dfd-vrwbb: secret "image-registry-tls" not found Jul 05 13:19:26.261771 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:26.261650 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 05 13:19:26.261771 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:26.261653 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/80ca46e3-b000-4760-89ad-2ab808957f27-registry-tls podName:80ca46e3-b000-4760-89ad-2ab808957f27 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:30.26163321 +0000 UTC m=+43.157291295 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/80ca46e3-b000-4760-89ad-2ab808957f27-registry-tls") pod "image-registry-78fc4b5dfd-vrwbb" (UID: "80ca46e3-b000-4760-89ad-2ab808957f27") : secret "image-registry-tls" not found Jul 05 13:19:26.261771 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:26.261717 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d52daf5f-c799-45a9-b187-a9613bfe066c-metrics-tls podName:d52daf5f-c799-45a9-b187-a9613bfe066c nodeName:}" failed. No retries permitted until 2026-07-05 13:19:30.261700555 +0000 UTC m=+43.157358631 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d52daf5f-c799-45a9-b187-a9613bfe066c-metrics-tls") pod "dns-default-gvrpq" (UID: "d52daf5f-c799-45a9-b187-a9613bfe066c") : secret "dns-default-metrics-tls" not found Jul 05 13:19:26.343075 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:26.343037 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:27.023910 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:27.023881 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fw42d" event={"ID":"91b3802b-db71-4a40-ad33-0f2ced5ef31d","Type":"ContainerStarted","Data":"ea15a5afa788551dc86cb0671f4a6576a6c6daa2e006a6cba15b541b016525ce"} Jul 05 13:19:27.026044 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:27.026017 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gw7sw" event={"ID":"b385ab14-87cd-4c01-81fe-b2e4e37528f0","Type":"ContainerStarted","Data":"b5a3d0dbe587c0758eb7ea1b6da000aa97407ce8c9f463a4f8811ca7b652c6f8"} Jul 05 13:19:27.028072 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:27.028047 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-qv6qs" event={"ID":"320513ee-d812-4b15-87d2-f0bc0a35eaa6","Type":"ContainerStarted","Data":"011ec56ea8d7901753da1f0b993d22b97625c00c9983841b706ae6500687d10f"} Jul 05 13:19:27.067242 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:27.067147 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-fw42d" podStartSLOduration=10.912541556 podStartE2EDuration="40.067128262s" totalImagesPullingTime="29.154586706s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:18:47 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:19:27.060722575 +0000 UTC m=+39.956380665" watchObservedRunningTime="2026-07-05 13:19:27.067128262 +0000 UTC m=+39.962786352" Jul 05 13:19:27.097894 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:27.097852 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-gw7sw" podStartSLOduration=2.30541739 podStartE2EDuration="5.097837865s" totalImagesPullingTime="2.792420475s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:19:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:19:27.097690489 +0000 UTC m=+39.993348585" watchObservedRunningTime="2026-07-05 13:19:27.097837865 +0000 UTC m=+39.993496003" Jul 05 13:19:27.238892 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:27.238854 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:27.342607 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:27.342532 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:27.371166 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:27.371140 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0b88064d-dcff-40b1-af6a-195f267576c6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t6rnf\" (UID: \"0b88064d-dcff-40b1-af6a-195f267576c6\") " pod="openshift-insights/insights-runtime-extractor-t6rnf" Jul 05 13:19:27.371260 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:27.371244 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0b88064d-dcff-40b1-af6a-195f267576c6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t6rnf\" (UID: \"0b88064d-dcff-40b1-af6a-195f267576c6\") " pod="openshift-insights/insights-runtime-extractor-t6rnf" Jul 05 13:19:27.371311 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:27.371296 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 05 13:19:27.371369 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:27.371359 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b88064d-dcff-40b1-af6a-195f267576c6-insights-runtime-extractor-tls podName:0b88064d-dcff-40b1-af6a-195f267576c6 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:31.37134378 +0000 UTC m=+44.267001849 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0b88064d-dcff-40b1-af6a-195f267576c6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t6rnf" (UID: "0b88064d-dcff-40b1-af6a-195f267576c6") : secret "insights-runtime-extractor-tls" not found Jul 05 13:19:27.371757 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:27.371738 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0b88064d-dcff-40b1-af6a-195f267576c6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t6rnf\" (UID: \"0b88064d-dcff-40b1-af6a-195f267576c6\") " pod="openshift-insights/insights-runtime-extractor-t6rnf" Jul 05 13:19:27.968679 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:27.968646 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:28.034516 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:28.034477 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-qv6qs" event={"ID":"320513ee-d812-4b15-87d2-f0bc0a35eaa6","Type":"ContainerStarted","Data":"8c129d81212197f49ee109d5be732717db4934cca524d763ddb65961106c06ac"} Jul 05 13:19:28.083400 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:28.083359 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-qv6qs" podStartSLOduration=2.117608727 podStartE2EDuration="4.083342272s" totalImagesPullingTime="1.965733545s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:19:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:19:28.082895532 +0000 UTC m=+40.978553620" watchObservedRunningTime="2026-07-05 13:19:28.083342272 +0000 UTC m=+40.979000360" Jul 05 13:19:28.214000 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:28.213967 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:30.010489 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:30.010455 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:30.293390 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:30.293305 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/80ca46e3-b000-4760-89ad-2ab808957f27-registry-tls\") pod \"image-registry-78fc4b5dfd-vrwbb\" (UID: \"80ca46e3-b000-4760-89ad-2ab808957f27\") " pod="openshift-image-registry/image-registry-78fc4b5dfd-vrwbb" Jul 05 13:19:30.293390 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:30.293351 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d52daf5f-c799-45a9-b187-a9613bfe066c-metrics-tls\") pod \"dns-default-gvrpq\" (UID: \"d52daf5f-c799-45a9-b187-a9613bfe066c\") " pod="openshift-dns/dns-default-gvrpq" Jul 05 13:19:30.293558 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:30.293446 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 05 13:19:30.293558 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:30.293461 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 05 13:19:30.293558 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:30.293484 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78fc4b5dfd-vrwbb: secret "image-registry-tls" not found Jul 05 13:19:30.293558 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:30.293495 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d52daf5f-c799-45a9-b187-a9613bfe066c-metrics-tls podName:d52daf5f-c799-45a9-b187-a9613bfe066c nodeName:}" failed. No retries permitted until 2026-07-05 13:19:38.293482679 +0000 UTC m=+51.189140747 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d52daf5f-c799-45a9-b187-a9613bfe066c-metrics-tls") pod "dns-default-gvrpq" (UID: "d52daf5f-c799-45a9-b187-a9613bfe066c") : secret "dns-default-metrics-tls" not found Jul 05 13:19:30.293558 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:30.293528 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/80ca46e3-b000-4760-89ad-2ab808957f27-registry-tls podName:80ca46e3-b000-4760-89ad-2ab808957f27 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:38.29351407 +0000 UTC m=+51.189172137 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/80ca46e3-b000-4760-89ad-2ab808957f27-registry-tls") pod "image-registry-78fc4b5dfd-vrwbb" (UID: "80ca46e3-b000-4760-89ad-2ab808957f27") : secret "image-registry-tls" not found Jul 05 13:19:30.445333 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:30.445301 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:30.577224 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:30.577147 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:30.698367 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:30.698334 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:30.841282 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:30.841194 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:30.942093 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:30.942063 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:31.256681 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:31.256646 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:31.403179 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:31.403145 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0b88064d-dcff-40b1-af6a-195f267576c6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t6rnf\" (UID: \"0b88064d-dcff-40b1-af6a-195f267576c6\") " pod="openshift-insights/insights-runtime-extractor-t6rnf" Jul 05 13:19:31.403328 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:31.403302 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 05 13:19:31.403381 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:31.403368 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b88064d-dcff-40b1-af6a-195f267576c6-insights-runtime-extractor-tls podName:0b88064d-dcff-40b1-af6a-195f267576c6 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:39.403349312 +0000 UTC m=+52.299007402 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0b88064d-dcff-40b1-af6a-195f267576c6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t6rnf" (UID: "0b88064d-dcff-40b1-af6a-195f267576c6") : secret "insights-runtime-extractor-tls" not found Jul 05 13:19:31.404087 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:31.404062 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:31.969675 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:31.969644 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:32.146344 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:32.146312 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:32.760214 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:32.760181 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:33.367078 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:33.367047 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:33.538482 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:33.538454 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:33.987565 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:33.987532 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:34.136430 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:34.136393 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:34.210024 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:34.209982 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:34.577690 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:34.577658 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:34.699299 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:34.699274 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:35.143443 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:35.143412 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:35.222349 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:35.222315 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:35.832211 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:35.832127 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:35.949856 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:35.949824 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:36.146536 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:36.146465 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:36.497381 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:36.497346 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:36.580819 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:36.580790 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:36.784354 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:36.784275 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:36.932842 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:36.932812 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:37.727455 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:37.727423 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:37.930927 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:37.930893 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:38.355380 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:38.355351 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d52daf5f-c799-45a9-b187-a9613bfe066c-metrics-tls\") pod \"dns-default-gvrpq\" (UID: \"d52daf5f-c799-45a9-b187-a9613bfe066c\") " pod="openshift-dns/dns-default-gvrpq" Jul 05 13:19:38.355551 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:38.355429 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/80ca46e3-b000-4760-89ad-2ab808957f27-registry-tls\") pod \"image-registry-78fc4b5dfd-vrwbb\" (UID: \"80ca46e3-b000-4760-89ad-2ab808957f27\") " pod="openshift-image-registry/image-registry-78fc4b5dfd-vrwbb" Jul 05 13:19:38.357634 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:38.357603 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:38.358782 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:38.358751 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d52daf5f-c799-45a9-b187-a9613bfe066c-metrics-tls\") pod \"dns-default-gvrpq\" (UID: \"d52daf5f-c799-45a9-b187-a9613bfe066c\") " pod="openshift-dns/dns-default-gvrpq" Jul 05 13:19:38.358782 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:38.358769 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/80ca46e3-b000-4760-89ad-2ab808957f27-registry-tls\") pod \"image-registry-78fc4b5dfd-vrwbb\" (UID: \"80ca46e3-b000-4760-89ad-2ab808957f27\") " pod="openshift-image-registry/image-registry-78fc4b5dfd-vrwbb" Jul 05 13:19:38.400152 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:38.400111 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78fc4b5dfd-vrwbb" Jul 05 13:19:38.418425 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:38.418289 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gvrpq" Jul 05 13:19:38.520398 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:38.520364 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:38.575665 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:38.575637 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-78fc4b5dfd-vrwbb"] Jul 05 13:19:38.580606 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:19:38.580587 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80ca46e3_b000_4760_89ad_2ab808957f27.slice/crio-60df47041f2cb69e19385f3f7a162146623f5557fcf036556105f13d79c3d5b0 WatchSource:0}: Error finding container 60df47041f2cb69e19385f3f7a162146623f5557fcf036556105f13d79c3d5b0: Status 404 returned error can't find the container with id 60df47041f2cb69e19385f3f7a162146623f5557fcf036556105f13d79c3d5b0 Jul 05 13:19:38.613677 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:38.613614 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gvrpq"] Jul 05 13:19:38.619176 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:19:38.619152 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd52daf5f_c799_45a9_b187_a9613bfe066c.slice/crio-06a3fc85087296b6f0ff136eb998d76dae434d851c8058a22c61b7df6afa4925 WatchSource:0}: Error finding container 06a3fc85087296b6f0ff136eb998d76dae434d851c8058a22c61b7df6afa4925: Status 404 returned error can't find the container with id 06a3fc85087296b6f0ff136eb998d76dae434d851c8058a22c61b7df6afa4925 Jul 05 13:19:38.745797 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:38.745725 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:38.947604 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:38.947570 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:39.062417 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:39.062379 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78fc4b5dfd-vrwbb" event={"ID":"80ca46e3-b000-4760-89ad-2ab808957f27","Type":"ContainerStarted","Data":"833edfcb7e977db3b135b35ed8ea05c6c0d45a1f92967199f198e8ece435ce9c"} Jul 05 13:19:39.062599 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:39.062432 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78fc4b5dfd-vrwbb" event={"ID":"80ca46e3-b000-4760-89ad-2ab808957f27","Type":"ContainerStarted","Data":"60df47041f2cb69e19385f3f7a162146623f5557fcf036556105f13d79c3d5b0"} Jul 05 13:19:39.062599 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:39.062496 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-78fc4b5dfd-vrwbb" Jul 05 13:19:39.063529 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:39.063503 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gvrpq" event={"ID":"d52daf5f-c799-45a9-b187-a9613bfe066c","Type":"ContainerStarted","Data":"06a3fc85087296b6f0ff136eb998d76dae434d851c8058a22c61b7df6afa4925"} Jul 05 13:19:39.464366 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:39.464327 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0b88064d-dcff-40b1-af6a-195f267576c6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t6rnf\" (UID: \"0b88064d-dcff-40b1-af6a-195f267576c6\") " pod="openshift-insights/insights-runtime-extractor-t6rnf" Jul 05 13:19:39.466776 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:39.466749 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0b88064d-dcff-40b1-af6a-195f267576c6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t6rnf\" (UID: \"0b88064d-dcff-40b1-af6a-195f267576c6\") " pod="openshift-insights/insights-runtime-extractor-t6rnf" Jul 05 13:19:39.482023 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:39.481992 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-t6rnf" Jul 05 13:19:39.657062 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:39.657007 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-78fc4b5dfd-vrwbb" podStartSLOduration=24.656984836 podStartE2EDuration="24.656984836s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:19:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:19:39.098678936 +0000 UTC m=+51.994337024" watchObservedRunningTime="2026-07-05 13:19:39.656984836 +0000 UTC m=+52.552642926" Jul 05 13:19:39.658029 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:39.657946 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-t6rnf"] Jul 05 13:19:39.661497 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:19:39.661471 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b88064d_dcff_40b1_af6a_195f267576c6.slice/crio-6a0dc898b8494051c6a931fb517714a953e0ce60b3d699a8c59399239e6fd625 WatchSource:0}: Error finding container 6a0dc898b8494051c6a931fb517714a953e0ce60b3d699a8c59399239e6fd625: Status 404 returned error can't find the container with id 6a0dc898b8494051c6a931fb517714a953e0ce60b3d699a8c59399239e6fd625 Jul 05 13:19:39.759748 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:39.759666 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:40.067727 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:40.067653 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t6rnf" event={"ID":"0b88064d-dcff-40b1-af6a-195f267576c6","Type":"ContainerStarted","Data":"6a0dc898b8494051c6a931fb517714a953e0ce60b3d699a8c59399239e6fd625"} Jul 05 13:19:40.132639 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:40.132602 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:40.334018 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:40.333924 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:40.548222 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:40.548183 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:40.738933 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:40.738903 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:41.076259 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:41.076218 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gvrpq" event={"ID":"d52daf5f-c799-45a9-b187-a9613bfe066c","Type":"ContainerStarted","Data":"b89310cd79063030459fc48239bd08779ad3271a251cfc6d9696c6b14066f7f5"} Jul 05 13:19:41.077774 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:41.076286 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gvrpq" event={"ID":"d52daf5f-c799-45a9-b187-a9613bfe066c","Type":"ContainerStarted","Data":"58332e1523fa151eed45b6285a352a8eea1b127acdb674a295b451e7865c11b6"} Jul 05 13:19:41.077774 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:41.077124 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-gvrpq" Jul 05 13:19:41.121862 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:41.121798 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-gvrpq" podStartSLOduration=17.094099259 podStartE2EDuration="19.1217722s" totalImagesPullingTime="2.027672941s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:19:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:19:41.121061436 +0000 UTC m=+54.016719518" watchObservedRunningTime="2026-07-05 13:19:41.1217722 +0000 UTC m=+54.017430290" Jul 05 13:19:41.148698 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:41.148666 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:45.090665 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:45.090631 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t6rnf" event={"ID":"0b88064d-dcff-40b1-af6a-195f267576c6","Type":"ContainerStarted","Data":"1d71d23896ef58b03493840cfcb49fd26d1f35e68de1db2702ec3fe2850d0353"} Jul 05 13:19:45.091083 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:45.090690 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t6rnf" event={"ID":"0b88064d-dcff-40b1-af6a-195f267576c6","Type":"ContainerStarted","Data":"bbad8b38e792eabaa37f5fd033d986db5ab4ca48f581b4833296370d5e086f23"} Jul 05 13:19:47.291436 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:47.291387 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:47.384423 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:47.384387 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:47.525083 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:47.525001 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:47.883233 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:47.883144 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:47.982362 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:47.982330 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:48.000099 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:48.000079 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bzhlz" Jul 05 13:19:48.079498 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:48.079470 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:48.101646 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:48.101621 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t6rnf" event={"ID":"0b88064d-dcff-40b1-af6a-195f267576c6","Type":"ContainerStarted","Data":"9a4683a741ed7d6a1d8153cfd8a25dd992f5f6848e59dfbe618917f51236d17b"} Jul 05 13:19:48.958140 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:48.958100 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:49.169508 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:49.169466 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:49.471826 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:49.471790 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:49.708843 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:49.708811 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:49.834604 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:49.834529 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:49.873905 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:49.873856 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-t6rnf" podStartSLOduration=19.4601128 podStartE2EDuration="26.873839636s" totalImagesPullingTime="7.413726836s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:19:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:19:48.128810407 +0000 UTC m=+61.024468498" watchObservedRunningTime="2026-07-05 13:19:49.873839636 +0000 UTC m=+62.769497725" Jul 05 13:19:49.874362 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:49.874347 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-57c7d758f-z9nxw"] Jul 05 13:19:49.877077 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:49.877062 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57c7d758f-z9nxw" Jul 05 13:19:49.881912 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:49.881896 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 05 13:19:49.882491 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:49.882473 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 05 13:19:49.882576 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:49.882550 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 05 13:19:49.883143 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:49.883126 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 05 13:19:49.894943 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:49.894920 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-57c7d758f-z9nxw"] Jul 05 13:19:49.939427 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:49.939405 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-78fc4b5dfd-vrwbb"] Jul 05 13:19:49.940163 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:49.940139 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-khbr5"] Jul 05 13:19:49.941983 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:49.941969 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-khbr5" Jul 05 13:19:49.944194 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:49.944171 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-4t7js\"" Jul 05 13:19:49.944291 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:49.944261 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 05 13:19:49.944549 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:49.944532 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 05 13:19:49.944814 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:49.944794 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a3c35e89-4240-4723-a51d-9b7c89f594df-tmp\") pod \"klusterlet-addon-workmgr-57c7d758f-z9nxw\" (UID: \"a3c35e89-4240-4723-a51d-9b7c89f594df\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57c7d758f-z9nxw" Jul 05 13:19:49.944884 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:49.944867 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcdlp\" (UniqueName: \"kubernetes.io/projected/a3c35e89-4240-4723-a51d-9b7c89f594df-kube-api-access-wcdlp\") pod \"klusterlet-addon-workmgr-57c7d758f-z9nxw\" (UID: \"a3c35e89-4240-4723-a51d-9b7c89f594df\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57c7d758f-z9nxw" Jul 05 13:19:49.944923 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:49.944900 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a3c35e89-4240-4723-a51d-9b7c89f594df-klusterlet-config\") pod \"klusterlet-addon-workmgr-57c7d758f-z9nxw\" (UID: \"a3c35e89-4240-4723-a51d-9b7c89f594df\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57c7d758f-z9nxw" Jul 05 13:19:49.962410 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:49.962388 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-khbr5"] Jul 05 13:19:50.006589 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:50.006562 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:50.045565 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:50.045534 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a3c35e89-4240-4723-a51d-9b7c89f594df-tmp\") pod \"klusterlet-addon-workmgr-57c7d758f-z9nxw\" (UID: \"a3c35e89-4240-4723-a51d-9b7c89f594df\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57c7d758f-z9nxw" Jul 05 13:19:50.045685 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:50.045571 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5r478\" (UniqueName: \"kubernetes.io/projected/9f72532b-b2f9-42a8-90ae-f0d74cfe7a04-kube-api-access-5r478\") pod \"downloads-685bfb5f64-khbr5\" (UID: \"9f72532b-b2f9-42a8-90ae-f0d74cfe7a04\") " pod="openshift-console/downloads-685bfb5f64-khbr5" Jul 05 13:19:50.045685 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:50.045638 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcdlp\" (UniqueName: \"kubernetes.io/projected/a3c35e89-4240-4723-a51d-9b7c89f594df-kube-api-access-wcdlp\") pod \"klusterlet-addon-workmgr-57c7d758f-z9nxw\" (UID: \"a3c35e89-4240-4723-a51d-9b7c89f594df\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57c7d758f-z9nxw" Jul 05 13:19:50.045685 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:50.045668 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a3c35e89-4240-4723-a51d-9b7c89f594df-klusterlet-config\") pod \"klusterlet-addon-workmgr-57c7d758f-z9nxw\" (UID: \"a3c35e89-4240-4723-a51d-9b7c89f594df\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57c7d758f-z9nxw" Jul 05 13:19:50.046000 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:50.045974 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a3c35e89-4240-4723-a51d-9b7c89f594df-tmp\") pod \"klusterlet-addon-workmgr-57c7d758f-z9nxw\" (UID: \"a3c35e89-4240-4723-a51d-9b7c89f594df\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57c7d758f-z9nxw" Jul 05 13:19:50.048143 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:50.048125 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a3c35e89-4240-4723-a51d-9b7c89f594df-klusterlet-config\") pod \"klusterlet-addon-workmgr-57c7d758f-z9nxw\" (UID: \"a3c35e89-4240-4723-a51d-9b7c89f594df\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57c7d758f-z9nxw" Jul 05 13:19:50.074189 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:50.074165 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcdlp\" (UniqueName: \"kubernetes.io/projected/a3c35e89-4240-4723-a51d-9b7c89f594df-kube-api-access-wcdlp\") pod \"klusterlet-addon-workmgr-57c7d758f-z9nxw\" (UID: \"a3c35e89-4240-4723-a51d-9b7c89f594df\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57c7d758f-z9nxw" Jul 05 13:19:50.085978 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:50.085898 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-db95784f7-lfv57"] Jul 05 13:19:50.088037 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:50.088019 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-db95784f7-lfv57" Jul 05 13:19:50.121550 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:50.121521 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-db95784f7-lfv57"] Jul 05 13:19:50.127614 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:50.127584 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:50.147045 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:50.147013 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1be77740-54dc-45dc-a42f-afb9356c8023-image-registry-private-configuration\") pod \"image-registry-db95784f7-lfv57\" (UID: \"1be77740-54dc-45dc-a42f-afb9356c8023\") " pod="openshift-image-registry/image-registry-db95784f7-lfv57" Jul 05 13:19:50.147045 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:50.147051 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7xpq\" (UniqueName: \"kubernetes.io/projected/1be77740-54dc-45dc-a42f-afb9356c8023-kube-api-access-p7xpq\") pod \"image-registry-db95784f7-lfv57\" (UID: \"1be77740-54dc-45dc-a42f-afb9356c8023\") " pod="openshift-image-registry/image-registry-db95784f7-lfv57" Jul 05 13:19:50.147255 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:50.147078 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1be77740-54dc-45dc-a42f-afb9356c8023-ca-trust-extracted\") pod \"image-registry-db95784f7-lfv57\" (UID: \"1be77740-54dc-45dc-a42f-afb9356c8023\") " pod="openshift-image-registry/image-registry-db95784f7-lfv57" Jul 05 13:19:50.147255 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:50.147110 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1be77740-54dc-45dc-a42f-afb9356c8023-registry-tls\") pod \"image-registry-db95784f7-lfv57\" (UID: \"1be77740-54dc-45dc-a42f-afb9356c8023\") " pod="openshift-image-registry/image-registry-db95784f7-lfv57" Jul 05 13:19:50.147255 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:50.147163 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1be77740-54dc-45dc-a42f-afb9356c8023-registry-certificates\") pod \"image-registry-db95784f7-lfv57\" (UID: \"1be77740-54dc-45dc-a42f-afb9356c8023\") " pod="openshift-image-registry/image-registry-db95784f7-lfv57" Jul 05 13:19:50.147255 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:50.147192 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5r478\" (UniqueName: \"kubernetes.io/projected/9f72532b-b2f9-42a8-90ae-f0d74cfe7a04-kube-api-access-5r478\") pod \"downloads-685bfb5f64-khbr5\" (UID: \"9f72532b-b2f9-42a8-90ae-f0d74cfe7a04\") " pod="openshift-console/downloads-685bfb5f64-khbr5" Jul 05 13:19:50.147255 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:50.147224 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1be77740-54dc-45dc-a42f-afb9356c8023-installation-pull-secrets\") pod \"image-registry-db95784f7-lfv57\" (UID: \"1be77740-54dc-45dc-a42f-afb9356c8023\") " pod="openshift-image-registry/image-registry-db95784f7-lfv57" Jul 05 13:19:50.147489 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:50.147343 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1be77740-54dc-45dc-a42f-afb9356c8023-bound-sa-token\") pod \"image-registry-db95784f7-lfv57\" (UID: \"1be77740-54dc-45dc-a42f-afb9356c8023\") " pod="openshift-image-registry/image-registry-db95784f7-lfv57" Jul 05 13:19:50.147489 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:50.147381 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1be77740-54dc-45dc-a42f-afb9356c8023-trusted-ca\") pod \"image-registry-db95784f7-lfv57\" (UID: \"1be77740-54dc-45dc-a42f-afb9356c8023\") " pod="openshift-image-registry/image-registry-db95784f7-lfv57" Jul 05 13:19:50.166306 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:50.166274 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5r478\" (UniqueName: \"kubernetes.io/projected/9f72532b-b2f9-42a8-90ae-f0d74cfe7a04-kube-api-access-5r478\") pod \"downloads-685bfb5f64-khbr5\" (UID: \"9f72532b-b2f9-42a8-90ae-f0d74cfe7a04\") " pod="openshift-console/downloads-685bfb5f64-khbr5" Jul 05 13:19:50.186692 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:50.186660 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57c7d758f-z9nxw" Jul 05 13:19:50.248293 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:50.248121 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1be77740-54dc-45dc-a42f-afb9356c8023-bound-sa-token\") pod \"image-registry-db95784f7-lfv57\" (UID: \"1be77740-54dc-45dc-a42f-afb9356c8023\") " pod="openshift-image-registry/image-registry-db95784f7-lfv57" Jul 05 13:19:50.248293 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:50.248171 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1be77740-54dc-45dc-a42f-afb9356c8023-trusted-ca\") pod \"image-registry-db95784f7-lfv57\" (UID: \"1be77740-54dc-45dc-a42f-afb9356c8023\") " pod="openshift-image-registry/image-registry-db95784f7-lfv57" Jul 05 13:19:50.249668 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:50.248524 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1be77740-54dc-45dc-a42f-afb9356c8023-image-registry-private-configuration\") pod \"image-registry-db95784f7-lfv57\" (UID: \"1be77740-54dc-45dc-a42f-afb9356c8023\") " pod="openshift-image-registry/image-registry-db95784f7-lfv57" Jul 05 13:19:50.249668 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:50.248569 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7xpq\" (UniqueName: \"kubernetes.io/projected/1be77740-54dc-45dc-a42f-afb9356c8023-kube-api-access-p7xpq\") pod \"image-registry-db95784f7-lfv57\" (UID: \"1be77740-54dc-45dc-a42f-afb9356c8023\") " pod="openshift-image-registry/image-registry-db95784f7-lfv57" Jul 05 13:19:50.249668 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:50.248639 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1be77740-54dc-45dc-a42f-afb9356c8023-ca-trust-extracted\") pod \"image-registry-db95784f7-lfv57\" (UID: \"1be77740-54dc-45dc-a42f-afb9356c8023\") " pod="openshift-image-registry/image-registry-db95784f7-lfv57" Jul 05 13:19:50.249668 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:50.248668 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1be77740-54dc-45dc-a42f-afb9356c8023-registry-tls\") pod \"image-registry-db95784f7-lfv57\" (UID: \"1be77740-54dc-45dc-a42f-afb9356c8023\") " pod="openshift-image-registry/image-registry-db95784f7-lfv57" Jul 05 13:19:50.249668 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:50.248701 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1be77740-54dc-45dc-a42f-afb9356c8023-registry-certificates\") pod \"image-registry-db95784f7-lfv57\" (UID: \"1be77740-54dc-45dc-a42f-afb9356c8023\") " pod="openshift-image-registry/image-registry-db95784f7-lfv57" Jul 05 13:19:50.249668 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:50.248756 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1be77740-54dc-45dc-a42f-afb9356c8023-installation-pull-secrets\") pod \"image-registry-db95784f7-lfv57\" (UID: \"1be77740-54dc-45dc-a42f-afb9356c8023\") " pod="openshift-image-registry/image-registry-db95784f7-lfv57" Jul 05 13:19:50.249668 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:50.249438 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1be77740-54dc-45dc-a42f-afb9356c8023-trusted-ca\") pod \"image-registry-db95784f7-lfv57\" (UID: \"1be77740-54dc-45dc-a42f-afb9356c8023\") " pod="openshift-image-registry/image-registry-db95784f7-lfv57" Jul 05 13:19:50.250405 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:50.250381 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-khbr5" Jul 05 13:19:50.250556 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:50.250535 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1be77740-54dc-45dc-a42f-afb9356c8023-ca-trust-extracted\") pod \"image-registry-db95784f7-lfv57\" (UID: \"1be77740-54dc-45dc-a42f-afb9356c8023\") " pod="openshift-image-registry/image-registry-db95784f7-lfv57" Jul 05 13:19:50.250624 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:50.250575 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1be77740-54dc-45dc-a42f-afb9356c8023-registry-certificates\") pod \"image-registry-db95784f7-lfv57\" (UID: \"1be77740-54dc-45dc-a42f-afb9356c8023\") " pod="openshift-image-registry/image-registry-db95784f7-lfv57" Jul 05 13:19:50.251086 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:50.251024 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:50.252059 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:50.252036 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1be77740-54dc-45dc-a42f-afb9356c8023-installation-pull-secrets\") pod \"image-registry-db95784f7-lfv57\" (UID: \"1be77740-54dc-45dc-a42f-afb9356c8023\") " pod="openshift-image-registry/image-registry-db95784f7-lfv57" Jul 05 13:19:50.252143 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:50.252097 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1be77740-54dc-45dc-a42f-afb9356c8023-registry-tls\") pod \"image-registry-db95784f7-lfv57\" (UID: \"1be77740-54dc-45dc-a42f-afb9356c8023\") " pod="openshift-image-registry/image-registry-db95784f7-lfv57" Jul 05 13:19:50.252572 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:50.252554 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1be77740-54dc-45dc-a42f-afb9356c8023-image-registry-private-configuration\") pod \"image-registry-db95784f7-lfv57\" (UID: \"1be77740-54dc-45dc-a42f-afb9356c8023\") " pod="openshift-image-registry/image-registry-db95784f7-lfv57" Jul 05 13:19:50.271306 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:50.271283 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7xpq\" (UniqueName: \"kubernetes.io/projected/1be77740-54dc-45dc-a42f-afb9356c8023-kube-api-access-p7xpq\") pod \"image-registry-db95784f7-lfv57\" (UID: \"1be77740-54dc-45dc-a42f-afb9356c8023\") " pod="openshift-image-registry/image-registry-db95784f7-lfv57" Jul 05 13:19:50.271898 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:50.271883 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1be77740-54dc-45dc-a42f-afb9356c8023-bound-sa-token\") pod \"image-registry-db95784f7-lfv57\" (UID: \"1be77740-54dc-45dc-a42f-afb9356c8023\") " pod="openshift-image-registry/image-registry-db95784f7-lfv57" Jul 05 13:19:50.334160 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:50.334127 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-57c7d758f-z9nxw"] Jul 05 13:19:50.337731 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:19:50.337661 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3c35e89_4240_4723_a51d_9b7c89f594df.slice/crio-f8814d263d224c1246cb1cf1121ad90f744577318787691a1a05f1527b9877d9 WatchSource:0}: Error finding container f8814d263d224c1246cb1cf1121ad90f744577318787691a1a05f1527b9877d9: Status 404 returned error can't find the container with id f8814d263d224c1246cb1cf1121ad90f744577318787691a1a05f1527b9877d9 Jul 05 13:19:50.344518 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:50.344489 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:50.392095 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:50.392069 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-khbr5"] Jul 05 13:19:50.394738 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:19:50.394711 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f72532b_b2f9_42a8_90ae_f0d74cfe7a04.slice/crio-adbb78bd9c05fdfe32ff3760985b61dd9538c8d49d7ac50ccfe797f609e0864f WatchSource:0}: Error finding container adbb78bd9c05fdfe32ff3760985b61dd9538c8d49d7ac50ccfe797f609e0864f: Status 404 returned error can't find the container with id adbb78bd9c05fdfe32ff3760985b61dd9538c8d49d7ac50ccfe797f609e0864f Jul 05 13:19:50.396185 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:50.396163 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-db95784f7-lfv57" Jul 05 13:19:50.426852 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:50.426820 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:50.530838 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:50.530289 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-db95784f7-lfv57"] Jul 05 13:19:50.535458 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:19:50.535430 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1be77740_54dc_45dc_a42f_afb9356c8023.slice/crio-0d0a370a244c40ec1ee23df8e13da346b9b56e859cdb844bc5cdc51c4c33eca8 WatchSource:0}: Error finding container 0d0a370a244c40ec1ee23df8e13da346b9b56e859cdb844bc5cdc51c4c33eca8: Status 404 returned error can't find the container with id 0d0a370a244c40ec1ee23df8e13da346b9b56e859cdb844bc5cdc51c4c33eca8 Jul 05 13:19:51.113183 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:51.113119 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57c7d758f-z9nxw" event={"ID":"a3c35e89-4240-4723-a51d-9b7c89f594df","Type":"ContainerStarted","Data":"f8814d263d224c1246cb1cf1121ad90f744577318787691a1a05f1527b9877d9"} Jul 05 13:19:51.116048 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:51.115261 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-db95784f7-lfv57" event={"ID":"1be77740-54dc-45dc-a42f-afb9356c8023","Type":"ContainerStarted","Data":"2635f6e74b4eea52240f3d6bf2958df24afb540bc71750de643d273a2d31f19a"} Jul 05 13:19:51.116048 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:51.115310 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-db95784f7-lfv57" event={"ID":"1be77740-54dc-45dc-a42f-afb9356c8023","Type":"ContainerStarted","Data":"0d0a370a244c40ec1ee23df8e13da346b9b56e859cdb844bc5cdc51c4c33eca8"} Jul 05 13:19:51.116048 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:51.116003 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-db95784f7-lfv57" Jul 05 13:19:51.119154 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:51.119111 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-khbr5" event={"ID":"9f72532b-b2f9-42a8-90ae-f0d74cfe7a04","Type":"ContainerStarted","Data":"adbb78bd9c05fdfe32ff3760985b61dd9538c8d49d7ac50ccfe797f609e0864f"} Jul 05 13:19:51.242059 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:51.241934 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:51.344610 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:51.344529 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:51.462186 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:51.461699 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-db95784f7-lfv57" podStartSLOduration=1.461677798 podStartE2EDuration="1.461677798s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:19:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:19:51.143851946 +0000 UTC m=+64.039510037" watchObservedRunningTime="2026-07-05 13:19:51.461677798 +0000 UTC m=+64.357335880" Jul 05 13:19:51.462186 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:51.462143 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:51.462441 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:51.462153 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-mxdnm"] Jul 05 13:19:51.464585 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:51.464559 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-mxdnm" Jul 05 13:19:51.468169 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:51.467630 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 05 13:19:51.468169 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:51.467882 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-m4q2f\"" Jul 05 13:19:51.492053 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:51.492030 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-mxdnm"] Jul 05 13:19:51.547298 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:51.547255 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:51.560100 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:51.560063 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/6936197e-9cd2-4582-a142-0ee0fff64166-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-mxdnm\" (UID: \"6936197e-9cd2-4582-a142-0ee0fff64166\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-mxdnm" Jul 05 13:19:51.639837 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:51.639798 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:51.661595 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:51.661560 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/6936197e-9cd2-4582-a142-0ee0fff64166-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-mxdnm\" (UID: \"6936197e-9cd2-4582-a142-0ee0fff64166\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-mxdnm" Jul 05 13:19:51.665504 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:51.665452 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/6936197e-9cd2-4582-a142-0ee0fff64166-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-mxdnm\" (UID: \"6936197e-9cd2-4582-a142-0ee0fff64166\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-mxdnm" Jul 05 13:19:51.771029 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:51.770918 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:51.777914 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:51.777788 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-mxdnm" Jul 05 13:19:51.845694 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:51.845659 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:52.085535 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:52.084965 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-gvrpq" Jul 05 13:19:52.857164 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:52.857132 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-mxdnm"] Jul 05 13:19:52.860604 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:19:52.860572 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6936197e_9cd2_4582_a142_0ee0fff64166.slice/crio-c55c2626aa82ca9f5f7e72c249f25c5cffd66a71117058bfc126fc500d54711d WatchSource:0}: Error finding container c55c2626aa82ca9f5f7e72c249f25c5cffd66a71117058bfc126fc500d54711d: Status 404 returned error can't find the container with id c55c2626aa82ca9f5f7e72c249f25c5cffd66a71117058bfc126fc500d54711d Jul 05 13:19:53.128627 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:53.128493 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-mxdnm" event={"ID":"6936197e-9cd2-4582-a142-0ee0fff64166","Type":"ContainerStarted","Data":"c55c2626aa82ca9f5f7e72c249f25c5cffd66a71117058bfc126fc500d54711d"} Jul 05 13:19:53.376704 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:53.376664 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2010fc9b-2713-4036-8131-071a9349d36e-metrics-certs\") pod \"network-metrics-daemon-g9tcx\" (UID: \"2010fc9b-2713-4036-8131-071a9349d36e\") " pod="openshift-multus/network-metrics-daemon-g9tcx" Jul 05 13:19:53.376864 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:53.376724 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/33610912-09f3-406f-be0f-a6ffbff4c9ba-original-pull-secret\") pod \"global-pull-secret-syncer-l6cc6\" (UID: \"33610912-09f3-406f-be0f-a6ffbff4c9ba\") " pod="kube-system/global-pull-secret-syncer-l6cc6" Jul 05 13:19:53.379892 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:53.379828 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 05 13:19:53.380618 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:53.380593 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 05 13:19:53.390624 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:53.390603 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/33610912-09f3-406f-be0f-a6ffbff4c9ba-original-pull-secret\") pod \"global-pull-secret-syncer-l6cc6\" (UID: \"33610912-09f3-406f-be0f-a6ffbff4c9ba\") " pod="kube-system/global-pull-secret-syncer-l6cc6" Jul 05 13:19:53.390740 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:53.390602 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2010fc9b-2713-4036-8131-071a9349d36e-metrics-certs\") pod \"network-metrics-daemon-g9tcx\" (UID: \"2010fc9b-2713-4036-8131-071a9349d36e\") " pod="openshift-multus/network-metrics-daemon-g9tcx" Jul 05 13:19:53.443112 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:53.443070 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:53.477376 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:53.477319 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krvhl\" (UniqueName: \"kubernetes.io/projected/09e0d252-5d66-4697-b469-24d2cdf32679-kube-api-access-krvhl\") pod \"network-check-target-2nr4r\" (UID: \"09e0d252-5d66-4697-b469-24d2cdf32679\") " pod="openshift-network-diagnostics/network-check-target-2nr4r" Jul 05 13:19:53.499853 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:53.499774 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7858944fc4-vgmdb"] Jul 05 13:19:53.502590 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:53.502565 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7858944fc4-vgmdb" Jul 05 13:19:53.514976 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:53.514588 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 05 13:19:53.514976 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:19:53.514657 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"console-serving-cert\" is forbidden: User \"system:node:ip-10-0-139-155.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-console\": no relationship found between node 'ip-10-0-139-155.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-console\"/\"console-serving-cert\"" type="*v1.Secret" Jul 05 13:19:53.514976 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:53.514794 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 05 13:19:53.515230 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:53.514943 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 05 13:19:53.515720 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:53.515661 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 05 13:19:53.515908 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:53.515889 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-497td\"" Jul 05 13:19:53.517830 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:53.517722 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 05 13:19:53.536294 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:53.536269 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 05 13:19:53.541277 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:53.541255 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-krvhl\" (UniqueName: \"kubernetes.io/projected/09e0d252-5d66-4697-b469-24d2cdf32679-kube-api-access-krvhl\") pod \"network-check-target-2nr4r\" (UID: \"09e0d252-5d66-4697-b469-24d2cdf32679\") " pod="openshift-network-diagnostics/network-check-target-2nr4r" Jul 05 13:19:53.547808 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:53.547782 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7858944fc4-vgmdb"] Jul 05 13:19:53.577722 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:53.577697 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/02bb34ab-5247-48c9-85e0-d4eb6b4fd00d-console-serving-cert\") pod \"console-7858944fc4-vgmdb\" (UID: \"02bb34ab-5247-48c9-85e0-d4eb6b4fd00d\") " pod="openshift-console/console-7858944fc4-vgmdb" Jul 05 13:19:53.577819 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:53.577738 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/02bb34ab-5247-48c9-85e0-d4eb6b4fd00d-oauth-serving-cert\") pod \"console-7858944fc4-vgmdb\" (UID: \"02bb34ab-5247-48c9-85e0-d4eb6b4fd00d\") " pod="openshift-console/console-7858944fc4-vgmdb" Jul 05 13:19:53.577893 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:53.577828 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79v4q\" (UniqueName: \"kubernetes.io/projected/02bb34ab-5247-48c9-85e0-d4eb6b4fd00d-kube-api-access-79v4q\") pod \"console-7858944fc4-vgmdb\" (UID: \"02bb34ab-5247-48c9-85e0-d4eb6b4fd00d\") " pod="openshift-console/console-7858944fc4-vgmdb" Jul 05 13:19:53.577893 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:53.577857 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/02bb34ab-5247-48c9-85e0-d4eb6b4fd00d-console-oauth-config\") pod \"console-7858944fc4-vgmdb\" (UID: \"02bb34ab-5247-48c9-85e0-d4eb6b4fd00d\") " pod="openshift-console/console-7858944fc4-vgmdb" Jul 05 13:19:53.577893 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:53.577882 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/02bb34ab-5247-48c9-85e0-d4eb6b4fd00d-service-ca\") pod \"console-7858944fc4-vgmdb\" (UID: \"02bb34ab-5247-48c9-85e0-d4eb6b4fd00d\") " pod="openshift-console/console-7858944fc4-vgmdb" Jul 05 13:19:53.578090 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:53.577995 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/02bb34ab-5247-48c9-85e0-d4eb6b4fd00d-console-config\") pod \"console-7858944fc4-vgmdb\" (UID: \"02bb34ab-5247-48c9-85e0-d4eb6b4fd00d\") " pod="openshift-console/console-7858944fc4-vgmdb" Jul 05 13:19:53.582608 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:53.582584 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:53.671213 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:53.671129 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-kn7g2\"" Jul 05 13:19:53.671422 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:53.671407 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l6cc6" Jul 05 13:19:53.676391 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:53.676365 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g9tcx" Jul 05 13:19:53.678338 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:53.678273 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/02bb34ab-5247-48c9-85e0-d4eb6b4fd00d-console-config\") pod \"console-7858944fc4-vgmdb\" (UID: \"02bb34ab-5247-48c9-85e0-d4eb6b4fd00d\") " pod="openshift-console/console-7858944fc4-vgmdb" Jul 05 13:19:53.678338 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:53.678307 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/02bb34ab-5247-48c9-85e0-d4eb6b4fd00d-console-serving-cert\") pod \"console-7858944fc4-vgmdb\" (UID: \"02bb34ab-5247-48c9-85e0-d4eb6b4fd00d\") " pod="openshift-console/console-7858944fc4-vgmdb" Jul 05 13:19:53.678338 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:53.678307 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6n5k8\"" Jul 05 13:19:53.678338 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:53.678328 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/02bb34ab-5247-48c9-85e0-d4eb6b4fd00d-oauth-serving-cert\") pod \"console-7858944fc4-vgmdb\" (UID: \"02bb34ab-5247-48c9-85e0-d4eb6b4fd00d\") " pod="openshift-console/console-7858944fc4-vgmdb" Jul 05 13:19:53.678536 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:53.678377 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79v4q\" (UniqueName: \"kubernetes.io/projected/02bb34ab-5247-48c9-85e0-d4eb6b4fd00d-kube-api-access-79v4q\") pod \"console-7858944fc4-vgmdb\" (UID: \"02bb34ab-5247-48c9-85e0-d4eb6b4fd00d\") " pod="openshift-console/console-7858944fc4-vgmdb" Jul 05 13:19:53.678536 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:53.678400 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/02bb34ab-5247-48c9-85e0-d4eb6b4fd00d-console-oauth-config\") pod \"console-7858944fc4-vgmdb\" (UID: \"02bb34ab-5247-48c9-85e0-d4eb6b4fd00d\") " pod="openshift-console/console-7858944fc4-vgmdb" Jul 05 13:19:53.678536 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:53.678420 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/02bb34ab-5247-48c9-85e0-d4eb6b4fd00d-service-ca\") pod \"console-7858944fc4-vgmdb\" (UID: \"02bb34ab-5247-48c9-85e0-d4eb6b4fd00d\") " pod="openshift-console/console-7858944fc4-vgmdb" Jul 05 13:19:53.679088 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:53.679067 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/02bb34ab-5247-48c9-85e0-d4eb6b4fd00d-service-ca\") pod \"console-7858944fc4-vgmdb\" (UID: \"02bb34ab-5247-48c9-85e0-d4eb6b4fd00d\") " pod="openshift-console/console-7858944fc4-vgmdb" Jul 05 13:19:53.679186 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:53.679105 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/02bb34ab-5247-48c9-85e0-d4eb6b4fd00d-oauth-serving-cert\") pod \"console-7858944fc4-vgmdb\" (UID: \"02bb34ab-5247-48c9-85e0-d4eb6b4fd00d\") " pod="openshift-console/console-7858944fc4-vgmdb" Jul 05 13:19:53.679663 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:53.679639 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/02bb34ab-5247-48c9-85e0-d4eb6b4fd00d-console-config\") pod \"console-7858944fc4-vgmdb\" (UID: \"02bb34ab-5247-48c9-85e0-d4eb6b4fd00d\") " pod="openshift-console/console-7858944fc4-vgmdb" Jul 05 13:19:53.679772 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:53.679713 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2nr4r" Jul 05 13:19:53.682122 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:53.682102 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/02bb34ab-5247-48c9-85e0-d4eb6b4fd00d-console-oauth-config\") pod \"console-7858944fc4-vgmdb\" (UID: \"02bb34ab-5247-48c9-85e0-d4eb6b4fd00d\") " pod="openshift-console/console-7858944fc4-vgmdb" Jul 05 13:19:53.705537 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:53.705502 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-79v4q\" (UniqueName: \"kubernetes.io/projected/02bb34ab-5247-48c9-85e0-d4eb6b4fd00d-kube-api-access-79v4q\") pod \"console-7858944fc4-vgmdb\" (UID: \"02bb34ab-5247-48c9-85e0-d4eb6b4fd00d\") " pod="openshift-console/console-7858944fc4-vgmdb" Jul 05 13:19:54.378471 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:54.378240 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 05 13:19:54.383755 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:54.383710 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/02bb34ab-5247-48c9-85e0-d4eb6b4fd00d-console-serving-cert\") pod \"console-7858944fc4-vgmdb\" (UID: \"02bb34ab-5247-48c9-85e0-d4eb6b4fd00d\") " pod="openshift-console/console-7858944fc4-vgmdb" Jul 05 13:19:54.415286 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:54.415221 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7858944fc4-vgmdb" Jul 05 13:19:54.579696 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:54.579663 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2nr4r"] Jul 05 13:19:54.582801 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:19:54.582739 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09e0d252_5d66_4697_b469_24d2cdf32679.slice/crio-72b15bcdd68e617ae9a38a839d9a441478606eed172f10cb44d68c3ff24e1998 WatchSource:0}: Error finding container 72b15bcdd68e617ae9a38a839d9a441478606eed172f10cb44d68c3ff24e1998: Status 404 returned error can't find the container with id 72b15bcdd68e617ae9a38a839d9a441478606eed172f10cb44d68c3ff24e1998 Jul 05 13:19:54.602466 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:54.602406 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7858944fc4-vgmdb"] Jul 05 13:19:54.605853 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:19:54.605825 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02bb34ab_5247_48c9_85e0_d4eb6b4fd00d.slice/crio-1ba59af80117d2eaeab2d4ec0a05fdb784f44bafd4e7eea57b9b895172680a66 WatchSource:0}: Error finding container 1ba59af80117d2eaeab2d4ec0a05fdb784f44bafd4e7eea57b9b895172680a66: Status 404 returned error can't find the container with id 1ba59af80117d2eaeab2d4ec0a05fdb784f44bafd4e7eea57b9b895172680a66 Jul 05 13:19:54.737132 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:54.737030 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-l6cc6"] Jul 05 13:19:54.746732 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:54.744723 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-g9tcx"] Jul 05 13:19:54.749312 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:19:54.749283 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2010fc9b_2713_4036_8131_071a9349d36e.slice/crio-4a08cd05fbac9d90a697fb1535db91056c29796e1efff082830db669fb27aa69 WatchSource:0}: Error finding container 4a08cd05fbac9d90a697fb1535db91056c29796e1efff082830db669fb27aa69: Status 404 returned error can't find the container with id 4a08cd05fbac9d90a697fb1535db91056c29796e1efff082830db669fb27aa69 Jul 05 13:19:55.138778 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:55.138736 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57c7d758f-z9nxw" event={"ID":"a3c35e89-4240-4723-a51d-9b7c89f594df","Type":"ContainerStarted","Data":"e204e00e9beace59e81c0f614fcd971debe662355dcd42bfe66b6753de907736"} Jul 05 13:19:55.139028 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:55.138924 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57c7d758f-z9nxw" Jul 05 13:19:55.140208 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:55.140158 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-g9tcx" event={"ID":"2010fc9b-2713-4036-8131-071a9349d36e","Type":"ContainerStarted","Data":"4a08cd05fbac9d90a697fb1535db91056c29796e1efff082830db669fb27aa69"} Jul 05 13:19:55.140605 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:55.140585 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57c7d758f-z9nxw" Jul 05 13:19:55.141505 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:55.141484 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7858944fc4-vgmdb" event={"ID":"02bb34ab-5247-48c9-85e0-d4eb6b4fd00d","Type":"ContainerStarted","Data":"1ba59af80117d2eaeab2d4ec0a05fdb784f44bafd4e7eea57b9b895172680a66"} Jul 05 13:19:55.142848 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:55.142823 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2nr4r" event={"ID":"09e0d252-5d66-4697-b469-24d2cdf32679","Type":"ContainerStarted","Data":"72b15bcdd68e617ae9a38a839d9a441478606eed172f10cb44d68c3ff24e1998"} Jul 05 13:19:55.144143 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:55.144109 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-l6cc6" event={"ID":"33610912-09f3-406f-be0f-a6ffbff4c9ba","Type":"ContainerStarted","Data":"680b65b2046056741e12da4d650c062193c78739d682d0176ccf29bc8dc27151"} Jul 05 13:19:55.164217 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:55.164124 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57c7d758f-z9nxw" podStartSLOduration=2.125664178 podStartE2EDuration="6.164107733s" totalImagesPullingTime="4.038443555s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:19:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:19:55.163701056 +0000 UTC m=+68.059359146" watchObservedRunningTime="2026-07-05 13:19:55.164107733 +0000 UTC m=+68.059765823" Jul 05 13:19:55.815134 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:55.815044 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:55.823589 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:55.822474 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-784b9f44d6-4hcbs"] Jul 05 13:19:55.848309 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:55.848244 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-784b9f44d6-4hcbs"] Jul 05 13:19:55.848439 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:55.848373 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-784b9f44d6-4hcbs" Jul 05 13:19:55.877117 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:55.877088 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:55.883742 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:55.883705 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 05 13:19:55.896099 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:55.896070 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fa6433c9-e321-45f0-b5eb-11d0ac362a6a-oauth-serving-cert\") pod \"console-784b9f44d6-4hcbs\" (UID: \"fa6433c9-e321-45f0-b5eb-11d0ac362a6a\") " pod="openshift-console/console-784b9f44d6-4hcbs" Jul 05 13:19:55.896242 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:55.896121 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44rxl\" (UniqueName: \"kubernetes.io/projected/fa6433c9-e321-45f0-b5eb-11d0ac362a6a-kube-api-access-44rxl\") pod \"console-784b9f44d6-4hcbs\" (UID: \"fa6433c9-e321-45f0-b5eb-11d0ac362a6a\") " pod="openshift-console/console-784b9f44d6-4hcbs" Jul 05 13:19:55.896242 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:55.896148 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fa6433c9-e321-45f0-b5eb-11d0ac362a6a-console-config\") pod \"console-784b9f44d6-4hcbs\" (UID: \"fa6433c9-e321-45f0-b5eb-11d0ac362a6a\") " pod="openshift-console/console-784b9f44d6-4hcbs" Jul 05 13:19:55.896242 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:55.896181 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fa6433c9-e321-45f0-b5eb-11d0ac362a6a-service-ca\") pod \"console-784b9f44d6-4hcbs\" (UID: \"fa6433c9-e321-45f0-b5eb-11d0ac362a6a\") " pod="openshift-console/console-784b9f44d6-4hcbs" Jul 05 13:19:55.896242 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:55.896215 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa6433c9-e321-45f0-b5eb-11d0ac362a6a-trusted-ca-bundle\") pod \"console-784b9f44d6-4hcbs\" (UID: \"fa6433c9-e321-45f0-b5eb-11d0ac362a6a\") " pod="openshift-console/console-784b9f44d6-4hcbs" Jul 05 13:19:55.896459 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:55.896259 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fa6433c9-e321-45f0-b5eb-11d0ac362a6a-console-serving-cert\") pod \"console-784b9f44d6-4hcbs\" (UID: \"fa6433c9-e321-45f0-b5eb-11d0ac362a6a\") " pod="openshift-console/console-784b9f44d6-4hcbs" Jul 05 13:19:55.896459 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:55.896294 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fa6433c9-e321-45f0-b5eb-11d0ac362a6a-console-oauth-config\") pod \"console-784b9f44d6-4hcbs\" (UID: \"fa6433c9-e321-45f0-b5eb-11d0ac362a6a\") " pod="openshift-console/console-784b9f44d6-4hcbs" Jul 05 13:19:55.996795 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:55.996752 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fa6433c9-e321-45f0-b5eb-11d0ac362a6a-oauth-serving-cert\") pod \"console-784b9f44d6-4hcbs\" (UID: \"fa6433c9-e321-45f0-b5eb-11d0ac362a6a\") " pod="openshift-console/console-784b9f44d6-4hcbs" Jul 05 13:19:55.996981 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:55.996815 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44rxl\" (UniqueName: \"kubernetes.io/projected/fa6433c9-e321-45f0-b5eb-11d0ac362a6a-kube-api-access-44rxl\") pod \"console-784b9f44d6-4hcbs\" (UID: \"fa6433c9-e321-45f0-b5eb-11d0ac362a6a\") " pod="openshift-console/console-784b9f44d6-4hcbs" Jul 05 13:19:55.996981 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:55.996843 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fa6433c9-e321-45f0-b5eb-11d0ac362a6a-console-config\") pod \"console-784b9f44d6-4hcbs\" (UID: \"fa6433c9-e321-45f0-b5eb-11d0ac362a6a\") " pod="openshift-console/console-784b9f44d6-4hcbs" Jul 05 13:19:55.996981 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:55.996874 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fa6433c9-e321-45f0-b5eb-11d0ac362a6a-service-ca\") pod \"console-784b9f44d6-4hcbs\" (UID: \"fa6433c9-e321-45f0-b5eb-11d0ac362a6a\") " pod="openshift-console/console-784b9f44d6-4hcbs" Jul 05 13:19:55.996981 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:55.996910 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa6433c9-e321-45f0-b5eb-11d0ac362a6a-trusted-ca-bundle\") pod \"console-784b9f44d6-4hcbs\" (UID: \"fa6433c9-e321-45f0-b5eb-11d0ac362a6a\") " pod="openshift-console/console-784b9f44d6-4hcbs" Jul 05 13:19:55.996981 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:55.996971 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fa6433c9-e321-45f0-b5eb-11d0ac362a6a-console-serving-cert\") pod \"console-784b9f44d6-4hcbs\" (UID: \"fa6433c9-e321-45f0-b5eb-11d0ac362a6a\") " pod="openshift-console/console-784b9f44d6-4hcbs" Jul 05 13:19:55.997332 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:55.997007 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fa6433c9-e321-45f0-b5eb-11d0ac362a6a-console-oauth-config\") pod \"console-784b9f44d6-4hcbs\" (UID: \"fa6433c9-e321-45f0-b5eb-11d0ac362a6a\") " pod="openshift-console/console-784b9f44d6-4hcbs" Jul 05 13:19:55.998158 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:55.998129 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fa6433c9-e321-45f0-b5eb-11d0ac362a6a-console-config\") pod \"console-784b9f44d6-4hcbs\" (UID: \"fa6433c9-e321-45f0-b5eb-11d0ac362a6a\") " pod="openshift-console/console-784b9f44d6-4hcbs" Jul 05 13:19:55.998772 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:55.998692 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fa6433c9-e321-45f0-b5eb-11d0ac362a6a-oauth-serving-cert\") pod \"console-784b9f44d6-4hcbs\" (UID: \"fa6433c9-e321-45f0-b5eb-11d0ac362a6a\") " pod="openshift-console/console-784b9f44d6-4hcbs" Jul 05 13:19:55.999842 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:55.999816 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa6433c9-e321-45f0-b5eb-11d0ac362a6a-trusted-ca-bundle\") pod \"console-784b9f44d6-4hcbs\" (UID: \"fa6433c9-e321-45f0-b5eb-11d0ac362a6a\") " pod="openshift-console/console-784b9f44d6-4hcbs" Jul 05 13:19:56.001270 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:56.000403 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fa6433c9-e321-45f0-b5eb-11d0ac362a6a-service-ca\") pod \"console-784b9f44d6-4hcbs\" (UID: \"fa6433c9-e321-45f0-b5eb-11d0ac362a6a\") " pod="openshift-console/console-784b9f44d6-4hcbs" Jul 05 13:19:56.008788 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:56.008762 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fa6433c9-e321-45f0-b5eb-11d0ac362a6a-console-serving-cert\") pod \"console-784b9f44d6-4hcbs\" (UID: \"fa6433c9-e321-45f0-b5eb-11d0ac362a6a\") " pod="openshift-console/console-784b9f44d6-4hcbs" Jul 05 13:19:56.008898 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:56.008815 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fa6433c9-e321-45f0-b5eb-11d0ac362a6a-console-oauth-config\") pod \"console-784b9f44d6-4hcbs\" (UID: \"fa6433c9-e321-45f0-b5eb-11d0ac362a6a\") " pod="openshift-console/console-784b9f44d6-4hcbs" Jul 05 13:19:56.017470 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:56.017424 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-44rxl\" (UniqueName: \"kubernetes.io/projected/fa6433c9-e321-45f0-b5eb-11d0ac362a6a-kube-api-access-44rxl\") pod \"console-784b9f44d6-4hcbs\" (UID: \"fa6433c9-e321-45f0-b5eb-11d0ac362a6a\") " pod="openshift-console/console-784b9f44d6-4hcbs" Jul 05 13:19:56.152092 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:56.152012 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-mxdnm" event={"ID":"6936197e-9cd2-4582-a142-0ee0fff64166","Type":"ContainerStarted","Data":"edf138f4a976b045edb16306522d1ae9caa0dcdb59cf5f6386d3739489d18138"} Jul 05 13:19:56.152393 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:56.152371 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-mxdnm" Jul 05 13:19:56.158660 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:56.158636 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-mxdnm" Jul 05 13:19:56.167942 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:56.167917 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-784b9f44d6-4hcbs" Jul 05 13:19:56.171288 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:56.171118 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-mxdnm" podStartSLOduration=2.447090686 podStartE2EDuration="5.171070564s" totalImagesPullingTime="2.723979878s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:19:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:19:56.170357744 +0000 UTC m=+69.066015834" watchObservedRunningTime="2026-07-05 13:19:56.171070564 +0000 UTC m=+69.066728656" Jul 05 13:19:56.439771 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:56.439741 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:56.513700 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:56.513656 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:56.881861 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:56.881827 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-784b9f44d6-4hcbs"] Jul 05 13:19:57.049428 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:19:57.049343 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa6433c9_e321_45f0_b5eb_11d0ac362a6a.slice/crio-c2e8893df0ac9ed40aaae8c3923cffef83addee0e20e7881ed5f63cc984559fc WatchSource:0}: Error finding container c2e8893df0ac9ed40aaae8c3923cffef83addee0e20e7881ed5f63cc984559fc: Status 404 returned error can't find the container with id c2e8893df0ac9ed40aaae8c3923cffef83addee0e20e7881ed5f63cc984559fc Jul 05 13:19:57.158775 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:57.157318 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:57.163368 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:57.163339 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-784b9f44d6-4hcbs" event={"ID":"fa6433c9-e321-45f0-b5eb-11d0ac362a6a","Type":"ContainerStarted","Data":"c2e8893df0ac9ed40aaae8c3923cffef83addee0e20e7881ed5f63cc984559fc"} Jul 05 13:19:58.171872 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:58.171732 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-g9tcx" event={"ID":"2010fc9b-2713-4036-8131-071a9349d36e","Type":"ContainerStarted","Data":"1ba1d86e4bda5eb0360d71a92f352fda9f6d3bd772e76da1f20bdd0650e60a63"} Jul 05 13:19:58.171872 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:58.171814 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-g9tcx" event={"ID":"2010fc9b-2713-4036-8131-071a9349d36e","Type":"ContainerStarted","Data":"3df494f1ac1fe5bc8aae4872fed487495cda9d5c2092afc9093f31180c664938"} Jul 05 13:19:58.190341 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:58.190281 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-g9tcx" podStartSLOduration=68.859059611 podStartE2EDuration="1m11.190259288s" totalImagesPullingTime="2.331199677s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:18:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:19:58.188003693 +0000 UTC m=+71.083661783" watchObservedRunningTime="2026-07-05 13:19:58.190259288 +0000 UTC m=+71.085917378" Jul 05 13:19:58.288666 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:58.288632 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:58.365310 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:58.365245 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:59.008912 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:59.008835 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:19:59.946580 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:19:59.946549 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-78fc4b5dfd-vrwbb" Jul 05 13:20:01.238421 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:01.238377 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:20:01.336833 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:01.336751 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:20:01.698712 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:01.698677 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:20:01.926757 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:01.926711 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:20:02.037901 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.037808 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:20:02.141366 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.141323 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:20:02.157376 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.157337 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-hvhgg"] Jul 05 13:20:02.159608 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.159589 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hvhgg" Jul 05 13:20:02.164770 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.164747 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-7rv2x\"" Jul 05 13:20:02.165894 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.165750 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 05 13:20:02.165894 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.165770 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 05 13:20:02.165894 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.165841 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 05 13:20:02.165894 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.165862 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 05 13:20:02.165894 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.165775 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 05 13:20:02.177226 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.177199 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-hvhgg"] Jul 05 13:20:02.252306 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.252268 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5711bc78-2a69-45f9-921f-0e00b7d51e39-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-hvhgg\" (UID: \"5711bc78-2a69-45f9-921f-0e00b7d51e39\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hvhgg" Jul 05 13:20:02.252740 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.252327 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5711bc78-2a69-45f9-921f-0e00b7d51e39-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-hvhgg\" (UID: \"5711bc78-2a69-45f9-921f-0e00b7d51e39\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hvhgg" Jul 05 13:20:02.252740 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.252428 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2555\" (UniqueName: \"kubernetes.io/projected/5711bc78-2a69-45f9-921f-0e00b7d51e39-kube-api-access-v2555\") pod \"openshift-state-metrics-54cf96dfcf-hvhgg\" (UID: \"5711bc78-2a69-45f9-921f-0e00b7d51e39\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hvhgg" Jul 05 13:20:02.252740 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.252461 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5711bc78-2a69-45f9-921f-0e00b7d51e39-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-hvhgg\" (UID: \"5711bc78-2a69-45f9-921f-0e00b7d51e39\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hvhgg" Jul 05 13:20:02.326822 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.326789 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:20:02.329221 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.329198 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-6zff6"] Jul 05 13:20:02.331550 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.331529 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6zff6" Jul 05 13:20:02.334919 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.334887 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 05 13:20:02.335147 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.335111 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-vrcb7\"" Jul 05 13:20:02.335233 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.335152 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 05 13:20:02.335293 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.335229 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 05 13:20:02.352966 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.352927 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5711bc78-2a69-45f9-921f-0e00b7d51e39-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-hvhgg\" (UID: \"5711bc78-2a69-45f9-921f-0e00b7d51e39\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hvhgg" Jul 05 13:20:02.353060 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.353023 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2555\" (UniqueName: \"kubernetes.io/projected/5711bc78-2a69-45f9-921f-0e00b7d51e39-kube-api-access-v2555\") pod \"openshift-state-metrics-54cf96dfcf-hvhgg\" (UID: \"5711bc78-2a69-45f9-921f-0e00b7d51e39\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hvhgg" Jul 05 13:20:02.353060 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.353053 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5711bc78-2a69-45f9-921f-0e00b7d51e39-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-hvhgg\" (UID: \"5711bc78-2a69-45f9-921f-0e00b7d51e39\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hvhgg" Jul 05 13:20:02.353164 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.353114 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5711bc78-2a69-45f9-921f-0e00b7d51e39-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-hvhgg\" (UID: \"5711bc78-2a69-45f9-921f-0e00b7d51e39\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hvhgg" Jul 05 13:20:02.353868 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.353841 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5711bc78-2a69-45f9-921f-0e00b7d51e39-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-hvhgg\" (UID: \"5711bc78-2a69-45f9-921f-0e00b7d51e39\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hvhgg" Jul 05 13:20:02.355617 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.355595 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5711bc78-2a69-45f9-921f-0e00b7d51e39-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-hvhgg\" (UID: \"5711bc78-2a69-45f9-921f-0e00b7d51e39\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hvhgg" Jul 05 13:20:02.355803 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.355782 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5711bc78-2a69-45f9-921f-0e00b7d51e39-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-hvhgg\" (UID: \"5711bc78-2a69-45f9-921f-0e00b7d51e39\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hvhgg" Jul 05 13:20:02.379757 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.379719 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2555\" (UniqueName: \"kubernetes.io/projected/5711bc78-2a69-45f9-921f-0e00b7d51e39-kube-api-access-v2555\") pod \"openshift-state-metrics-54cf96dfcf-hvhgg\" (UID: \"5711bc78-2a69-45f9-921f-0e00b7d51e39\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hvhgg" Jul 05 13:20:02.453988 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.453940 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1-node-exporter-wtmp\") pod \"node-exporter-6zff6\" (UID: \"ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1\") " pod="openshift-monitoring/node-exporter-6zff6" Jul 05 13:20:02.454182 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.453996 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1-node-exporter-accelerators-collector-config\") pod \"node-exporter-6zff6\" (UID: \"ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1\") " pod="openshift-monitoring/node-exporter-6zff6" Jul 05 13:20:02.454182 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.454036 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6zff6\" (UID: \"ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1\") " pod="openshift-monitoring/node-exporter-6zff6" Jul 05 13:20:02.454182 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.454078 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1-metrics-client-ca\") pod \"node-exporter-6zff6\" (UID: \"ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1\") " pod="openshift-monitoring/node-exporter-6zff6" Jul 05 13:20:02.454182 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.454129 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1-node-exporter-textfile\") pod \"node-exporter-6zff6\" (UID: \"ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1\") " pod="openshift-monitoring/node-exporter-6zff6" Jul 05 13:20:02.454182 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.454154 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1-node-exporter-tls\") pod \"node-exporter-6zff6\" (UID: \"ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1\") " pod="openshift-monitoring/node-exporter-6zff6" Jul 05 13:20:02.454182 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.454174 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58pfj\" (UniqueName: \"kubernetes.io/projected/ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1-kube-api-access-58pfj\") pod \"node-exporter-6zff6\" (UID: \"ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1\") " pod="openshift-monitoring/node-exporter-6zff6" Jul 05 13:20:02.454486 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.454199 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1-sys\") pod \"node-exporter-6zff6\" (UID: \"ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1\") " pod="openshift-monitoring/node-exporter-6zff6" Jul 05 13:20:02.454486 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.454221 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1-root\") pod \"node-exporter-6zff6\" (UID: \"ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1\") " pod="openshift-monitoring/node-exporter-6zff6" Jul 05 13:20:02.470137 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.470096 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hvhgg" Jul 05 13:20:02.555418 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.555328 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6zff6\" (UID: \"ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1\") " pod="openshift-monitoring/node-exporter-6zff6" Jul 05 13:20:02.555418 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.555385 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1-metrics-client-ca\") pod \"node-exporter-6zff6\" (UID: \"ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1\") " pod="openshift-monitoring/node-exporter-6zff6" Jul 05 13:20:02.555648 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.555422 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1-node-exporter-textfile\") pod \"node-exporter-6zff6\" (UID: \"ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1\") " pod="openshift-monitoring/node-exporter-6zff6" Jul 05 13:20:02.555648 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.555449 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1-node-exporter-tls\") pod \"node-exporter-6zff6\" (UID: \"ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1\") " pod="openshift-monitoring/node-exporter-6zff6" Jul 05 13:20:02.555648 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.555478 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58pfj\" (UniqueName: \"kubernetes.io/projected/ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1-kube-api-access-58pfj\") pod \"node-exporter-6zff6\" (UID: \"ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1\") " pod="openshift-monitoring/node-exporter-6zff6" Jul 05 13:20:02.555648 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.555506 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1-sys\") pod \"node-exporter-6zff6\" (UID: \"ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1\") " pod="openshift-monitoring/node-exporter-6zff6" Jul 05 13:20:02.555648 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.555529 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1-root\") pod \"node-exporter-6zff6\" (UID: \"ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1\") " pod="openshift-monitoring/node-exporter-6zff6" Jul 05 13:20:02.555648 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.555597 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1-node-exporter-wtmp\") pod \"node-exporter-6zff6\" (UID: \"ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1\") " pod="openshift-monitoring/node-exporter-6zff6" Jul 05 13:20:02.555648 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.555623 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1-node-exporter-accelerators-collector-config\") pod \"node-exporter-6zff6\" (UID: \"ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1\") " pod="openshift-monitoring/node-exporter-6zff6" Jul 05 13:20:02.556080 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.555900 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1-node-exporter-textfile\") pod \"node-exporter-6zff6\" (UID: \"ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1\") " pod="openshift-monitoring/node-exporter-6zff6" Jul 05 13:20:02.556080 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.555925 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1-root\") pod \"node-exporter-6zff6\" (UID: \"ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1\") " pod="openshift-monitoring/node-exporter-6zff6" Jul 05 13:20:02.556080 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.556005 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1-sys\") pod \"node-exporter-6zff6\" (UID: \"ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1\") " pod="openshift-monitoring/node-exporter-6zff6" Jul 05 13:20:02.556080 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.556056 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1-metrics-client-ca\") pod \"node-exporter-6zff6\" (UID: \"ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1\") " pod="openshift-monitoring/node-exporter-6zff6" Jul 05 13:20:02.556080 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.556066 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1-node-exporter-wtmp\") pod \"node-exporter-6zff6\" (UID: \"ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1\") " pod="openshift-monitoring/node-exporter-6zff6" Jul 05 13:20:02.556333 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.556197 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1-node-exporter-accelerators-collector-config\") pod \"node-exporter-6zff6\" (UID: \"ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1\") " pod="openshift-monitoring/node-exporter-6zff6" Jul 05 13:20:02.558449 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.558341 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6zff6\" (UID: \"ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1\") " pod="openshift-monitoring/node-exporter-6zff6" Jul 05 13:20:02.558578 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.558459 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1-node-exporter-tls\") pod \"node-exporter-6zff6\" (UID: \"ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1\") " pod="openshift-monitoring/node-exporter-6zff6" Jul 05 13:20:02.566547 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.566517 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-58pfj\" (UniqueName: \"kubernetes.io/projected/ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1-kube-api-access-58pfj\") pod \"node-exporter-6zff6\" (UID: \"ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1\") " pod="openshift-monitoring/node-exporter-6zff6" Jul 05 13:20:02.641223 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.641181 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6zff6" Jul 05 13:20:02.809492 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.809417 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:20:02.973737 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:02.973705 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:20:03.123351 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:03.123261 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:20:03.229382 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:03.229347 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:20:03.374813 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:03.374720 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:20:05.381114 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:05.381079 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:20:05.482253 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:05.482220 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:20:05.591237 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:05.591186 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:20:06.025613 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:06.025563 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:20:06.114394 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:06.114362 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:20:06.461515 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:06.461477 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:20:06.566903 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:06.566872 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:20:06.722062 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:06.721975 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:20:06.808398 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:06.808361 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7858944fc4-vgmdb"] Jul 05 13:20:06.831078 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:06.831044 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-b5df8"] Jul 05 13:20:06.887123 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:06.887081 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:20:07.000721 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.000640 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-b5df8"] Jul 05 13:20:07.000721 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.000685 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7c6cbc47f-j45zf"] Jul 05 13:20:07.000939 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.000767 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-b5df8" Jul 05 13:20:07.004398 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.004369 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-kfwf9\"" Jul 05 13:20:07.005305 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.005285 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 05 13:20:07.041296 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.040940 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c6cbc47f-j45zf"] Jul 05 13:20:07.041296 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.041065 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c6cbc47f-j45zf" Jul 05 13:20:07.049743 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.049714 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:20:07.058581 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.058557 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-847bdf55cb-xdd6s"] Jul 05 13:20:07.080812 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.080784 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-847bdf55cb-xdd6s" Jul 05 13:20:07.088823 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.088799 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 05 13:20:07.088977 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.088851 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 05 13:20:07.089576 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.089558 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 05 13:20:07.091249 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.091226 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-9esldnm9f2a6f\"" Jul 05 13:20:07.094267 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.094249 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-kg4mg\"" Jul 05 13:20:07.094800 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.094778 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 05 13:20:07.098162 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.098138 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20ee6668-7a04-483a-8bdc-bc212e16e892-oauth-serving-cert\") pod \"console-7c6cbc47f-j45zf\" (UID: \"20ee6668-7a04-483a-8bdc-bc212e16e892\") " pod="openshift-console/console-7c6cbc47f-j45zf" Jul 05 13:20:07.098255 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.098177 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgrq9\" (UniqueName: \"kubernetes.io/projected/46d56b08-161c-465d-9adc-dbd9273e69c7-kube-api-access-zgrq9\") pod \"monitoring-plugin-d5dcff7cf-b5df8\" (UID: \"46d56b08-161c-465d-9adc-dbd9273e69c7\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-b5df8" Jul 05 13:20:07.098255 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.098196 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20ee6668-7a04-483a-8bdc-bc212e16e892-console-serving-cert\") pod \"console-7c6cbc47f-j45zf\" (UID: \"20ee6668-7a04-483a-8bdc-bc212e16e892\") " pod="openshift-console/console-7c6cbc47f-j45zf" Jul 05 13:20:07.098255 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.098214 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20ee6668-7a04-483a-8bdc-bc212e16e892-service-ca\") pod \"console-7c6cbc47f-j45zf\" (UID: \"20ee6668-7a04-483a-8bdc-bc212e16e892\") " pod="openshift-console/console-7c6cbc47f-j45zf" Jul 05 13:20:07.098402 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.098286 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20ee6668-7a04-483a-8bdc-bc212e16e892-console-config\") pod \"console-7c6cbc47f-j45zf\" (UID: \"20ee6668-7a04-483a-8bdc-bc212e16e892\") " pod="openshift-console/console-7c6cbc47f-j45zf" Jul 05 13:20:07.098402 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.098355 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/46d56b08-161c-465d-9adc-dbd9273e69c7-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-b5df8\" (UID: \"46d56b08-161c-465d-9adc-dbd9273e69c7\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-b5df8" Jul 05 13:20:07.098480 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.098403 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20ee6668-7a04-483a-8bdc-bc212e16e892-console-oauth-config\") pod \"console-7c6cbc47f-j45zf\" (UID: \"20ee6668-7a04-483a-8bdc-bc212e16e892\") " pod="openshift-console/console-7c6cbc47f-j45zf" Jul 05 13:20:07.098480 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.098423 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20ee6668-7a04-483a-8bdc-bc212e16e892-trusted-ca-bundle\") pod \"console-7c6cbc47f-j45zf\" (UID: \"20ee6668-7a04-483a-8bdc-bc212e16e892\") " pod="openshift-console/console-7c6cbc47f-j45zf" Jul 05 13:20:07.098480 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.098443 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtnqr\" (UniqueName: \"kubernetes.io/projected/20ee6668-7a04-483a-8bdc-bc212e16e892-kube-api-access-mtnqr\") pod \"console-7c6cbc47f-j45zf\" (UID: \"20ee6668-7a04-483a-8bdc-bc212e16e892\") " pod="openshift-console/console-7c6cbc47f-j45zf" Jul 05 13:20:07.120025 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.119999 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-847bdf55cb-xdd6s"] Jul 05 13:20:07.199714 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.199679 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/161555b1-1894-4a2c-9770-faac3772cb12-metrics-server-audit-profiles\") pod \"metrics-server-847bdf55cb-xdd6s\" (UID: \"161555b1-1894-4a2c-9770-faac3772cb12\") " pod="openshift-monitoring/metrics-server-847bdf55cb-xdd6s" Jul 05 13:20:07.199940 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.199726 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqvr7\" (UniqueName: \"kubernetes.io/projected/161555b1-1894-4a2c-9770-faac3772cb12-kube-api-access-wqvr7\") pod \"metrics-server-847bdf55cb-xdd6s\" (UID: \"161555b1-1894-4a2c-9770-faac3772cb12\") " pod="openshift-monitoring/metrics-server-847bdf55cb-xdd6s" Jul 05 13:20:07.199940 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.199779 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20ee6668-7a04-483a-8bdc-bc212e16e892-console-oauth-config\") pod \"console-7c6cbc47f-j45zf\" (UID: \"20ee6668-7a04-483a-8bdc-bc212e16e892\") " pod="openshift-console/console-7c6cbc47f-j45zf" Jul 05 13:20:07.199940 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.199807 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20ee6668-7a04-483a-8bdc-bc212e16e892-trusted-ca-bundle\") pod \"console-7c6cbc47f-j45zf\" (UID: \"20ee6668-7a04-483a-8bdc-bc212e16e892\") " pod="openshift-console/console-7c6cbc47f-j45zf" Jul 05 13:20:07.199940 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.199837 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtnqr\" (UniqueName: \"kubernetes.io/projected/20ee6668-7a04-483a-8bdc-bc212e16e892-kube-api-access-mtnqr\") pod \"console-7c6cbc47f-j45zf\" (UID: \"20ee6668-7a04-483a-8bdc-bc212e16e892\") " pod="openshift-console/console-7c6cbc47f-j45zf" Jul 05 13:20:07.199940 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.199881 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20ee6668-7a04-483a-8bdc-bc212e16e892-oauth-serving-cert\") pod \"console-7c6cbc47f-j45zf\" (UID: \"20ee6668-7a04-483a-8bdc-bc212e16e892\") " pod="openshift-console/console-7c6cbc47f-j45zf" Jul 05 13:20:07.199940 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.199911 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgrq9\" (UniqueName: \"kubernetes.io/projected/46d56b08-161c-465d-9adc-dbd9273e69c7-kube-api-access-zgrq9\") pod \"monitoring-plugin-d5dcff7cf-b5df8\" (UID: \"46d56b08-161c-465d-9adc-dbd9273e69c7\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-b5df8" Jul 05 13:20:07.199940 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.199935 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20ee6668-7a04-483a-8bdc-bc212e16e892-console-serving-cert\") pod \"console-7c6cbc47f-j45zf\" (UID: \"20ee6668-7a04-483a-8bdc-bc212e16e892\") " pod="openshift-console/console-7c6cbc47f-j45zf" Jul 05 13:20:07.200380 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.200155 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20ee6668-7a04-483a-8bdc-bc212e16e892-service-ca\") pod \"console-7c6cbc47f-j45zf\" (UID: \"20ee6668-7a04-483a-8bdc-bc212e16e892\") " pod="openshift-console/console-7c6cbc47f-j45zf" Jul 05 13:20:07.200380 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.200210 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20ee6668-7a04-483a-8bdc-bc212e16e892-console-config\") pod \"console-7c6cbc47f-j45zf\" (UID: \"20ee6668-7a04-483a-8bdc-bc212e16e892\") " pod="openshift-console/console-7c6cbc47f-j45zf" Jul 05 13:20:07.200380 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.200255 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/161555b1-1894-4a2c-9770-faac3772cb12-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-847bdf55cb-xdd6s\" (UID: \"161555b1-1894-4a2c-9770-faac3772cb12\") " pod="openshift-monitoring/metrics-server-847bdf55cb-xdd6s" Jul 05 13:20:07.200380 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.200300 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/161555b1-1894-4a2c-9770-faac3772cb12-client-ca-bundle\") pod \"metrics-server-847bdf55cb-xdd6s\" (UID: \"161555b1-1894-4a2c-9770-faac3772cb12\") " pod="openshift-monitoring/metrics-server-847bdf55cb-xdd6s" Jul 05 13:20:07.200380 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.200330 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/46d56b08-161c-465d-9adc-dbd9273e69c7-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-b5df8\" (UID: \"46d56b08-161c-465d-9adc-dbd9273e69c7\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-b5df8" Jul 05 13:20:07.200380 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.200377 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/161555b1-1894-4a2c-9770-faac3772cb12-secret-metrics-server-client-certs\") pod \"metrics-server-847bdf55cb-xdd6s\" (UID: \"161555b1-1894-4a2c-9770-faac3772cb12\") " pod="openshift-monitoring/metrics-server-847bdf55cb-xdd6s" Jul 05 13:20:07.200664 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.200394 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/161555b1-1894-4a2c-9770-faac3772cb12-audit-log\") pod \"metrics-server-847bdf55cb-xdd6s\" (UID: \"161555b1-1894-4a2c-9770-faac3772cb12\") " pod="openshift-monitoring/metrics-server-847bdf55cb-xdd6s" Jul 05 13:20:07.200664 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.200416 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/161555b1-1894-4a2c-9770-faac3772cb12-secret-metrics-server-tls\") pod \"metrics-server-847bdf55cb-xdd6s\" (UID: \"161555b1-1894-4a2c-9770-faac3772cb12\") " pod="openshift-monitoring/metrics-server-847bdf55cb-xdd6s" Jul 05 13:20:07.200778 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.200698 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20ee6668-7a04-483a-8bdc-bc212e16e892-oauth-serving-cert\") pod \"console-7c6cbc47f-j45zf\" (UID: \"20ee6668-7a04-483a-8bdc-bc212e16e892\") " pod="openshift-console/console-7c6cbc47f-j45zf" Jul 05 13:20:07.201078 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.201053 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20ee6668-7a04-483a-8bdc-bc212e16e892-service-ca\") pod \"console-7c6cbc47f-j45zf\" (UID: \"20ee6668-7a04-483a-8bdc-bc212e16e892\") " pod="openshift-console/console-7c6cbc47f-j45zf" Jul 05 13:20:07.201325 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.201284 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20ee6668-7a04-483a-8bdc-bc212e16e892-trusted-ca-bundle\") pod \"console-7c6cbc47f-j45zf\" (UID: \"20ee6668-7a04-483a-8bdc-bc212e16e892\") " pod="openshift-console/console-7c6cbc47f-j45zf" Jul 05 13:20:07.201325 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.201302 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20ee6668-7a04-483a-8bdc-bc212e16e892-console-config\") pod \"console-7c6cbc47f-j45zf\" (UID: \"20ee6668-7a04-483a-8bdc-bc212e16e892\") " pod="openshift-console/console-7c6cbc47f-j45zf" Jul 05 13:20:07.203063 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.203034 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20ee6668-7a04-483a-8bdc-bc212e16e892-console-oauth-config\") pod \"console-7c6cbc47f-j45zf\" (UID: \"20ee6668-7a04-483a-8bdc-bc212e16e892\") " pod="openshift-console/console-7c6cbc47f-j45zf" Jul 05 13:20:07.203229 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.203208 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20ee6668-7a04-483a-8bdc-bc212e16e892-console-serving-cert\") pod \"console-7c6cbc47f-j45zf\" (UID: \"20ee6668-7a04-483a-8bdc-bc212e16e892\") " pod="openshift-console/console-7c6cbc47f-j45zf" Jul 05 13:20:07.203339 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.203242 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/46d56b08-161c-465d-9adc-dbd9273e69c7-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-b5df8\" (UID: \"46d56b08-161c-465d-9adc-dbd9273e69c7\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-b5df8" Jul 05 13:20:07.213074 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.213050 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgrq9\" (UniqueName: \"kubernetes.io/projected/46d56b08-161c-465d-9adc-dbd9273e69c7-kube-api-access-zgrq9\") pod \"monitoring-plugin-d5dcff7cf-b5df8\" (UID: \"46d56b08-161c-465d-9adc-dbd9273e69c7\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-b5df8" Jul 05 13:20:07.214482 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.214443 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtnqr\" (UniqueName: \"kubernetes.io/projected/20ee6668-7a04-483a-8bdc-bc212e16e892-kube-api-access-mtnqr\") pod \"console-7c6cbc47f-j45zf\" (UID: \"20ee6668-7a04-483a-8bdc-bc212e16e892\") " pod="openshift-console/console-7c6cbc47f-j45zf" Jul 05 13:20:07.301150 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.301055 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/161555b1-1894-4a2c-9770-faac3772cb12-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-847bdf55cb-xdd6s\" (UID: \"161555b1-1894-4a2c-9770-faac3772cb12\") " pod="openshift-monitoring/metrics-server-847bdf55cb-xdd6s" Jul 05 13:20:07.301150 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.301114 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/161555b1-1894-4a2c-9770-faac3772cb12-client-ca-bundle\") pod \"metrics-server-847bdf55cb-xdd6s\" (UID: \"161555b1-1894-4a2c-9770-faac3772cb12\") " pod="openshift-monitoring/metrics-server-847bdf55cb-xdd6s" Jul 05 13:20:07.301379 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.301157 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/161555b1-1894-4a2c-9770-faac3772cb12-secret-metrics-server-client-certs\") pod \"metrics-server-847bdf55cb-xdd6s\" (UID: \"161555b1-1894-4a2c-9770-faac3772cb12\") " pod="openshift-monitoring/metrics-server-847bdf55cb-xdd6s" Jul 05 13:20:07.301379 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.301183 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/161555b1-1894-4a2c-9770-faac3772cb12-audit-log\") pod \"metrics-server-847bdf55cb-xdd6s\" (UID: \"161555b1-1894-4a2c-9770-faac3772cb12\") " pod="openshift-monitoring/metrics-server-847bdf55cb-xdd6s" Jul 05 13:20:07.301379 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.301219 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/161555b1-1894-4a2c-9770-faac3772cb12-secret-metrics-server-tls\") pod \"metrics-server-847bdf55cb-xdd6s\" (UID: \"161555b1-1894-4a2c-9770-faac3772cb12\") " pod="openshift-monitoring/metrics-server-847bdf55cb-xdd6s" Jul 05 13:20:07.301379 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.301244 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/161555b1-1894-4a2c-9770-faac3772cb12-metrics-server-audit-profiles\") pod \"metrics-server-847bdf55cb-xdd6s\" (UID: \"161555b1-1894-4a2c-9770-faac3772cb12\") " pod="openshift-monitoring/metrics-server-847bdf55cb-xdd6s" Jul 05 13:20:07.301379 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.301259 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqvr7\" (UniqueName: \"kubernetes.io/projected/161555b1-1894-4a2c-9770-faac3772cb12-kube-api-access-wqvr7\") pod \"metrics-server-847bdf55cb-xdd6s\" (UID: \"161555b1-1894-4a2c-9770-faac3772cb12\") " pod="openshift-monitoring/metrics-server-847bdf55cb-xdd6s" Jul 05 13:20:07.301749 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.301695 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/161555b1-1894-4a2c-9770-faac3772cb12-audit-log\") pod \"metrics-server-847bdf55cb-xdd6s\" (UID: \"161555b1-1894-4a2c-9770-faac3772cb12\") " pod="openshift-monitoring/metrics-server-847bdf55cb-xdd6s" Jul 05 13:20:07.301886 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.301851 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/161555b1-1894-4a2c-9770-faac3772cb12-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-847bdf55cb-xdd6s\" (UID: \"161555b1-1894-4a2c-9770-faac3772cb12\") " pod="openshift-monitoring/metrics-server-847bdf55cb-xdd6s" Jul 05 13:20:07.302308 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.302288 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/161555b1-1894-4a2c-9770-faac3772cb12-metrics-server-audit-profiles\") pod \"metrics-server-847bdf55cb-xdd6s\" (UID: \"161555b1-1894-4a2c-9770-faac3772cb12\") " pod="openshift-monitoring/metrics-server-847bdf55cb-xdd6s" Jul 05 13:20:07.303987 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.303942 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/161555b1-1894-4a2c-9770-faac3772cb12-secret-metrics-server-tls\") pod \"metrics-server-847bdf55cb-xdd6s\" (UID: \"161555b1-1894-4a2c-9770-faac3772cb12\") " pod="openshift-monitoring/metrics-server-847bdf55cb-xdd6s" Jul 05 13:20:07.304098 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.304078 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/161555b1-1894-4a2c-9770-faac3772cb12-secret-metrics-server-client-certs\") pod \"metrics-server-847bdf55cb-xdd6s\" (UID: \"161555b1-1894-4a2c-9770-faac3772cb12\") " pod="openshift-monitoring/metrics-server-847bdf55cb-xdd6s" Jul 05 13:20:07.304151 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.304133 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/161555b1-1894-4a2c-9770-faac3772cb12-client-ca-bundle\") pod \"metrics-server-847bdf55cb-xdd6s\" (UID: \"161555b1-1894-4a2c-9770-faac3772cb12\") " pod="openshift-monitoring/metrics-server-847bdf55cb-xdd6s" Jul 05 13:20:07.311861 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.311842 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-b5df8" Jul 05 13:20:07.316255 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.316232 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqvr7\" (UniqueName: \"kubernetes.io/projected/161555b1-1894-4a2c-9770-faac3772cb12-kube-api-access-wqvr7\") pod \"metrics-server-847bdf55cb-xdd6s\" (UID: \"161555b1-1894-4a2c-9770-faac3772cb12\") " pod="openshift-monitoring/metrics-server-847bdf55cb-xdd6s" Jul 05 13:20:07.351820 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.351798 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c6cbc47f-j45zf" Jul 05 13:20:07.391794 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.391769 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-847bdf55cb-xdd6s" Jul 05 13:20:07.834180 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.834125 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:20:07.916100 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:07.912380 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:20:09.017826 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:09.017789 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:20:09.582032 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:09.581900 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-hvhgg"] Jul 05 13:20:09.633380 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:09.633354 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-b5df8"] Jul 05 13:20:09.690802 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:20:09.690768 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5711bc78_2a69_45f9_921f_0e00b7d51e39.slice/crio-8755353aa24e811bad6f4b270b1001b5af66b2b508f5a3bbca59b54b58a8de00 WatchSource:0}: Error finding container 8755353aa24e811bad6f4b270b1001b5af66b2b508f5a3bbca59b54b58a8de00: Status 404 returned error can't find the container with id 8755353aa24e811bad6f4b270b1001b5af66b2b508f5a3bbca59b54b58a8de00 Jul 05 13:20:09.692330 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:20:09.692299 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46d56b08_161c_465d_9adc_dbd9273e69c7.slice/crio-d3920ea0664f19efd0e688d3f332472a2708c68df21507a924b316f925e31e38 WatchSource:0}: Error finding container d3920ea0664f19efd0e688d3f332472a2708c68df21507a924b316f925e31e38: Status 404 returned error can't find the container with id d3920ea0664f19efd0e688d3f332472a2708c68df21507a924b316f925e31e38 Jul 05 13:20:09.839072 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:09.839043 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c6cbc47f-j45zf"] Jul 05 13:20:09.841603 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:09.841585 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-847bdf55cb-xdd6s"] Jul 05 13:20:09.842153 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:20:09.842130 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20ee6668_7a04_483a_8bdc_bc212e16e892.slice/crio-7c1f2ab7f6a7b48d5c589ba2e448a89d5680a9cc4df2d4680037e087d74f8f89 WatchSource:0}: Error finding container 7c1f2ab7f6a7b48d5c589ba2e448a89d5680a9cc4df2d4680037e087d74f8f89: Status 404 returned error can't find the container with id 7c1f2ab7f6a7b48d5c589ba2e448a89d5680a9cc4df2d4680037e087d74f8f89 Jul 05 13:20:09.843322 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:20:09.843253 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod161555b1_1894_4a2c_9770_faac3772cb12.slice/crio-92d6664437314a40091e230b3c9ba7f41ef2a2b3fcc68aef653a733e0d604c37 WatchSource:0}: Error finding container 92d6664437314a40091e230b3c9ba7f41ef2a2b3fcc68aef653a733e0d604c37: Status 404 returned error can't find the container with id 92d6664437314a40091e230b3c9ba7f41ef2a2b3fcc68aef653a733e0d604c37 Jul 05 13:20:10.227598 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:10.227443 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hvhgg" event={"ID":"5711bc78-2a69-45f9-921f-0e00b7d51e39","Type":"ContainerStarted","Data":"3240f0546eaed0ca1b54858862e8ea5060cfd19b7211ca9b469c5086aa36dd08"} Jul 05 13:20:10.227598 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:10.227516 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hvhgg" event={"ID":"5711bc78-2a69-45f9-921f-0e00b7d51e39","Type":"ContainerStarted","Data":"56ef41e2fb8502a23da100bbcb07dd31f93c878b2060bab9b81f00e9d5f5c0e0"} Jul 05 13:20:10.227598 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:10.227546 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hvhgg" event={"ID":"5711bc78-2a69-45f9-921f-0e00b7d51e39","Type":"ContainerStarted","Data":"8755353aa24e811bad6f4b270b1001b5af66b2b508f5a3bbca59b54b58a8de00"} Jul 05 13:20:10.233775 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:10.233023 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-l6cc6" event={"ID":"33610912-09f3-406f-be0f-a6ffbff4c9ba","Type":"ContainerStarted","Data":"e5c67a19b71ad2d4a6d91b1e6c47d3c6e941d8d0111c6202591300ccd60c63be"} Jul 05 13:20:10.239600 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:10.239083 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-784b9f44d6-4hcbs" event={"ID":"fa6433c9-e321-45f0-b5eb-11d0ac362a6a","Type":"ContainerStarted","Data":"b516a31793829b8b4c2b0de87d7c02b1632e50ccc9fcc1793b9f3ef4e18b88cb"} Jul 05 13:20:10.241834 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:10.241747 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-847bdf55cb-xdd6s" event={"ID":"161555b1-1894-4a2c-9770-faac3772cb12","Type":"ContainerStarted","Data":"92d6664437314a40091e230b3c9ba7f41ef2a2b3fcc68aef653a733e0d604c37"} Jul 05 13:20:10.245635 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:10.245192 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7858944fc4-vgmdb" event={"ID":"02bb34ab-5247-48c9-85e0-d4eb6b4fd00d","Type":"ContainerStarted","Data":"556d1d27387313ef4f051db0b989730529cca4d10b7d268bc89db4228e7e9ee3"} Jul 05 13:20:10.250653 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:10.250086 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c6cbc47f-j45zf" event={"ID":"20ee6668-7a04-483a-8bdc-bc212e16e892","Type":"ContainerStarted","Data":"bf23d0e98f0e5295034d9a5254c2b0776ecd316d765013285a27df7e0ce8f597"} Jul 05 13:20:10.250653 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:10.250137 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c6cbc47f-j45zf" event={"ID":"20ee6668-7a04-483a-8bdc-bc212e16e892","Type":"ContainerStarted","Data":"7c1f2ab7f6a7b48d5c589ba2e448a89d5680a9cc4df2d4680037e087d74f8f89"} Jul 05 13:20:10.256180 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:10.256110 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6zff6" event={"ID":"ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1","Type":"ContainerStarted","Data":"27dda4bd93abf3f0bfbb75ea86a74e61b60c33757d1ae8787e1b5757dd9a3dab"} Jul 05 13:20:10.261226 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:10.260862 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2nr4r" event={"ID":"09e0d252-5d66-4697-b469-24d2cdf32679","Type":"ContainerStarted","Data":"2c4230aebe78d0b8032a33aaf93d69f0b18089e21008e12058b3181afb7f62f0"} Jul 05 13:20:10.261698 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:10.261440 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-2nr4r" Jul 05 13:20:10.268638 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:10.266243 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-khbr5" event={"ID":"9f72532b-b2f9-42a8-90ae-f0d74cfe7a04","Type":"ContainerStarted","Data":"710dd0ccac67e286efb44a638caefbaf6f7d8bb01febb8e64ea74f358d8d196b"} Jul 05 13:20:10.268638 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:10.267003 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-khbr5" Jul 05 13:20:10.270365 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:10.270336 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-b5df8" event={"ID":"46d56b08-161c-465d-9adc-dbd9273e69c7","Type":"ContainerStarted","Data":"d3920ea0664f19efd0e688d3f332472a2708c68df21507a924b316f925e31e38"} Jul 05 13:20:10.277506 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:10.277453 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-khbr5" Jul 05 13:20:10.309437 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:10.308435 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-l6cc6" podStartSLOduration=68.676504169 podStartE2EDuration="1m23.308412908s" totalImagesPullingTime="14.631908739s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:18:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:20:10.26555255 +0000 UTC m=+83.161210639" watchObservedRunningTime="2026-07-05 13:20:10.308412908 +0000 UTC m=+83.204070977" Jul 05 13:20:10.310813 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:10.310201 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-784b9f44d6-4hcbs" podStartSLOduration=3.030709778 podStartE2EDuration="15.310188709s" totalImagesPullingTime="12.279478931s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:19:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:20:10.309804114 +0000 UTC m=+83.205462204" watchObservedRunningTime="2026-07-05 13:20:10.310188709 +0000 UTC m=+83.205846799" Jul 05 13:20:10.350522 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:10.350457 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7858944fc4-vgmdb" podStartSLOduration=2.600936909 podStartE2EDuration="17.350436481s" totalImagesPullingTime="14.749499572s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:19:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:20:10.34815215 +0000 UTC m=+83.243810239" watchObservedRunningTime="2026-07-05 13:20:10.350436481 +0000 UTC m=+83.246094571" Jul 05 13:20:10.356266 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:10.356229 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:20:10.438968 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:10.438040 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7c6cbc47f-j45zf" podStartSLOduration=4.438026512 podStartE2EDuration="4.438026512s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:20:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:20:10.432820499 +0000 UTC m=+83.328478592" watchObservedRunningTime="2026-07-05 13:20:10.438026512 +0000 UTC m=+83.333684602" Jul 05 13:20:10.438968 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:10.438645 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-2nr4r" podStartSLOduration=68.65665257 podStartE2EDuration="1m23.438636558s" totalImagesPullingTime="14.781983988s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:18:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:20:10.382929059 +0000 UTC m=+83.278587142" watchObservedRunningTime="2026-07-05 13:20:10.438636558 +0000 UTC m=+83.334294651" Jul 05 13:20:10.475053 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:10.474989 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-khbr5" podStartSLOduration=2.111953316 podStartE2EDuration="21.474965848s" totalImagesPullingTime="19.363012532s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:19:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:20:10.473374774 +0000 UTC m=+83.369032875" watchObservedRunningTime="2026-07-05 13:20:10.474965848 +0000 UTC m=+83.370623929" Jul 05 13:20:10.481386 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:10.481266 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:20:13.134307 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:13.134278 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-db95784f7-lfv57" Jul 05 13:20:13.286553 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:13.286522 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-b5df8" event={"ID":"46d56b08-161c-465d-9adc-dbd9273e69c7","Type":"ContainerStarted","Data":"a0331e8203f034936451462576243b8c12754d3fbd75e1ccb140d87cdee8916e"} Jul 05 13:20:13.287093 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:13.287070 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-b5df8" Jul 05 13:20:13.289509 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:13.289444 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-847bdf55cb-xdd6s" event={"ID":"161555b1-1894-4a2c-9770-faac3772cb12","Type":"ContainerStarted","Data":"0002a02c46f333e640533d89b002166a52bb030b9be1d966d3b131a1d404d6c2"} Jul 05 13:20:13.291687 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:13.291477 2573 generic.go:356] "Generic (PLEG): container finished" podID="ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1" containerID="4ef7f90dfa21a32f155704d2e56b01473a27375ecfa3a711eae088413dbd2633" exitCode=0 Jul 05 13:20:13.291687 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:13.291520 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6zff6" event={"ID":"ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1","Type":"ContainerDied","Data":"4ef7f90dfa21a32f155704d2e56b01473a27375ecfa3a711eae088413dbd2633"} Jul 05 13:20:13.292947 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:13.292924 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-b5df8" Jul 05 13:20:13.363244 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:13.363163 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:20:13.363413 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:13.363361 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-b5df8" podStartSLOduration=4.162661713 podStartE2EDuration="7.363341611s" totalImagesPullingTime="3.200679898s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:20:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:20:13.362472674 +0000 UTC m=+86.258130763" watchObservedRunningTime="2026-07-05 13:20:13.363341611 +0000 UTC m=+86.258999700" Jul 05 13:20:13.425192 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:13.425133 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-847bdf55cb-xdd6s" podStartSLOduration=3.364727465 podStartE2EDuration="6.425033185s" totalImagesPullingTime="3.06030572s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:20:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:20:13.42111128 +0000 UTC m=+86.316769370" watchObservedRunningTime="2026-07-05 13:20:13.425033185 +0000 UTC m=+86.320691275" Jul 05 13:20:13.638331 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:13.638269 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:20:13.843888 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:13.843782 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:20:13.949969 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:13.949908 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:20:14.073512 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:14.073473 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:20:14.298746 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:14.298704 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6zff6" event={"ID":"ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1","Type":"ContainerStarted","Data":"798badf113ba029ed03dd723096e9033bbe02a7e4f747bbfdb8b098697ad88b6"} Jul 05 13:20:14.299347 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:14.298762 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6zff6" event={"ID":"ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1","Type":"ContainerStarted","Data":"d076765035e8765f65ce564d9f4b45a9dd6c90fb10bfe8c892865e5ccde9c251"} Jul 05 13:20:14.301220 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:14.301194 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hvhgg" event={"ID":"5711bc78-2a69-45f9-921f-0e00b7d51e39","Type":"ContainerStarted","Data":"2d097f8ec4ffed3f818bb9155a93cf01e067b7b5b89cce6404d2ca82b8481b06"} Jul 05 13:20:14.335913 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:14.335852 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-6zff6" podStartSLOduration=8.701965414 podStartE2EDuration="12.335832889s" totalImagesPullingTime="3.525687113s" totalInitContainerRuntime="108.180362ms" isStatefulPod=true podCreationTimestamp="2026-07-05 13:20:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:20:14.331743212 +0000 UTC m=+87.227401314" watchObservedRunningTime="2026-07-05 13:20:14.335832889 +0000 UTC m=+87.231490979" Jul 05 13:20:14.380484 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:14.380451 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:20:14.400190 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:14.400136 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hvhgg" podStartSLOduration=9.022935982 podStartE2EDuration="12.400115951s" totalImagesPullingTime="3.377179969s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:20:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:20:14.398817766 +0000 UTC m=+87.294475881" watchObservedRunningTime="2026-07-05 13:20:14.400115951 +0000 UTC m=+87.295774042" Jul 05 13:20:14.416034 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:14.415991 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7858944fc4-vgmdb" Jul 05 13:20:14.464617 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:14.464582 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:20:14.959560 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:14.959513 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-78fc4b5dfd-vrwbb" podUID="80ca46e3-b000-4760-89ad-2ab808957f27" containerName="registry" containerID="cri-o://833edfcb7e977db3b135b35ed8ea05c6c0d45a1f92967199f198e8ece435ce9c" gracePeriod=30 Jul 05 13:20:15.252880 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:15.252849 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78fc4b5dfd-vrwbb" Jul 05 13:20:15.306937 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:15.306901 2573 generic.go:356] "Generic (PLEG): container finished" podID="80ca46e3-b000-4760-89ad-2ab808957f27" containerID="833edfcb7e977db3b135b35ed8ea05c6c0d45a1f92967199f198e8ece435ce9c" exitCode=0 Jul 05 13:20:15.307426 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:15.306980 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78fc4b5dfd-vrwbb" Jul 05 13:20:15.307426 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:15.307000 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78fc4b5dfd-vrwbb" event={"ID":"80ca46e3-b000-4760-89ad-2ab808957f27","Type":"ContainerDied","Data":"833edfcb7e977db3b135b35ed8ea05c6c0d45a1f92967199f198e8ece435ce9c"} Jul 05 13:20:15.307426 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:15.307064 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78fc4b5dfd-vrwbb" event={"ID":"80ca46e3-b000-4760-89ad-2ab808957f27","Type":"ContainerDied","Data":"60df47041f2cb69e19385f3f7a162146623f5557fcf036556105f13d79c3d5b0"} Jul 05 13:20:15.307426 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:15.307108 2573 scope.go:117] "RemoveContainer" containerID="833edfcb7e977db3b135b35ed8ea05c6c0d45a1f92967199f198e8ece435ce9c" Jul 05 13:20:15.318458 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:15.318442 2573 scope.go:117] "RemoveContainer" containerID="833edfcb7e977db3b135b35ed8ea05c6c0d45a1f92967199f198e8ece435ce9c" Jul 05 13:20:15.318776 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:20:15.318744 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"833edfcb7e977db3b135b35ed8ea05c6c0d45a1f92967199f198e8ece435ce9c\": container with ID starting with 833edfcb7e977db3b135b35ed8ea05c6c0d45a1f92967199f198e8ece435ce9c not found: ID does not exist" containerID="833edfcb7e977db3b135b35ed8ea05c6c0d45a1f92967199f198e8ece435ce9c" Jul 05 13:20:15.318893 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:15.318784 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"833edfcb7e977db3b135b35ed8ea05c6c0d45a1f92967199f198e8ece435ce9c"} err="failed to get container status \"833edfcb7e977db3b135b35ed8ea05c6c0d45a1f92967199f198e8ece435ce9c\": rpc error: code = NotFound desc = could not find container \"833edfcb7e977db3b135b35ed8ea05c6c0d45a1f92967199f198e8ece435ce9c\": container with ID starting with 833edfcb7e977db3b135b35ed8ea05c6c0d45a1f92967199f198e8ece435ce9c not found: ID does not exist" Jul 05 13:20:15.386198 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:15.386168 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/80ca46e3-b000-4760-89ad-2ab808957f27-ca-trust-extracted\") pod \"80ca46e3-b000-4760-89ad-2ab808957f27\" (UID: \"80ca46e3-b000-4760-89ad-2ab808957f27\") " Jul 05 13:20:15.386198 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:15.386201 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/80ca46e3-b000-4760-89ad-2ab808957f27-installation-pull-secrets\") pod \"80ca46e3-b000-4760-89ad-2ab808957f27\" (UID: \"80ca46e3-b000-4760-89ad-2ab808957f27\") " Jul 05 13:20:15.386391 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:15.386260 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wkbqc\" (UniqueName: \"kubernetes.io/projected/80ca46e3-b000-4760-89ad-2ab808957f27-kube-api-access-wkbqc\") pod \"80ca46e3-b000-4760-89ad-2ab808957f27\" (UID: \"80ca46e3-b000-4760-89ad-2ab808957f27\") " Jul 05 13:20:15.386391 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:15.386295 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/80ca46e3-b000-4760-89ad-2ab808957f27-registry-tls\") pod \"80ca46e3-b000-4760-89ad-2ab808957f27\" (UID: \"80ca46e3-b000-4760-89ad-2ab808957f27\") " Jul 05 13:20:15.386391 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:15.386330 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/80ca46e3-b000-4760-89ad-2ab808957f27-image-registry-private-configuration\") pod \"80ca46e3-b000-4760-89ad-2ab808957f27\" (UID: \"80ca46e3-b000-4760-89ad-2ab808957f27\") " Jul 05 13:20:15.386391 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:15.386374 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/80ca46e3-b000-4760-89ad-2ab808957f27-trusted-ca\") pod \"80ca46e3-b000-4760-89ad-2ab808957f27\" (UID: \"80ca46e3-b000-4760-89ad-2ab808957f27\") " Jul 05 13:20:15.386596 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:15.386407 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/80ca46e3-b000-4760-89ad-2ab808957f27-bound-sa-token\") pod \"80ca46e3-b000-4760-89ad-2ab808957f27\" (UID: \"80ca46e3-b000-4760-89ad-2ab808957f27\") " Jul 05 13:20:15.386596 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:15.386444 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/80ca46e3-b000-4760-89ad-2ab808957f27-registry-certificates\") pod \"80ca46e3-b000-4760-89ad-2ab808957f27\" (UID: \"80ca46e3-b000-4760-89ad-2ab808957f27\") " Jul 05 13:20:15.388363 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:15.387456 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80ca46e3-b000-4760-89ad-2ab808957f27-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "80ca46e3-b000-4760-89ad-2ab808957f27" (UID: "80ca46e3-b000-4760-89ad-2ab808957f27"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:20:15.388363 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:15.388141 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80ca46e3-b000-4760-89ad-2ab808957f27-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "80ca46e3-b000-4760-89ad-2ab808957f27" (UID: "80ca46e3-b000-4760-89ad-2ab808957f27"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:20:15.389549 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:15.389375 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80ca46e3-b000-4760-89ad-2ab808957f27-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "80ca46e3-b000-4760-89ad-2ab808957f27" (UID: "80ca46e3-b000-4760-89ad-2ab808957f27"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:20:15.390150 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:15.390126 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80ca46e3-b000-4760-89ad-2ab808957f27-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "80ca46e3-b000-4760-89ad-2ab808957f27" (UID: "80ca46e3-b000-4760-89ad-2ab808957f27"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:20:15.391032 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:15.390990 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80ca46e3-b000-4760-89ad-2ab808957f27-kube-api-access-wkbqc" (OuterVolumeSpecName: "kube-api-access-wkbqc") pod "80ca46e3-b000-4760-89ad-2ab808957f27" (UID: "80ca46e3-b000-4760-89ad-2ab808957f27"). InnerVolumeSpecName "kube-api-access-wkbqc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:20:15.391513 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:15.391403 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80ca46e3-b000-4760-89ad-2ab808957f27-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "80ca46e3-b000-4760-89ad-2ab808957f27" (UID: "80ca46e3-b000-4760-89ad-2ab808957f27"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:20:15.391865 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:15.391838 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80ca46e3-b000-4760-89ad-2ab808957f27-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "80ca46e3-b000-4760-89ad-2ab808957f27" (UID: "80ca46e3-b000-4760-89ad-2ab808957f27"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:20:15.398305 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:15.398280 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80ca46e3-b000-4760-89ad-2ab808957f27-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "80ca46e3-b000-4760-89ad-2ab808957f27" (UID: "80ca46e3-b000-4760-89ad-2ab808957f27"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:20:15.487507 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:15.487423 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wkbqc\" (UniqueName: \"kubernetes.io/projected/80ca46e3-b000-4760-89ad-2ab808957f27-kube-api-access-wkbqc\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:20:15.487507 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:15.487465 2573 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/80ca46e3-b000-4760-89ad-2ab808957f27-registry-tls\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:20:15.487507 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:15.487482 2573 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/80ca46e3-b000-4760-89ad-2ab808957f27-image-registry-private-configuration\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:20:15.487507 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:15.487494 2573 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/80ca46e3-b000-4760-89ad-2ab808957f27-trusted-ca\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:20:15.487507 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:15.487503 2573 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/80ca46e3-b000-4760-89ad-2ab808957f27-bound-sa-token\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:20:15.487507 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:15.487511 2573 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/80ca46e3-b000-4760-89ad-2ab808957f27-registry-certificates\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:20:15.487846 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:15.487521 2573 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/80ca46e3-b000-4760-89ad-2ab808957f27-ca-trust-extracted\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:20:15.487846 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:15.487530 2573 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/80ca46e3-b000-4760-89ad-2ab808957f27-installation-pull-secrets\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:20:15.638459 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:15.638424 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-78fc4b5dfd-vrwbb"] Jul 05 13:20:15.649335 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:15.649304 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-78fc4b5dfd-vrwbb"] Jul 05 13:20:15.741831 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:15.741749 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80ca46e3-b000-4760-89ad-2ab808957f27" path="/var/lib/kubelet/pods/80ca46e3-b000-4760-89ad-2ab808957f27/volumes" Jul 05 13:20:16.168416 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:16.168332 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-784b9f44d6-4hcbs" Jul 05 13:20:16.168416 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:16.168411 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-784b9f44d6-4hcbs" Jul 05 13:20:16.174040 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:16.174012 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-784b9f44d6-4hcbs" Jul 05 13:20:16.317430 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:16.317395 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-784b9f44d6-4hcbs" Jul 05 13:20:17.352610 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:17.352577 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7c6cbc47f-j45zf" Jul 05 13:20:17.353106 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:17.352674 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7c6cbc47f-j45zf" Jul 05 13:20:17.358339 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:17.358314 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7c6cbc47f-j45zf" Jul 05 13:20:17.588608 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:17.588574 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:20:17.687714 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:17.687682 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:20:18.327471 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:18.327438 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7c6cbc47f-j45zf" Jul 05 13:20:18.402796 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:18.402739 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-784b9f44d6-4hcbs"] Jul 05 13:20:18.777494 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:18.777440 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:20:18.880811 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:18.880774 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:20:19.209368 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:19.209334 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:20:19.310197 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:19.310160 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:20:19.975516 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:19.975480 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:20:20.077663 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:20.077629 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:20:21.443313 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:21.443281 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:20:21.518092 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:21.518060 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:20:23.024355 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:23.024320 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:20:23.124580 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:23.124548 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:20:23.479510 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:23.479477 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:20:23.577918 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:23.577884 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:20:27.391937 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:27.391904 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-847bdf55cb-xdd6s" Jul 05 13:20:27.392535 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:27.392008 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-847bdf55cb-xdd6s" Jul 05 13:20:35.277880 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:35.277814 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-7858944fc4-vgmdb" podUID="02bb34ab-5247-48c9-85e0-d4eb6b4fd00d" containerName="console" containerID="cri-o://556d1d27387313ef4f051db0b989730529cca4d10b7d268bc89db4228e7e9ee3" gracePeriod=15 Jul 05 13:20:35.553508 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:35.553439 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7858944fc4-vgmdb" Jul 05 13:20:35.663896 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:35.663857 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/02bb34ab-5247-48c9-85e0-d4eb6b4fd00d-console-oauth-config\") pod \"02bb34ab-5247-48c9-85e0-d4eb6b4fd00d\" (UID: \"02bb34ab-5247-48c9-85e0-d4eb6b4fd00d\") " Jul 05 13:20:35.663896 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:35.663904 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/02bb34ab-5247-48c9-85e0-d4eb6b4fd00d-oauth-serving-cert\") pod \"02bb34ab-5247-48c9-85e0-d4eb6b4fd00d\" (UID: \"02bb34ab-5247-48c9-85e0-d4eb6b4fd00d\") " Jul 05 13:20:35.664170 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:35.663922 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/02bb34ab-5247-48c9-85e0-d4eb6b4fd00d-console-serving-cert\") pod \"02bb34ab-5247-48c9-85e0-d4eb6b4fd00d\" (UID: \"02bb34ab-5247-48c9-85e0-d4eb6b4fd00d\") " Jul 05 13:20:35.664170 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:35.663971 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/02bb34ab-5247-48c9-85e0-d4eb6b4fd00d-service-ca\") pod \"02bb34ab-5247-48c9-85e0-d4eb6b4fd00d\" (UID: \"02bb34ab-5247-48c9-85e0-d4eb6b4fd00d\") " Jul 05 13:20:35.664170 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:35.664100 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-79v4q\" (UniqueName: \"kubernetes.io/projected/02bb34ab-5247-48c9-85e0-d4eb6b4fd00d-kube-api-access-79v4q\") pod \"02bb34ab-5247-48c9-85e0-d4eb6b4fd00d\" (UID: \"02bb34ab-5247-48c9-85e0-d4eb6b4fd00d\") " Jul 05 13:20:35.664316 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:35.664211 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/02bb34ab-5247-48c9-85e0-d4eb6b4fd00d-console-config\") pod \"02bb34ab-5247-48c9-85e0-d4eb6b4fd00d\" (UID: \"02bb34ab-5247-48c9-85e0-d4eb6b4fd00d\") " Jul 05 13:20:35.664397 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:35.664369 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02bb34ab-5247-48c9-85e0-d4eb6b4fd00d-service-ca" (OuterVolumeSpecName: "service-ca") pod "02bb34ab-5247-48c9-85e0-d4eb6b4fd00d" (UID: "02bb34ab-5247-48c9-85e0-d4eb6b4fd00d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:20:35.664465 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:35.664419 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02bb34ab-5247-48c9-85e0-d4eb6b4fd00d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "02bb34ab-5247-48c9-85e0-d4eb6b4fd00d" (UID: "02bb34ab-5247-48c9-85e0-d4eb6b4fd00d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:20:35.664601 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:35.664582 2573 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/02bb34ab-5247-48c9-85e0-d4eb6b4fd00d-service-ca\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:20:35.664675 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:35.664653 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02bb34ab-5247-48c9-85e0-d4eb6b4fd00d-console-config" (OuterVolumeSpecName: "console-config") pod "02bb34ab-5247-48c9-85e0-d4eb6b4fd00d" (UID: "02bb34ab-5247-48c9-85e0-d4eb6b4fd00d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:20:35.666409 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:35.666381 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/02bb34ab-5247-48c9-85e0-d4eb6b4fd00d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "02bb34ab-5247-48c9-85e0-d4eb6b4fd00d" (UID: "02bb34ab-5247-48c9-85e0-d4eb6b4fd00d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:20:35.666409 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:35.666386 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02bb34ab-5247-48c9-85e0-d4eb6b4fd00d-kube-api-access-79v4q" (OuterVolumeSpecName: "kube-api-access-79v4q") pod "02bb34ab-5247-48c9-85e0-d4eb6b4fd00d" (UID: "02bb34ab-5247-48c9-85e0-d4eb6b4fd00d"). InnerVolumeSpecName "kube-api-access-79v4q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:20:35.666591 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:35.666452 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/02bb34ab-5247-48c9-85e0-d4eb6b4fd00d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "02bb34ab-5247-48c9-85e0-d4eb6b4fd00d" (UID: "02bb34ab-5247-48c9-85e0-d4eb6b4fd00d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:20:35.764948 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:35.764924 2573 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/02bb34ab-5247-48c9-85e0-d4eb6b4fd00d-console-config\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:20:35.765072 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:35.764969 2573 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/02bb34ab-5247-48c9-85e0-d4eb6b4fd00d-console-oauth-config\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:20:35.765072 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:35.764985 2573 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/02bb34ab-5247-48c9-85e0-d4eb6b4fd00d-oauth-serving-cert\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:20:35.765072 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:35.764998 2573 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/02bb34ab-5247-48c9-85e0-d4eb6b4fd00d-console-serving-cert\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:20:35.765072 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:35.765012 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-79v4q\" (UniqueName: \"kubernetes.io/projected/02bb34ab-5247-48c9-85e0-d4eb6b4fd00d-kube-api-access-79v4q\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:20:36.391293 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:36.391256 2573 generic.go:356] "Generic (PLEG): container finished" podID="02bb34ab-5247-48c9-85e0-d4eb6b4fd00d" containerID="556d1d27387313ef4f051db0b989730529cca4d10b7d268bc89db4228e7e9ee3" exitCode=0 Jul 05 13:20:36.391681 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:36.391299 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7858944fc4-vgmdb" Jul 05 13:20:36.391681 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:36.391333 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7858944fc4-vgmdb" event={"ID":"02bb34ab-5247-48c9-85e0-d4eb6b4fd00d","Type":"ContainerDied","Data":"556d1d27387313ef4f051db0b989730529cca4d10b7d268bc89db4228e7e9ee3"} Jul 05 13:20:36.391681 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:36.391373 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7858944fc4-vgmdb" event={"ID":"02bb34ab-5247-48c9-85e0-d4eb6b4fd00d","Type":"ContainerDied","Data":"1ba59af80117d2eaeab2d4ec0a05fdb784f44bafd4e7eea57b9b895172680a66"} Jul 05 13:20:36.391681 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:36.391393 2573 scope.go:117] "RemoveContainer" containerID="556d1d27387313ef4f051db0b989730529cca4d10b7d268bc89db4228e7e9ee3" Jul 05 13:20:36.399520 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:36.399505 2573 scope.go:117] "RemoveContainer" containerID="556d1d27387313ef4f051db0b989730529cca4d10b7d268bc89db4228e7e9ee3" Jul 05 13:20:36.399742 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:20:36.399726 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"556d1d27387313ef4f051db0b989730529cca4d10b7d268bc89db4228e7e9ee3\": container with ID starting with 556d1d27387313ef4f051db0b989730529cca4d10b7d268bc89db4228e7e9ee3 not found: ID does not exist" containerID="556d1d27387313ef4f051db0b989730529cca4d10b7d268bc89db4228e7e9ee3" Jul 05 13:20:36.399790 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:36.399749 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"556d1d27387313ef4f051db0b989730529cca4d10b7d268bc89db4228e7e9ee3"} err="failed to get container status \"556d1d27387313ef4f051db0b989730529cca4d10b7d268bc89db4228e7e9ee3\": rpc error: code = NotFound desc = could not find container \"556d1d27387313ef4f051db0b989730529cca4d10b7d268bc89db4228e7e9ee3\": container with ID starting with 556d1d27387313ef4f051db0b989730529cca4d10b7d268bc89db4228e7e9ee3 not found: ID does not exist" Jul 05 13:20:36.407501 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:36.407481 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7858944fc4-vgmdb"] Jul 05 13:20:36.412059 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:36.412041 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7858944fc4-vgmdb"] Jul 05 13:20:36.447370 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:36.447338 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:20:36.501755 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:36.501733 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:20:36.937545 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:36.937511 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:20:37.001738 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:37.001698 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:20:37.734680 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:37.734648 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02bb34ab-5247-48c9-85e0-d4eb6b4fd00d" path="/var/lib/kubelet/pods/02bb34ab-5247-48c9-85e0-d4eb6b4fd00d/volumes" Jul 05 13:20:42.281300 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:42.281271 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-2nr4r" Jul 05 13:20:43.429678 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:43.429619 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-784b9f44d6-4hcbs" podUID="fa6433c9-e321-45f0-b5eb-11d0ac362a6a" containerName="console" containerID="cri-o://b516a31793829b8b4c2b0de87d7c02b1632e50ccc9fcc1793b9f3ef4e18b88cb" gracePeriod=15 Jul 05 13:20:43.719012 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:43.718987 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-784b9f44d6-4hcbs" Jul 05 13:20:43.827947 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:43.827911 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa6433c9-e321-45f0-b5eb-11d0ac362a6a-trusted-ca-bundle\") pod \"fa6433c9-e321-45f0-b5eb-11d0ac362a6a\" (UID: \"fa6433c9-e321-45f0-b5eb-11d0ac362a6a\") " Jul 05 13:20:43.828136 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:43.827977 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fa6433c9-e321-45f0-b5eb-11d0ac362a6a-console-config\") pod \"fa6433c9-e321-45f0-b5eb-11d0ac362a6a\" (UID: \"fa6433c9-e321-45f0-b5eb-11d0ac362a6a\") " Jul 05 13:20:43.828136 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:43.827999 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fa6433c9-e321-45f0-b5eb-11d0ac362a6a-console-serving-cert\") pod \"fa6433c9-e321-45f0-b5eb-11d0ac362a6a\" (UID: \"fa6433c9-e321-45f0-b5eb-11d0ac362a6a\") " Jul 05 13:20:43.828136 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:43.828036 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fa6433c9-e321-45f0-b5eb-11d0ac362a6a-console-oauth-config\") pod \"fa6433c9-e321-45f0-b5eb-11d0ac362a6a\" (UID: \"fa6433c9-e321-45f0-b5eb-11d0ac362a6a\") " Jul 05 13:20:43.828136 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:43.828084 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fa6433c9-e321-45f0-b5eb-11d0ac362a6a-service-ca\") pod \"fa6433c9-e321-45f0-b5eb-11d0ac362a6a\" (UID: \"fa6433c9-e321-45f0-b5eb-11d0ac362a6a\") " Jul 05 13:20:43.828136 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:43.828113 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fa6433c9-e321-45f0-b5eb-11d0ac362a6a-oauth-serving-cert\") pod \"fa6433c9-e321-45f0-b5eb-11d0ac362a6a\" (UID: \"fa6433c9-e321-45f0-b5eb-11d0ac362a6a\") " Jul 05 13:20:43.828383 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:43.828141 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-44rxl\" (UniqueName: \"kubernetes.io/projected/fa6433c9-e321-45f0-b5eb-11d0ac362a6a-kube-api-access-44rxl\") pod \"fa6433c9-e321-45f0-b5eb-11d0ac362a6a\" (UID: \"fa6433c9-e321-45f0-b5eb-11d0ac362a6a\") " Jul 05 13:20:43.828500 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:43.828469 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa6433c9-e321-45f0-b5eb-11d0ac362a6a-console-config" (OuterVolumeSpecName: "console-config") pod "fa6433c9-e321-45f0-b5eb-11d0ac362a6a" (UID: "fa6433c9-e321-45f0-b5eb-11d0ac362a6a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:20:43.828564 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:43.828471 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa6433c9-e321-45f0-b5eb-11d0ac362a6a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "fa6433c9-e321-45f0-b5eb-11d0ac362a6a" (UID: "fa6433c9-e321-45f0-b5eb-11d0ac362a6a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:20:43.828621 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:43.828571 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa6433c9-e321-45f0-b5eb-11d0ac362a6a-service-ca" (OuterVolumeSpecName: "service-ca") pod "fa6433c9-e321-45f0-b5eb-11d0ac362a6a" (UID: "fa6433c9-e321-45f0-b5eb-11d0ac362a6a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:20:43.828621 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:43.828579 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa6433c9-e321-45f0-b5eb-11d0ac362a6a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "fa6433c9-e321-45f0-b5eb-11d0ac362a6a" (UID: "fa6433c9-e321-45f0-b5eb-11d0ac362a6a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:20:43.830442 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:43.830410 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa6433c9-e321-45f0-b5eb-11d0ac362a6a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "fa6433c9-e321-45f0-b5eb-11d0ac362a6a" (UID: "fa6433c9-e321-45f0-b5eb-11d0ac362a6a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:20:43.830442 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:43.830428 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa6433c9-e321-45f0-b5eb-11d0ac362a6a-kube-api-access-44rxl" (OuterVolumeSpecName: "kube-api-access-44rxl") pod "fa6433c9-e321-45f0-b5eb-11d0ac362a6a" (UID: "fa6433c9-e321-45f0-b5eb-11d0ac362a6a"). InnerVolumeSpecName "kube-api-access-44rxl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:20:43.830586 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:43.830456 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa6433c9-e321-45f0-b5eb-11d0ac362a6a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "fa6433c9-e321-45f0-b5eb-11d0ac362a6a" (UID: "fa6433c9-e321-45f0-b5eb-11d0ac362a6a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:20:43.929416 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:43.929374 2573 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fa6433c9-e321-45f0-b5eb-11d0ac362a6a-console-config\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:20:43.929416 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:43.929407 2573 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fa6433c9-e321-45f0-b5eb-11d0ac362a6a-console-serving-cert\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:20:43.929416 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:43.929419 2573 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fa6433c9-e321-45f0-b5eb-11d0ac362a6a-console-oauth-config\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:20:43.929649 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:43.929432 2573 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fa6433c9-e321-45f0-b5eb-11d0ac362a6a-service-ca\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:20:43.929649 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:43.929445 2573 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fa6433c9-e321-45f0-b5eb-11d0ac362a6a-oauth-serving-cert\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:20:43.929649 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:43.929456 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-44rxl\" (UniqueName: \"kubernetes.io/projected/fa6433c9-e321-45f0-b5eb-11d0ac362a6a-kube-api-access-44rxl\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:20:43.929649 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:43.929467 2573 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa6433c9-e321-45f0-b5eb-11d0ac362a6a-trusted-ca-bundle\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:20:44.421530 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:44.421488 2573 generic.go:356] "Generic (PLEG): container finished" podID="fa6433c9-e321-45f0-b5eb-11d0ac362a6a" containerID="b516a31793829b8b4c2b0de87d7c02b1632e50ccc9fcc1793b9f3ef4e18b88cb" exitCode=0 Jul 05 13:20:44.421684 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:44.421538 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-784b9f44d6-4hcbs" Jul 05 13:20:44.421684 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:44.421545 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-784b9f44d6-4hcbs" event={"ID":"fa6433c9-e321-45f0-b5eb-11d0ac362a6a","Type":"ContainerDied","Data":"b516a31793829b8b4c2b0de87d7c02b1632e50ccc9fcc1793b9f3ef4e18b88cb"} Jul 05 13:20:44.421684 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:44.421595 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-784b9f44d6-4hcbs" event={"ID":"fa6433c9-e321-45f0-b5eb-11d0ac362a6a","Type":"ContainerDied","Data":"c2e8893df0ac9ed40aaae8c3923cffef83addee0e20e7881ed5f63cc984559fc"} Jul 05 13:20:44.421684 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:44.421623 2573 scope.go:117] "RemoveContainer" containerID="b516a31793829b8b4c2b0de87d7c02b1632e50ccc9fcc1793b9f3ef4e18b88cb" Jul 05 13:20:44.429542 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:44.429525 2573 scope.go:117] "RemoveContainer" containerID="b516a31793829b8b4c2b0de87d7c02b1632e50ccc9fcc1793b9f3ef4e18b88cb" Jul 05 13:20:44.429769 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:20:44.429750 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b516a31793829b8b4c2b0de87d7c02b1632e50ccc9fcc1793b9f3ef4e18b88cb\": container with ID starting with b516a31793829b8b4c2b0de87d7c02b1632e50ccc9fcc1793b9f3ef4e18b88cb not found: ID does not exist" containerID="b516a31793829b8b4c2b0de87d7c02b1632e50ccc9fcc1793b9f3ef4e18b88cb" Jul 05 13:20:44.430060 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:44.429782 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b516a31793829b8b4c2b0de87d7c02b1632e50ccc9fcc1793b9f3ef4e18b88cb"} err="failed to get container status \"b516a31793829b8b4c2b0de87d7c02b1632e50ccc9fcc1793b9f3ef4e18b88cb\": rpc error: code = NotFound desc = could not find container \"b516a31793829b8b4c2b0de87d7c02b1632e50ccc9fcc1793b9f3ef4e18b88cb\": container with ID starting with b516a31793829b8b4c2b0de87d7c02b1632e50ccc9fcc1793b9f3ef4e18b88cb not found: ID does not exist" Jul 05 13:20:44.442419 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:44.442397 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-784b9f44d6-4hcbs"] Jul 05 13:20:44.448553 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:44.448534 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-784b9f44d6-4hcbs"] Jul 05 13:20:45.735945 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:45.735908 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa6433c9-e321-45f0-b5eb-11d0ac362a6a" path="/var/lib/kubelet/pods/fa6433c9-e321-45f0-b5eb-11d0ac362a6a/volumes" Jul 05 13:20:47.397794 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:47.397764 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-847bdf55cb-xdd6s" Jul 05 13:20:47.402056 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:20:47.402032 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-847bdf55cb-xdd6s" Jul 05 13:21:21.820872 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:21.820834 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:21:21.923130 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:21.923087 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:21:26.917699 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:26.917615 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6d456696c8-75wb6"] Jul 05 13:21:26.918069 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:26.917890 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80ca46e3-b000-4760-89ad-2ab808957f27" containerName="registry" Jul 05 13:21:26.918069 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:26.917901 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="80ca46e3-b000-4760-89ad-2ab808957f27" containerName="registry" Jul 05 13:21:26.918069 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:26.917909 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa6433c9-e321-45f0-b5eb-11d0ac362a6a" containerName="console" Jul 05 13:21:26.918069 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:26.917914 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa6433c9-e321-45f0-b5eb-11d0ac362a6a" containerName="console" Jul 05 13:21:26.918069 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:26.917928 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02bb34ab-5247-48c9-85e0-d4eb6b4fd00d" containerName="console" Jul 05 13:21:26.918069 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:26.917934 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="02bb34ab-5247-48c9-85e0-d4eb6b4fd00d" containerName="console" Jul 05 13:21:26.918069 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:26.917997 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa6433c9-e321-45f0-b5eb-11d0ac362a6a" containerName="console" Jul 05 13:21:26.918069 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:26.918006 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="80ca46e3-b000-4760-89ad-2ab808957f27" containerName="registry" Jul 05 13:21:26.918069 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:26.918013 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="02bb34ab-5247-48c9-85e0-d4eb6b4fd00d" containerName="console" Jul 05 13:21:26.920803 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:26.920786 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6d456696c8-75wb6" Jul 05 13:21:26.923047 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:26.923025 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 05 13:21:26.923186 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:26.923070 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-kst95\"" Jul 05 13:21:26.923186 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:26.923099 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 05 13:21:26.923186 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:26.923134 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 05 13:21:26.923325 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:26.923288 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 05 13:21:26.924815 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:26.924798 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 05 13:21:26.932403 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:26.932341 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 05 13:21:26.934367 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:26.934317 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6d456696c8-75wb6"] Jul 05 13:21:26.976886 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:26.976860 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tr5t4\" (UniqueName: \"kubernetes.io/projected/55118c56-ac0a-4696-aefb-dd8bcb891cc7-kube-api-access-tr5t4\") pod \"telemeter-client-6d456696c8-75wb6\" (UID: \"55118c56-ac0a-4696-aefb-dd8bcb891cc7\") " pod="openshift-monitoring/telemeter-client-6d456696c8-75wb6" Jul 05 13:21:26.977027 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:26.976902 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/55118c56-ac0a-4696-aefb-dd8bcb891cc7-telemeter-client-tls\") pod \"telemeter-client-6d456696c8-75wb6\" (UID: \"55118c56-ac0a-4696-aefb-dd8bcb891cc7\") " pod="openshift-monitoring/telemeter-client-6d456696c8-75wb6" Jul 05 13:21:26.977027 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:26.976919 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55118c56-ac0a-4696-aefb-dd8bcb891cc7-serving-certs-ca-bundle\") pod \"telemeter-client-6d456696c8-75wb6\" (UID: \"55118c56-ac0a-4696-aefb-dd8bcb891cc7\") " pod="openshift-monitoring/telemeter-client-6d456696c8-75wb6" Jul 05 13:21:26.977027 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:26.976997 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/55118c56-ac0a-4696-aefb-dd8bcb891cc7-metrics-client-ca\") pod \"telemeter-client-6d456696c8-75wb6\" (UID: \"55118c56-ac0a-4696-aefb-dd8bcb891cc7\") " pod="openshift-monitoring/telemeter-client-6d456696c8-75wb6" Jul 05 13:21:26.977147 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:26.977053 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/55118c56-ac0a-4696-aefb-dd8bcb891cc7-secret-telemeter-client\") pod \"telemeter-client-6d456696c8-75wb6\" (UID: \"55118c56-ac0a-4696-aefb-dd8bcb891cc7\") " pod="openshift-monitoring/telemeter-client-6d456696c8-75wb6" Jul 05 13:21:26.977147 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:26.977084 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55118c56-ac0a-4696-aefb-dd8bcb891cc7-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6d456696c8-75wb6\" (UID: \"55118c56-ac0a-4696-aefb-dd8bcb891cc7\") " pod="openshift-monitoring/telemeter-client-6d456696c8-75wb6" Jul 05 13:21:26.977147 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:26.977105 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/55118c56-ac0a-4696-aefb-dd8bcb891cc7-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6d456696c8-75wb6\" (UID: \"55118c56-ac0a-4696-aefb-dd8bcb891cc7\") " pod="openshift-monitoring/telemeter-client-6d456696c8-75wb6" Jul 05 13:21:26.977147 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:26.977127 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/55118c56-ac0a-4696-aefb-dd8bcb891cc7-federate-client-tls\") pod \"telemeter-client-6d456696c8-75wb6\" (UID: \"55118c56-ac0a-4696-aefb-dd8bcb891cc7\") " pod="openshift-monitoring/telemeter-client-6d456696c8-75wb6" Jul 05 13:21:27.077420 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:27.077385 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/55118c56-ac0a-4696-aefb-dd8bcb891cc7-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6d456696c8-75wb6\" (UID: \"55118c56-ac0a-4696-aefb-dd8bcb891cc7\") " pod="openshift-monitoring/telemeter-client-6d456696c8-75wb6" Jul 05 13:21:27.077420 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:27.077424 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/55118c56-ac0a-4696-aefb-dd8bcb891cc7-federate-client-tls\") pod \"telemeter-client-6d456696c8-75wb6\" (UID: \"55118c56-ac0a-4696-aefb-dd8bcb891cc7\") " pod="openshift-monitoring/telemeter-client-6d456696c8-75wb6" Jul 05 13:21:27.077618 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:27.077445 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tr5t4\" (UniqueName: \"kubernetes.io/projected/55118c56-ac0a-4696-aefb-dd8bcb891cc7-kube-api-access-tr5t4\") pod \"telemeter-client-6d456696c8-75wb6\" (UID: \"55118c56-ac0a-4696-aefb-dd8bcb891cc7\") " pod="openshift-monitoring/telemeter-client-6d456696c8-75wb6" Jul 05 13:21:27.077618 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:27.077499 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/55118c56-ac0a-4696-aefb-dd8bcb891cc7-telemeter-client-tls\") pod \"telemeter-client-6d456696c8-75wb6\" (UID: \"55118c56-ac0a-4696-aefb-dd8bcb891cc7\") " pod="openshift-monitoring/telemeter-client-6d456696c8-75wb6" Jul 05 13:21:27.077618 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:27.077523 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55118c56-ac0a-4696-aefb-dd8bcb891cc7-serving-certs-ca-bundle\") pod \"telemeter-client-6d456696c8-75wb6\" (UID: \"55118c56-ac0a-4696-aefb-dd8bcb891cc7\") " pod="openshift-monitoring/telemeter-client-6d456696c8-75wb6" Jul 05 13:21:27.077618 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:27.077559 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/55118c56-ac0a-4696-aefb-dd8bcb891cc7-metrics-client-ca\") pod \"telemeter-client-6d456696c8-75wb6\" (UID: \"55118c56-ac0a-4696-aefb-dd8bcb891cc7\") " pod="openshift-monitoring/telemeter-client-6d456696c8-75wb6" Jul 05 13:21:27.077802 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:27.077773 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/55118c56-ac0a-4696-aefb-dd8bcb891cc7-secret-telemeter-client\") pod \"telemeter-client-6d456696c8-75wb6\" (UID: \"55118c56-ac0a-4696-aefb-dd8bcb891cc7\") " pod="openshift-monitoring/telemeter-client-6d456696c8-75wb6" Jul 05 13:21:27.077870 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:27.077819 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55118c56-ac0a-4696-aefb-dd8bcb891cc7-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6d456696c8-75wb6\" (UID: \"55118c56-ac0a-4696-aefb-dd8bcb891cc7\") " pod="openshift-monitoring/telemeter-client-6d456696c8-75wb6" Jul 05 13:21:27.078412 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:27.078384 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/55118c56-ac0a-4696-aefb-dd8bcb891cc7-metrics-client-ca\") pod \"telemeter-client-6d456696c8-75wb6\" (UID: \"55118c56-ac0a-4696-aefb-dd8bcb891cc7\") " pod="openshift-monitoring/telemeter-client-6d456696c8-75wb6" Jul 05 13:21:27.078609 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:27.078417 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55118c56-ac0a-4696-aefb-dd8bcb891cc7-serving-certs-ca-bundle\") pod \"telemeter-client-6d456696c8-75wb6\" (UID: \"55118c56-ac0a-4696-aefb-dd8bcb891cc7\") " pod="openshift-monitoring/telemeter-client-6d456696c8-75wb6" Jul 05 13:21:27.078609 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:27.078595 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55118c56-ac0a-4696-aefb-dd8bcb891cc7-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6d456696c8-75wb6\" (UID: \"55118c56-ac0a-4696-aefb-dd8bcb891cc7\") " pod="openshift-monitoring/telemeter-client-6d456696c8-75wb6" Jul 05 13:21:27.080244 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:27.080218 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/55118c56-ac0a-4696-aefb-dd8bcb891cc7-telemeter-client-tls\") pod \"telemeter-client-6d456696c8-75wb6\" (UID: \"55118c56-ac0a-4696-aefb-dd8bcb891cc7\") " pod="openshift-monitoring/telemeter-client-6d456696c8-75wb6" Jul 05 13:21:27.080346 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:27.080218 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/55118c56-ac0a-4696-aefb-dd8bcb891cc7-federate-client-tls\") pod \"telemeter-client-6d456696c8-75wb6\" (UID: \"55118c56-ac0a-4696-aefb-dd8bcb891cc7\") " pod="openshift-monitoring/telemeter-client-6d456696c8-75wb6" Jul 05 13:21:27.080405 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:27.080339 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/55118c56-ac0a-4696-aefb-dd8bcb891cc7-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6d456696c8-75wb6\" (UID: \"55118c56-ac0a-4696-aefb-dd8bcb891cc7\") " pod="openshift-monitoring/telemeter-client-6d456696c8-75wb6" Jul 05 13:21:27.080461 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:27.080403 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/55118c56-ac0a-4696-aefb-dd8bcb891cc7-secret-telemeter-client\") pod \"telemeter-client-6d456696c8-75wb6\" (UID: \"55118c56-ac0a-4696-aefb-dd8bcb891cc7\") " pod="openshift-monitoring/telemeter-client-6d456696c8-75wb6" Jul 05 13:21:27.088484 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:27.088464 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tr5t4\" (UniqueName: \"kubernetes.io/projected/55118c56-ac0a-4696-aefb-dd8bcb891cc7-kube-api-access-tr5t4\") pod \"telemeter-client-6d456696c8-75wb6\" (UID: \"55118c56-ac0a-4696-aefb-dd8bcb891cc7\") " pod="openshift-monitoring/telemeter-client-6d456696c8-75wb6" Jul 05 13:21:27.233606 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:27.233568 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6d456696c8-75wb6" Jul 05 13:21:27.361541 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:27.361509 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6d456696c8-75wb6"] Jul 05 13:21:27.364221 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:21:27.364196 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55118c56_ac0a_4696_aefb_dd8bcb891cc7.slice/crio-298b9754990ff276633ed3ddb41ed7735098c7357cff7b2e2f80b35457b71a89 WatchSource:0}: Error finding container 298b9754990ff276633ed3ddb41ed7735098c7357cff7b2e2f80b35457b71a89: Status 404 returned error can't find the container with id 298b9754990ff276633ed3ddb41ed7735098c7357cff7b2e2f80b35457b71a89 Jul 05 13:21:27.578297 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:27.578214 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6d456696c8-75wb6" event={"ID":"55118c56-ac0a-4696-aefb-dd8bcb891cc7","Type":"ContainerStarted","Data":"298b9754990ff276633ed3ddb41ed7735098c7357cff7b2e2f80b35457b71a89"} Jul 05 13:21:29.586696 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:29.586617 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6d456696c8-75wb6" event={"ID":"55118c56-ac0a-4696-aefb-dd8bcb891cc7","Type":"ContainerStarted","Data":"4d6bfebc0902f856b6950f5b8e65072282267f88e836c888053b6d5420dfbb5d"} Jul 05 13:21:30.592794 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:30.592755 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6d456696c8-75wb6" event={"ID":"55118c56-ac0a-4696-aefb-dd8bcb891cc7","Type":"ContainerStarted","Data":"239cbc3c511a27d255cde71e70643cd641fb7b33ae79ca8028dcb66920827fae"} Jul 05 13:21:30.593225 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:30.592813 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6d456696c8-75wb6" event={"ID":"55118c56-ac0a-4696-aefb-dd8bcb891cc7","Type":"ContainerStarted","Data":"d236b6135c63f32b20fa2572a5ebb18f00b9adeac57e3bad49a369b6cfad5558"} Jul 05 13:21:30.615828 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:30.615769 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6d456696c8-75wb6" podStartSLOduration=1.914520355 podStartE2EDuration="4.61574993s" totalImagesPullingTime="2.701229575s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:21:26 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:21:30.614739393 +0000 UTC m=+163.510397494" watchObservedRunningTime="2026-07-05 13:21:30.61574993 +0000 UTC m=+163.511408021" Jul 05 13:21:31.542524 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:31.542491 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-67cc6bdbcf-zgb44"] Jul 05 13:21:31.544780 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:31.544760 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67cc6bdbcf-zgb44" Jul 05 13:21:31.558031 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:31.558006 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67cc6bdbcf-zgb44"] Jul 05 13:21:31.605649 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:31.605618 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:21:31.609763 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:31.609740 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8p9s5\" (UniqueName: \"kubernetes.io/projected/68a07074-46be-4829-9988-2e251746c689-kube-api-access-8p9s5\") pod \"console-67cc6bdbcf-zgb44\" (UID: \"68a07074-46be-4829-9988-2e251746c689\") " pod="openshift-console/console-67cc6bdbcf-zgb44" Jul 05 13:21:31.609851 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:31.609784 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/68a07074-46be-4829-9988-2e251746c689-oauth-serving-cert\") pod \"console-67cc6bdbcf-zgb44\" (UID: \"68a07074-46be-4829-9988-2e251746c689\") " pod="openshift-console/console-67cc6bdbcf-zgb44" Jul 05 13:21:31.609904 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:31.609842 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/68a07074-46be-4829-9988-2e251746c689-console-serving-cert\") pod \"console-67cc6bdbcf-zgb44\" (UID: \"68a07074-46be-4829-9988-2e251746c689\") " pod="openshift-console/console-67cc6bdbcf-zgb44" Jul 05 13:21:31.609904 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:31.609877 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/68a07074-46be-4829-9988-2e251746c689-console-config\") pod \"console-67cc6bdbcf-zgb44\" (UID: \"68a07074-46be-4829-9988-2e251746c689\") " pod="openshift-console/console-67cc6bdbcf-zgb44" Jul 05 13:21:31.610049 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:31.610023 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68a07074-46be-4829-9988-2e251746c689-trusted-ca-bundle\") pod \"console-67cc6bdbcf-zgb44\" (UID: \"68a07074-46be-4829-9988-2e251746c689\") " pod="openshift-console/console-67cc6bdbcf-zgb44" Jul 05 13:21:31.610116 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:31.610093 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/68a07074-46be-4829-9988-2e251746c689-console-oauth-config\") pod \"console-67cc6bdbcf-zgb44\" (UID: \"68a07074-46be-4829-9988-2e251746c689\") " pod="openshift-console/console-67cc6bdbcf-zgb44" Jul 05 13:21:31.610175 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:31.610152 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/68a07074-46be-4829-9988-2e251746c689-service-ca\") pod \"console-67cc6bdbcf-zgb44\" (UID: \"68a07074-46be-4829-9988-2e251746c689\") " pod="openshift-console/console-67cc6bdbcf-zgb44" Jul 05 13:21:31.711364 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:31.711329 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/68a07074-46be-4829-9988-2e251746c689-service-ca\") pod \"console-67cc6bdbcf-zgb44\" (UID: \"68a07074-46be-4829-9988-2e251746c689\") " pod="openshift-console/console-67cc6bdbcf-zgb44" Jul 05 13:21:31.711521 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:31.711386 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8p9s5\" (UniqueName: \"kubernetes.io/projected/68a07074-46be-4829-9988-2e251746c689-kube-api-access-8p9s5\") pod \"console-67cc6bdbcf-zgb44\" (UID: \"68a07074-46be-4829-9988-2e251746c689\") " pod="openshift-console/console-67cc6bdbcf-zgb44" Jul 05 13:21:31.711521 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:31.711423 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/68a07074-46be-4829-9988-2e251746c689-oauth-serving-cert\") pod \"console-67cc6bdbcf-zgb44\" (UID: \"68a07074-46be-4829-9988-2e251746c689\") " pod="openshift-console/console-67cc6bdbcf-zgb44" Jul 05 13:21:31.711521 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:31.711455 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/68a07074-46be-4829-9988-2e251746c689-console-serving-cert\") pod \"console-67cc6bdbcf-zgb44\" (UID: \"68a07074-46be-4829-9988-2e251746c689\") " pod="openshift-console/console-67cc6bdbcf-zgb44" Jul 05 13:21:31.711521 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:31.711476 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/68a07074-46be-4829-9988-2e251746c689-console-config\") pod \"console-67cc6bdbcf-zgb44\" (UID: \"68a07074-46be-4829-9988-2e251746c689\") " pod="openshift-console/console-67cc6bdbcf-zgb44" Jul 05 13:21:31.711705 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:31.711525 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68a07074-46be-4829-9988-2e251746c689-trusted-ca-bundle\") pod \"console-67cc6bdbcf-zgb44\" (UID: \"68a07074-46be-4829-9988-2e251746c689\") " pod="openshift-console/console-67cc6bdbcf-zgb44" Jul 05 13:21:31.711705 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:31.711558 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/68a07074-46be-4829-9988-2e251746c689-console-oauth-config\") pod \"console-67cc6bdbcf-zgb44\" (UID: \"68a07074-46be-4829-9988-2e251746c689\") " pod="openshift-console/console-67cc6bdbcf-zgb44" Jul 05 13:21:31.712170 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:31.712142 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/68a07074-46be-4829-9988-2e251746c689-service-ca\") pod \"console-67cc6bdbcf-zgb44\" (UID: \"68a07074-46be-4829-9988-2e251746c689\") " pod="openshift-console/console-67cc6bdbcf-zgb44" Jul 05 13:21:31.712306 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:31.712171 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/68a07074-46be-4829-9988-2e251746c689-oauth-serving-cert\") pod \"console-67cc6bdbcf-zgb44\" (UID: \"68a07074-46be-4829-9988-2e251746c689\") " pod="openshift-console/console-67cc6bdbcf-zgb44" Jul 05 13:21:31.712306 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:31.712267 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/68a07074-46be-4829-9988-2e251746c689-console-config\") pod \"console-67cc6bdbcf-zgb44\" (UID: \"68a07074-46be-4829-9988-2e251746c689\") " pod="openshift-console/console-67cc6bdbcf-zgb44" Jul 05 13:21:31.712513 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:31.712489 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68a07074-46be-4829-9988-2e251746c689-trusted-ca-bundle\") pod \"console-67cc6bdbcf-zgb44\" (UID: \"68a07074-46be-4829-9988-2e251746c689\") " pod="openshift-console/console-67cc6bdbcf-zgb44" Jul 05 13:21:31.714106 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:31.714078 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/68a07074-46be-4829-9988-2e251746c689-console-oauth-config\") pod \"console-67cc6bdbcf-zgb44\" (UID: \"68a07074-46be-4829-9988-2e251746c689\") " pod="openshift-console/console-67cc6bdbcf-zgb44" Jul 05 13:21:31.714208 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:31.714086 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/68a07074-46be-4829-9988-2e251746c689-console-serving-cert\") pod \"console-67cc6bdbcf-zgb44\" (UID: \"68a07074-46be-4829-9988-2e251746c689\") " pod="openshift-console/console-67cc6bdbcf-zgb44" Jul 05 13:21:31.720036 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:31.720014 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8p9s5\" (UniqueName: \"kubernetes.io/projected/68a07074-46be-4829-9988-2e251746c689-kube-api-access-8p9s5\") pod \"console-67cc6bdbcf-zgb44\" (UID: \"68a07074-46be-4829-9988-2e251746c689\") " pod="openshift-console/console-67cc6bdbcf-zgb44" Jul 05 13:21:31.747915 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:31.747877 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:21:31.807456 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:31.807373 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:21:31.853558 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:31.853531 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67cc6bdbcf-zgb44" Jul 05 13:21:31.978363 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:31.978335 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67cc6bdbcf-zgb44"] Jul 05 13:21:31.980997 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:21:31.980970 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68a07074_46be_4829_9988_2e251746c689.slice/crio-6fbcaa0746637991a752a8853047066cf86eb70c616b2146a0a9a975deab98bb WatchSource:0}: Error finding container 6fbcaa0746637991a752a8853047066cf86eb70c616b2146a0a9a975deab98bb: Status 404 returned error can't find the container with id 6fbcaa0746637991a752a8853047066cf86eb70c616b2146a0a9a975deab98bb Jul 05 13:21:32.348469 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:32.348430 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:21:32.403839 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:32.403809 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:21:32.600821 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:32.600738 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67cc6bdbcf-zgb44" event={"ID":"68a07074-46be-4829-9988-2e251746c689","Type":"ContainerStarted","Data":"ef64bc1bf24c94b60490b8de68f8cf8a4a7bd6619dcb6a4e7da683c7376abb08"} Jul 05 13:21:32.600821 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:32.600787 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67cc6bdbcf-zgb44" event={"ID":"68a07074-46be-4829-9988-2e251746c689","Type":"ContainerStarted","Data":"6fbcaa0746637991a752a8853047066cf86eb70c616b2146a0a9a975deab98bb"} Jul 05 13:21:32.621554 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:32.621508 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-67cc6bdbcf-zgb44" podStartSLOduration=1.6214940169999998 podStartE2EDuration="1.621494017s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:21:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:21:32.620662505 +0000 UTC m=+165.516320595" watchObservedRunningTime="2026-07-05 13:21:32.621494017 +0000 UTC m=+165.517152105" Jul 05 13:21:41.854018 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:41.853904 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-67cc6bdbcf-zgb44" Jul 05 13:21:41.854018 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:41.853999 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-67cc6bdbcf-zgb44" Jul 05 13:21:41.858741 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:41.858717 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-67cc6bdbcf-zgb44" Jul 05 13:21:42.645222 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:42.645183 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-67cc6bdbcf-zgb44" Jul 05 13:21:42.692564 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:42.692537 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c6cbc47f-j45zf"] Jul 05 13:21:42.843190 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:42.843159 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:21:42.943415 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:42.943386 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:21:43.043554 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:43.043520 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:21:52.698079 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:52.698042 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:21:52.755816 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:21:52.755786 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:22:04.488820 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:04.488785 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:22:04.543879 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:04.543838 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:22:07.715488 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:07.715433 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-7c6cbc47f-j45zf" podUID="20ee6668-7a04-483a-8bdc-bc212e16e892" containerName="console" containerID="cri-o://bf23d0e98f0e5295034d9a5254c2b0776ecd316d765013285a27df7e0ce8f597" gracePeriod=15 Jul 05 13:22:07.953995 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:07.953973 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c6cbc47f-j45zf" Jul 05 13:22:08.110555 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:08.110463 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20ee6668-7a04-483a-8bdc-bc212e16e892-console-oauth-config\") pod \"20ee6668-7a04-483a-8bdc-bc212e16e892\" (UID: \"20ee6668-7a04-483a-8bdc-bc212e16e892\") " Jul 05 13:22:08.110555 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:08.110509 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20ee6668-7a04-483a-8bdc-bc212e16e892-console-serving-cert\") pod \"20ee6668-7a04-483a-8bdc-bc212e16e892\" (UID: \"20ee6668-7a04-483a-8bdc-bc212e16e892\") " Jul 05 13:22:08.110555 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:08.110542 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20ee6668-7a04-483a-8bdc-bc212e16e892-console-config\") pod \"20ee6668-7a04-483a-8bdc-bc212e16e892\" (UID: \"20ee6668-7a04-483a-8bdc-bc212e16e892\") " Jul 05 13:22:08.110824 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:08.110561 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20ee6668-7a04-483a-8bdc-bc212e16e892-service-ca\") pod \"20ee6668-7a04-483a-8bdc-bc212e16e892\" (UID: \"20ee6668-7a04-483a-8bdc-bc212e16e892\") " Jul 05 13:22:08.110824 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:08.110586 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20ee6668-7a04-483a-8bdc-bc212e16e892-trusted-ca-bundle\") pod \"20ee6668-7a04-483a-8bdc-bc212e16e892\" (UID: \"20ee6668-7a04-483a-8bdc-bc212e16e892\") " Jul 05 13:22:08.110824 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:08.110607 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20ee6668-7a04-483a-8bdc-bc212e16e892-oauth-serving-cert\") pod \"20ee6668-7a04-483a-8bdc-bc212e16e892\" (UID: \"20ee6668-7a04-483a-8bdc-bc212e16e892\") " Jul 05 13:22:08.110824 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:08.110650 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mtnqr\" (UniqueName: \"kubernetes.io/projected/20ee6668-7a04-483a-8bdc-bc212e16e892-kube-api-access-mtnqr\") pod \"20ee6668-7a04-483a-8bdc-bc212e16e892\" (UID: \"20ee6668-7a04-483a-8bdc-bc212e16e892\") " Jul 05 13:22:08.111054 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:08.111022 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20ee6668-7a04-483a-8bdc-bc212e16e892-service-ca" (OuterVolumeSpecName: "service-ca") pod "20ee6668-7a04-483a-8bdc-bc212e16e892" (UID: "20ee6668-7a04-483a-8bdc-bc212e16e892"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:22:08.111100 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:08.111039 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20ee6668-7a04-483a-8bdc-bc212e16e892-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "20ee6668-7a04-483a-8bdc-bc212e16e892" (UID: "20ee6668-7a04-483a-8bdc-bc212e16e892"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:22:08.111100 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:08.111032 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20ee6668-7a04-483a-8bdc-bc212e16e892-console-config" (OuterVolumeSpecName: "console-config") pod "20ee6668-7a04-483a-8bdc-bc212e16e892" (UID: "20ee6668-7a04-483a-8bdc-bc212e16e892"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:22:08.111100 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:08.111073 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20ee6668-7a04-483a-8bdc-bc212e16e892-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "20ee6668-7a04-483a-8bdc-bc212e16e892" (UID: "20ee6668-7a04-483a-8bdc-bc212e16e892"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:22:08.112742 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:08.112718 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20ee6668-7a04-483a-8bdc-bc212e16e892-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "20ee6668-7a04-483a-8bdc-bc212e16e892" (UID: "20ee6668-7a04-483a-8bdc-bc212e16e892"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:22:08.112904 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:08.112888 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20ee6668-7a04-483a-8bdc-bc212e16e892-kube-api-access-mtnqr" (OuterVolumeSpecName: "kube-api-access-mtnqr") pod "20ee6668-7a04-483a-8bdc-bc212e16e892" (UID: "20ee6668-7a04-483a-8bdc-bc212e16e892"). InnerVolumeSpecName "kube-api-access-mtnqr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:22:08.113013 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:08.112990 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20ee6668-7a04-483a-8bdc-bc212e16e892-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "20ee6668-7a04-483a-8bdc-bc212e16e892" (UID: "20ee6668-7a04-483a-8bdc-bc212e16e892"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:22:08.212051 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:08.212004 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mtnqr\" (UniqueName: \"kubernetes.io/projected/20ee6668-7a04-483a-8bdc-bc212e16e892-kube-api-access-mtnqr\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:22:08.212051 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:08.212049 2573 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20ee6668-7a04-483a-8bdc-bc212e16e892-console-oauth-config\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:22:08.212051 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:08.212062 2573 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20ee6668-7a04-483a-8bdc-bc212e16e892-console-serving-cert\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:22:08.212307 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:08.212122 2573 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20ee6668-7a04-483a-8bdc-bc212e16e892-console-config\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:22:08.212307 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:08.212135 2573 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20ee6668-7a04-483a-8bdc-bc212e16e892-service-ca\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:22:08.212307 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:08.212147 2573 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20ee6668-7a04-483a-8bdc-bc212e16e892-trusted-ca-bundle\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:22:08.212307 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:08.212159 2573 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20ee6668-7a04-483a-8bdc-bc212e16e892-oauth-serving-cert\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:22:08.731642 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:08.731603 2573 generic.go:356] "Generic (PLEG): container finished" podID="20ee6668-7a04-483a-8bdc-bc212e16e892" containerID="bf23d0e98f0e5295034d9a5254c2b0776ecd316d765013285a27df7e0ce8f597" exitCode=0 Jul 05 13:22:08.731642 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:08.731640 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c6cbc47f-j45zf" Jul 05 13:22:08.732073 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:08.731641 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c6cbc47f-j45zf" event={"ID":"20ee6668-7a04-483a-8bdc-bc212e16e892","Type":"ContainerDied","Data":"bf23d0e98f0e5295034d9a5254c2b0776ecd316d765013285a27df7e0ce8f597"} Jul 05 13:22:08.732073 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:08.731772 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c6cbc47f-j45zf" event={"ID":"20ee6668-7a04-483a-8bdc-bc212e16e892","Type":"ContainerDied","Data":"7c1f2ab7f6a7b48d5c589ba2e448a89d5680a9cc4df2d4680037e087d74f8f89"} Jul 05 13:22:08.732073 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:08.731799 2573 scope.go:117] "RemoveContainer" containerID="bf23d0e98f0e5295034d9a5254c2b0776ecd316d765013285a27df7e0ce8f597" Jul 05 13:22:08.739806 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:08.739789 2573 scope.go:117] "RemoveContainer" containerID="bf23d0e98f0e5295034d9a5254c2b0776ecd316d765013285a27df7e0ce8f597" Jul 05 13:22:08.740076 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:22:08.740057 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf23d0e98f0e5295034d9a5254c2b0776ecd316d765013285a27df7e0ce8f597\": container with ID starting with bf23d0e98f0e5295034d9a5254c2b0776ecd316d765013285a27df7e0ce8f597 not found: ID does not exist" containerID="bf23d0e98f0e5295034d9a5254c2b0776ecd316d765013285a27df7e0ce8f597" Jul 05 13:22:08.740122 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:08.740087 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf23d0e98f0e5295034d9a5254c2b0776ecd316d765013285a27df7e0ce8f597"} err="failed to get container status \"bf23d0e98f0e5295034d9a5254c2b0776ecd316d765013285a27df7e0ce8f597\": rpc error: code = NotFound desc = could not find container \"bf23d0e98f0e5295034d9a5254c2b0776ecd316d765013285a27df7e0ce8f597\": container with ID starting with bf23d0e98f0e5295034d9a5254c2b0776ecd316d765013285a27df7e0ce8f597 not found: ID does not exist" Jul 05 13:22:08.751788 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:08.751765 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c6cbc47f-j45zf"] Jul 05 13:22:08.754815 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:08.754796 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7c6cbc47f-j45zf"] Jul 05 13:22:09.735369 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:09.735330 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20ee6668-7a04-483a-8bdc-bc212e16e892" path="/var/lib/kubelet/pods/20ee6668-7a04-483a-8bdc-bc212e16e892/volumes" Jul 05 13:22:37.066740 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:37.066701 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:22:37.128757 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:37.128726 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:22:45.253825 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:45.253788 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:22:47.479620 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:47.479586 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-98ccc5478-p4dhl"] Jul 05 13:22:47.480112 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:47.479989 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20ee6668-7a04-483a-8bdc-bc212e16e892" containerName="console" Jul 05 13:22:47.480112 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:47.480008 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="20ee6668-7a04-483a-8bdc-bc212e16e892" containerName="console" Jul 05 13:22:47.480112 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:47.480101 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="20ee6668-7a04-483a-8bdc-bc212e16e892" containerName="console" Jul 05 13:22:47.482931 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:47.482909 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-98ccc5478-p4dhl" Jul 05 13:22:47.505070 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:47.505046 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-98ccc5478-p4dhl"] Jul 05 13:22:47.550429 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:47.550380 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:22:47.604030 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:47.603999 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5ffe0cee-a0a5-47ce-8c1e-8c6412844909-service-ca\") pod \"console-98ccc5478-p4dhl\" (UID: \"5ffe0cee-a0a5-47ce-8c1e-8c6412844909\") " pod="openshift-console/console-98ccc5478-p4dhl" Jul 05 13:22:47.604143 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:47.604044 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5ffe0cee-a0a5-47ce-8c1e-8c6412844909-console-serving-cert\") pod \"console-98ccc5478-p4dhl\" (UID: \"5ffe0cee-a0a5-47ce-8c1e-8c6412844909\") " pod="openshift-console/console-98ccc5478-p4dhl" Jul 05 13:22:47.604143 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:47.604071 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ffe0cee-a0a5-47ce-8c1e-8c6412844909-trusted-ca-bundle\") pod \"console-98ccc5478-p4dhl\" (UID: \"5ffe0cee-a0a5-47ce-8c1e-8c6412844909\") " pod="openshift-console/console-98ccc5478-p4dhl" Jul 05 13:22:47.604235 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:47.604143 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5ffe0cee-a0a5-47ce-8c1e-8c6412844909-console-config\") pod \"console-98ccc5478-p4dhl\" (UID: \"5ffe0cee-a0a5-47ce-8c1e-8c6412844909\") " pod="openshift-console/console-98ccc5478-p4dhl" Jul 05 13:22:47.604235 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:47.604171 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5ffe0cee-a0a5-47ce-8c1e-8c6412844909-console-oauth-config\") pod \"console-98ccc5478-p4dhl\" (UID: \"5ffe0cee-a0a5-47ce-8c1e-8c6412844909\") " pod="openshift-console/console-98ccc5478-p4dhl" Jul 05 13:22:47.604235 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:47.604190 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5ffe0cee-a0a5-47ce-8c1e-8c6412844909-oauth-serving-cert\") pod \"console-98ccc5478-p4dhl\" (UID: \"5ffe0cee-a0a5-47ce-8c1e-8c6412844909\") " pod="openshift-console/console-98ccc5478-p4dhl" Jul 05 13:22:47.604235 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:47.604207 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqwmm\" (UniqueName: \"kubernetes.io/projected/5ffe0cee-a0a5-47ce-8c1e-8c6412844909-kube-api-access-tqwmm\") pod \"console-98ccc5478-p4dhl\" (UID: \"5ffe0cee-a0a5-47ce-8c1e-8c6412844909\") " pod="openshift-console/console-98ccc5478-p4dhl" Jul 05 13:22:47.631965 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:47.631923 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:22:47.705075 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:47.705052 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5ffe0cee-a0a5-47ce-8c1e-8c6412844909-oauth-serving-cert\") pod \"console-98ccc5478-p4dhl\" (UID: \"5ffe0cee-a0a5-47ce-8c1e-8c6412844909\") " pod="openshift-console/console-98ccc5478-p4dhl" Jul 05 13:22:47.705194 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:47.705081 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqwmm\" (UniqueName: \"kubernetes.io/projected/5ffe0cee-a0a5-47ce-8c1e-8c6412844909-kube-api-access-tqwmm\") pod \"console-98ccc5478-p4dhl\" (UID: \"5ffe0cee-a0a5-47ce-8c1e-8c6412844909\") " pod="openshift-console/console-98ccc5478-p4dhl" Jul 05 13:22:47.705194 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:47.705124 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5ffe0cee-a0a5-47ce-8c1e-8c6412844909-service-ca\") pod \"console-98ccc5478-p4dhl\" (UID: \"5ffe0cee-a0a5-47ce-8c1e-8c6412844909\") " pod="openshift-console/console-98ccc5478-p4dhl" Jul 05 13:22:47.705194 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:47.705162 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5ffe0cee-a0a5-47ce-8c1e-8c6412844909-console-serving-cert\") pod \"console-98ccc5478-p4dhl\" (UID: \"5ffe0cee-a0a5-47ce-8c1e-8c6412844909\") " pod="openshift-console/console-98ccc5478-p4dhl" Jul 05 13:22:47.705194 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:47.705181 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ffe0cee-a0a5-47ce-8c1e-8c6412844909-trusted-ca-bundle\") pod \"console-98ccc5478-p4dhl\" (UID: \"5ffe0cee-a0a5-47ce-8c1e-8c6412844909\") " pod="openshift-console/console-98ccc5478-p4dhl" Jul 05 13:22:47.705412 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:47.705226 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5ffe0cee-a0a5-47ce-8c1e-8c6412844909-console-config\") pod \"console-98ccc5478-p4dhl\" (UID: \"5ffe0cee-a0a5-47ce-8c1e-8c6412844909\") " pod="openshift-console/console-98ccc5478-p4dhl" Jul 05 13:22:47.705412 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:47.705249 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5ffe0cee-a0a5-47ce-8c1e-8c6412844909-console-oauth-config\") pod \"console-98ccc5478-p4dhl\" (UID: \"5ffe0cee-a0a5-47ce-8c1e-8c6412844909\") " pod="openshift-console/console-98ccc5478-p4dhl" Jul 05 13:22:47.705892 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:47.705866 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5ffe0cee-a0a5-47ce-8c1e-8c6412844909-oauth-serving-cert\") pod \"console-98ccc5478-p4dhl\" (UID: \"5ffe0cee-a0a5-47ce-8c1e-8c6412844909\") " pod="openshift-console/console-98ccc5478-p4dhl" Jul 05 13:22:47.706039 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:47.705902 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5ffe0cee-a0a5-47ce-8c1e-8c6412844909-service-ca\") pod \"console-98ccc5478-p4dhl\" (UID: \"5ffe0cee-a0a5-47ce-8c1e-8c6412844909\") " pod="openshift-console/console-98ccc5478-p4dhl" Jul 05 13:22:47.706039 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:47.705937 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5ffe0cee-a0a5-47ce-8c1e-8c6412844909-console-config\") pod \"console-98ccc5478-p4dhl\" (UID: \"5ffe0cee-a0a5-47ce-8c1e-8c6412844909\") " pod="openshift-console/console-98ccc5478-p4dhl" Jul 05 13:22:47.706178 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:47.706069 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ffe0cee-a0a5-47ce-8c1e-8c6412844909-trusted-ca-bundle\") pod \"console-98ccc5478-p4dhl\" (UID: \"5ffe0cee-a0a5-47ce-8c1e-8c6412844909\") " pod="openshift-console/console-98ccc5478-p4dhl" Jul 05 13:22:47.707556 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:47.707527 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5ffe0cee-a0a5-47ce-8c1e-8c6412844909-console-serving-cert\") pod \"console-98ccc5478-p4dhl\" (UID: \"5ffe0cee-a0a5-47ce-8c1e-8c6412844909\") " pod="openshift-console/console-98ccc5478-p4dhl" Jul 05 13:22:47.707639 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:47.707627 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5ffe0cee-a0a5-47ce-8c1e-8c6412844909-console-oauth-config\") pod \"console-98ccc5478-p4dhl\" (UID: \"5ffe0cee-a0a5-47ce-8c1e-8c6412844909\") " pod="openshift-console/console-98ccc5478-p4dhl" Jul 05 13:22:47.721138 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:47.721116 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqwmm\" (UniqueName: \"kubernetes.io/projected/5ffe0cee-a0a5-47ce-8c1e-8c6412844909-kube-api-access-tqwmm\") pod \"console-98ccc5478-p4dhl\" (UID: \"5ffe0cee-a0a5-47ce-8c1e-8c6412844909\") " pod="openshift-console/console-98ccc5478-p4dhl" Jul 05 13:22:47.791401 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:47.791318 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-98ccc5478-p4dhl" Jul 05 13:22:47.926281 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:47.926248 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-98ccc5478-p4dhl"] Jul 05 13:22:47.929824 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:22:47.929797 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ffe0cee_a0a5_47ce_8c1e_8c6412844909.slice/crio-f3d169e5c2cbd66f3fb69439e8570d8e9873caf7c268533227af1063be6fefa2 WatchSource:0}: Error finding container f3d169e5c2cbd66f3fb69439e8570d8e9873caf7c268533227af1063be6fefa2: Status 404 returned error can't find the container with id f3d169e5c2cbd66f3fb69439e8570d8e9873caf7c268533227af1063be6fefa2 Jul 05 13:22:48.871510 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:48.871473 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-98ccc5478-p4dhl" event={"ID":"5ffe0cee-a0a5-47ce-8c1e-8c6412844909","Type":"ContainerStarted","Data":"9a26532d0fe4c251808257e2413f79338d7bca0c933f1c0ccfcc3982893070aa"} Jul 05 13:22:48.872001 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:48.871523 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-98ccc5478-p4dhl" event={"ID":"5ffe0cee-a0a5-47ce-8c1e-8c6412844909","Type":"ContainerStarted","Data":"f3d169e5c2cbd66f3fb69439e8570d8e9873caf7c268533227af1063be6fefa2"} Jul 05 13:22:49.120323 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:49.120288 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:22:49.179715 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:49.179643 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:22:57.792008 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:57.791974 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-98ccc5478-p4dhl" Jul 05 13:22:57.792452 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:57.792034 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-98ccc5478-p4dhl" Jul 05 13:22:57.801067 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:57.797495 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-98ccc5478-p4dhl" Jul 05 13:22:57.824309 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:57.824258 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-98ccc5478-p4dhl" podStartSLOduration=10.824245337 podStartE2EDuration="10.824245337s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:22:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:22:48.910813966 +0000 UTC m=+241.806472055" watchObservedRunningTime="2026-07-05 13:22:57.824245337 +0000 UTC m=+250.719903426" Jul 05 13:22:57.906671 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:57.906644 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-98ccc5478-p4dhl" Jul 05 13:22:57.957647 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:57.957616 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-67cc6bdbcf-zgb44"] Jul 05 13:22:58.055033 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:58.054959 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:22:58.155536 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:58.155503 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:22:58.206474 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:22:58.206439 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:23:06.458392 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:06.458359 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:23:06.570615 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:06.570577 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:23:06.656014 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:06.655985 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:23:15.351248 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:15.351161 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:23:15.426522 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:15.426489 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:23:17.922944 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:17.922906 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:23:22.978479 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:22.978419 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-67cc6bdbcf-zgb44" podUID="68a07074-46be-4829-9988-2e251746c689" containerName="console" containerID="cri-o://ef64bc1bf24c94b60490b8de68f8cf8a4a7bd6619dcb6a4e7da683c7376abb08" gracePeriod=15 Jul 05 13:23:23.217777 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:23.217747 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67cc6bdbcf-zgb44" Jul 05 13:23:23.301496 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:23.301429 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/68a07074-46be-4829-9988-2e251746c689-service-ca\") pod \"68a07074-46be-4829-9988-2e251746c689\" (UID: \"68a07074-46be-4829-9988-2e251746c689\") " Jul 05 13:23:23.301496 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:23.301460 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/68a07074-46be-4829-9988-2e251746c689-console-config\") pod \"68a07074-46be-4829-9988-2e251746c689\" (UID: \"68a07074-46be-4829-9988-2e251746c689\") " Jul 05 13:23:23.301496 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:23.301480 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/68a07074-46be-4829-9988-2e251746c689-oauth-serving-cert\") pod \"68a07074-46be-4829-9988-2e251746c689\" (UID: \"68a07074-46be-4829-9988-2e251746c689\") " Jul 05 13:23:23.301724 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:23.301501 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8p9s5\" (UniqueName: \"kubernetes.io/projected/68a07074-46be-4829-9988-2e251746c689-kube-api-access-8p9s5\") pod \"68a07074-46be-4829-9988-2e251746c689\" (UID: \"68a07074-46be-4829-9988-2e251746c689\") " Jul 05 13:23:23.301724 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:23.301630 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68a07074-46be-4829-9988-2e251746c689-trusted-ca-bundle\") pod \"68a07074-46be-4829-9988-2e251746c689\" (UID: \"68a07074-46be-4829-9988-2e251746c689\") " Jul 05 13:23:23.301724 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:23.301675 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/68a07074-46be-4829-9988-2e251746c689-console-oauth-config\") pod \"68a07074-46be-4829-9988-2e251746c689\" (UID: \"68a07074-46be-4829-9988-2e251746c689\") " Jul 05 13:23:23.301864 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:23.301722 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/68a07074-46be-4829-9988-2e251746c689-console-serving-cert\") pod \"68a07074-46be-4829-9988-2e251746c689\" (UID: \"68a07074-46be-4829-9988-2e251746c689\") " Jul 05 13:23:23.301994 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:23.301835 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68a07074-46be-4829-9988-2e251746c689-console-config" (OuterVolumeSpecName: "console-config") pod "68a07074-46be-4829-9988-2e251746c689" (UID: "68a07074-46be-4829-9988-2e251746c689"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:23:23.301994 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:23.301981 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68a07074-46be-4829-9988-2e251746c689-service-ca" (OuterVolumeSpecName: "service-ca") pod "68a07074-46be-4829-9988-2e251746c689" (UID: "68a07074-46be-4829-9988-2e251746c689"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:23:23.302129 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:23.301990 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68a07074-46be-4829-9988-2e251746c689-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "68a07074-46be-4829-9988-2e251746c689" (UID: "68a07074-46be-4829-9988-2e251746c689"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:23:23.302129 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:23.302051 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68a07074-46be-4829-9988-2e251746c689-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "68a07074-46be-4829-9988-2e251746c689" (UID: "68a07074-46be-4829-9988-2e251746c689"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:23:23.303643 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:23.303621 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68a07074-46be-4829-9988-2e251746c689-kube-api-access-8p9s5" (OuterVolumeSpecName: "kube-api-access-8p9s5") pod "68a07074-46be-4829-9988-2e251746c689" (UID: "68a07074-46be-4829-9988-2e251746c689"). InnerVolumeSpecName "kube-api-access-8p9s5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:23:23.303793 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:23.303770 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68a07074-46be-4829-9988-2e251746c689-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "68a07074-46be-4829-9988-2e251746c689" (UID: "68a07074-46be-4829-9988-2e251746c689"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:23:23.303852 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:23.303818 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68a07074-46be-4829-9988-2e251746c689-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "68a07074-46be-4829-9988-2e251746c689" (UID: "68a07074-46be-4829-9988-2e251746c689"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:23:23.402305 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:23.402273 2573 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/68a07074-46be-4829-9988-2e251746c689-service-ca\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:23:23.402305 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:23.402299 2573 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/68a07074-46be-4829-9988-2e251746c689-console-config\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:23:23.402305 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:23.402308 2573 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/68a07074-46be-4829-9988-2e251746c689-oauth-serving-cert\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:23:23.402515 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:23.402316 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8p9s5\" (UniqueName: \"kubernetes.io/projected/68a07074-46be-4829-9988-2e251746c689-kube-api-access-8p9s5\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:23:23.402515 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:23.402326 2573 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68a07074-46be-4829-9988-2e251746c689-trusted-ca-bundle\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:23:23.402515 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:23.402335 2573 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/68a07074-46be-4829-9988-2e251746c689-console-oauth-config\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:23:23.402515 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:23.402343 2573 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/68a07074-46be-4829-9988-2e251746c689-console-serving-cert\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:23:23.996811 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:23.996721 2573 generic.go:356] "Generic (PLEG): container finished" podID="68a07074-46be-4829-9988-2e251746c689" containerID="ef64bc1bf24c94b60490b8de68f8cf8a4a7bd6619dcb6a4e7da683c7376abb08" exitCode=0 Jul 05 13:23:23.996811 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:23.996777 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67cc6bdbcf-zgb44" Jul 05 13:23:23.997288 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:23.996782 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67cc6bdbcf-zgb44" event={"ID":"68a07074-46be-4829-9988-2e251746c689","Type":"ContainerDied","Data":"ef64bc1bf24c94b60490b8de68f8cf8a4a7bd6619dcb6a4e7da683c7376abb08"} Jul 05 13:23:23.997288 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:23.996928 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67cc6bdbcf-zgb44" event={"ID":"68a07074-46be-4829-9988-2e251746c689","Type":"ContainerDied","Data":"6fbcaa0746637991a752a8853047066cf86eb70c616b2146a0a9a975deab98bb"} Jul 05 13:23:23.997288 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:23.996979 2573 scope.go:117] "RemoveContainer" containerID="ef64bc1bf24c94b60490b8de68f8cf8a4a7bd6619dcb6a4e7da683c7376abb08" Jul 05 13:23:24.005028 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:24.005007 2573 scope.go:117] "RemoveContainer" containerID="ef64bc1bf24c94b60490b8de68f8cf8a4a7bd6619dcb6a4e7da683c7376abb08" Jul 05 13:23:24.005312 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:23:24.005293 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef64bc1bf24c94b60490b8de68f8cf8a4a7bd6619dcb6a4e7da683c7376abb08\": container with ID starting with ef64bc1bf24c94b60490b8de68f8cf8a4a7bd6619dcb6a4e7da683c7376abb08 not found: ID does not exist" containerID="ef64bc1bf24c94b60490b8de68f8cf8a4a7bd6619dcb6a4e7da683c7376abb08" Jul 05 13:23:24.005364 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:24.005320 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef64bc1bf24c94b60490b8de68f8cf8a4a7bd6619dcb6a4e7da683c7376abb08"} err="failed to get container status \"ef64bc1bf24c94b60490b8de68f8cf8a4a7bd6619dcb6a4e7da683c7376abb08\": rpc error: code = NotFound desc = could not find container \"ef64bc1bf24c94b60490b8de68f8cf8a4a7bd6619dcb6a4e7da683c7376abb08\": container with ID starting with ef64bc1bf24c94b60490b8de68f8cf8a4a7bd6619dcb6a4e7da683c7376abb08 not found: ID does not exist" Jul 05 13:23:24.024915 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:24.024891 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-67cc6bdbcf-zgb44"] Jul 05 13:23:24.031081 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:24.031062 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-67cc6bdbcf-zgb44"] Jul 05 13:23:25.735233 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:25.735198 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68a07074-46be-4829-9988-2e251746c689" path="/var/lib/kubelet/pods/68a07074-46be-4829-9988-2e251746c689/volumes" Jul 05 13:23:47.556623 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:47.556591 2573 kubelet.go:1648] "Image garbage collection succeeded" Jul 05 13:23:53.110628 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:53.110586 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:23:54.259561 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:54.259512 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:23:55.958019 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:55.957988 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q44vf"] Jul 05 13:23:55.958387 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:55.958256 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68a07074-46be-4829-9988-2e251746c689" containerName="console" Jul 05 13:23:55.958387 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:55.958268 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="68a07074-46be-4829-9988-2e251746c689" containerName="console" Jul 05 13:23:55.958387 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:55.958322 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="68a07074-46be-4829-9988-2e251746c689" containerName="console" Jul 05 13:23:55.961189 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:55.961173 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q44vf" Jul 05 13:23:55.964145 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:55.964124 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 05 13:23:55.964817 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:55.964798 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 05 13:23:55.964913 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:55.964852 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-gth6t\"" Jul 05 13:23:55.993863 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:55.993829 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q44vf"] Jul 05 13:23:56.045383 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:56.045352 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c6134dcb-0886-4021-b29c-30c72b89572a-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q44vf\" (UID: \"c6134dcb-0886-4021-b29c-30c72b89572a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q44vf" Jul 05 13:23:56.045535 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:56.045394 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfkst\" (UniqueName: \"kubernetes.io/projected/c6134dcb-0886-4021-b29c-30c72b89572a-kube-api-access-lfkst\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q44vf\" (UID: \"c6134dcb-0886-4021-b29c-30c72b89572a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q44vf" Jul 05 13:23:56.045535 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:56.045501 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c6134dcb-0886-4021-b29c-30c72b89572a-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q44vf\" (UID: \"c6134dcb-0886-4021-b29c-30c72b89572a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q44vf" Jul 05 13:23:56.146071 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:56.146038 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c6134dcb-0886-4021-b29c-30c72b89572a-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q44vf\" (UID: \"c6134dcb-0886-4021-b29c-30c72b89572a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q44vf" Jul 05 13:23:56.146238 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:56.146081 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfkst\" (UniqueName: \"kubernetes.io/projected/c6134dcb-0886-4021-b29c-30c72b89572a-kube-api-access-lfkst\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q44vf\" (UID: \"c6134dcb-0886-4021-b29c-30c72b89572a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q44vf" Jul 05 13:23:56.146238 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:56.146115 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c6134dcb-0886-4021-b29c-30c72b89572a-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q44vf\" (UID: \"c6134dcb-0886-4021-b29c-30c72b89572a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q44vf" Jul 05 13:23:56.146416 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:56.146398 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c6134dcb-0886-4021-b29c-30c72b89572a-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q44vf\" (UID: \"c6134dcb-0886-4021-b29c-30c72b89572a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q44vf" Jul 05 13:23:56.146479 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:56.146462 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c6134dcb-0886-4021-b29c-30c72b89572a-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q44vf\" (UID: \"c6134dcb-0886-4021-b29c-30c72b89572a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q44vf" Jul 05 13:23:56.159659 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:56.159622 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfkst\" (UniqueName: \"kubernetes.io/projected/c6134dcb-0886-4021-b29c-30c72b89572a-kube-api-access-lfkst\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q44vf\" (UID: \"c6134dcb-0886-4021-b29c-30c72b89572a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q44vf" Jul 05 13:23:56.269804 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:56.269726 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q44vf" Jul 05 13:23:56.415543 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:56.415510 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q44vf"] Jul 05 13:23:56.418473 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:23:56.418444 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6134dcb_0886_4021_b29c_30c72b89572a.slice/crio-a6aba8fcf23fb958e8950b4c52b4b92256f3bf5a1d5e9b81a0ed122fab56e331 WatchSource:0}: Error finding container a6aba8fcf23fb958e8950b4c52b4b92256f3bf5a1d5e9b81a0ed122fab56e331: Status 404 returned error can't find the container with id a6aba8fcf23fb958e8950b4c52b4b92256f3bf5a1d5e9b81a0ed122fab56e331 Jul 05 13:23:56.420465 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:56.420446 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 05 13:23:57.115345 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:23:57.115308 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q44vf" event={"ID":"c6134dcb-0886-4021-b29c-30c72b89572a","Type":"ContainerStarted","Data":"a6aba8fcf23fb958e8950b4c52b4b92256f3bf5a1d5e9b81a0ed122fab56e331"} Jul 05 13:24:03.141390 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:03.141361 2573 generic.go:356] "Generic (PLEG): container finished" podID="c6134dcb-0886-4021-b29c-30c72b89572a" containerID="af52276f5ce5ff50e62bdfa18772e927ab77adb4d60d568cd06b8ee19b5dee24" exitCode=0 Jul 05 13:24:03.141728 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:03.141438 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q44vf" event={"ID":"c6134dcb-0886-4021-b29c-30c72b89572a","Type":"ContainerDied","Data":"af52276f5ce5ff50e62bdfa18772e927ab77adb4d60d568cd06b8ee19b5dee24"} Jul 05 13:24:06.156207 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:06.156171 2573 generic.go:356] "Generic (PLEG): container finished" podID="c6134dcb-0886-4021-b29c-30c72b89572a" containerID="e28e232366e0a06ed4d83d94db8822a112679b23f47e2bae5ebbcd2f0db1f22f" exitCode=0 Jul 05 13:24:06.156594 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:06.156217 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q44vf" event={"ID":"c6134dcb-0886-4021-b29c-30c72b89572a","Type":"ContainerDied","Data":"e28e232366e0a06ed4d83d94db8822a112679b23f47e2bae5ebbcd2f0db1f22f"} Jul 05 13:24:13.186613 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:13.186577 2573 generic.go:356] "Generic (PLEG): container finished" podID="c6134dcb-0886-4021-b29c-30c72b89572a" containerID="dd53a008b12683c2b3bb4091a92b89e5420b8440ef2f0ecf6c52cbf4c4e852e2" exitCode=0 Jul 05 13:24:13.187040 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:13.186659 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q44vf" event={"ID":"c6134dcb-0886-4021-b29c-30c72b89572a","Type":"ContainerDied","Data":"dd53a008b12683c2b3bb4091a92b89e5420b8440ef2f0ecf6c52cbf4c4e852e2"} Jul 05 13:24:14.308502 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:14.308477 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q44vf" Jul 05 13:24:14.399608 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:14.399571 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c6134dcb-0886-4021-b29c-30c72b89572a-bundle\") pod \"c6134dcb-0886-4021-b29c-30c72b89572a\" (UID: \"c6134dcb-0886-4021-b29c-30c72b89572a\") " Jul 05 13:24:14.399784 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:14.399636 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lfkst\" (UniqueName: \"kubernetes.io/projected/c6134dcb-0886-4021-b29c-30c72b89572a-kube-api-access-lfkst\") pod \"c6134dcb-0886-4021-b29c-30c72b89572a\" (UID: \"c6134dcb-0886-4021-b29c-30c72b89572a\") " Jul 05 13:24:14.399784 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:14.399669 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c6134dcb-0886-4021-b29c-30c72b89572a-util\") pod \"c6134dcb-0886-4021-b29c-30c72b89572a\" (UID: \"c6134dcb-0886-4021-b29c-30c72b89572a\") " Jul 05 13:24:14.400323 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:14.400294 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6134dcb-0886-4021-b29c-30c72b89572a-bundle" (OuterVolumeSpecName: "bundle") pod "c6134dcb-0886-4021-b29c-30c72b89572a" (UID: "c6134dcb-0886-4021-b29c-30c72b89572a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:24:14.401861 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:14.401834 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6134dcb-0886-4021-b29c-30c72b89572a-kube-api-access-lfkst" (OuterVolumeSpecName: "kube-api-access-lfkst") pod "c6134dcb-0886-4021-b29c-30c72b89572a" (UID: "c6134dcb-0886-4021-b29c-30c72b89572a"). InnerVolumeSpecName "kube-api-access-lfkst". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:24:14.404039 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:14.404007 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6134dcb-0886-4021-b29c-30c72b89572a-util" (OuterVolumeSpecName: "util") pod "c6134dcb-0886-4021-b29c-30c72b89572a" (UID: "c6134dcb-0886-4021-b29c-30c72b89572a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:24:14.500564 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:14.500460 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lfkst\" (UniqueName: \"kubernetes.io/projected/c6134dcb-0886-4021-b29c-30c72b89572a-kube-api-access-lfkst\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:24:14.500564 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:14.500504 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c6134dcb-0886-4021-b29c-30c72b89572a-util\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:24:14.500564 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:14.500515 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c6134dcb-0886-4021-b29c-30c72b89572a-bundle\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:24:15.195983 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:15.195932 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q44vf" event={"ID":"c6134dcb-0886-4021-b29c-30c72b89572a","Type":"ContainerDied","Data":"a6aba8fcf23fb958e8950b4c52b4b92256f3bf5a1d5e9b81a0ed122fab56e331"} Jul 05 13:24:15.196164 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:15.196008 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a6aba8fcf23fb958e8950b4c52b4b92256f3bf5a1d5e9b81a0ed122fab56e331" Jul 05 13:24:15.196164 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:15.195944 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q44vf" Jul 05 13:24:18.179792 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:18.179754 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:24:18.464797 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:18.464765 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:24:22.759845 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:22.759796 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:24:22.871211 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:22.871151 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:24:25.304582 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:25.304544 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:24:25.489429 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:25.489389 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:24:25.607586 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:25.607481 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:24:25.773809 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:25.773773 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-z99p4"] Jul 05 13:24:25.774074 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:25.774060 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6134dcb-0886-4021-b29c-30c72b89572a" containerName="pull" Jul 05 13:24:25.774131 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:25.774076 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6134dcb-0886-4021-b29c-30c72b89572a" containerName="pull" Jul 05 13:24:25.774131 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:25.774090 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6134dcb-0886-4021-b29c-30c72b89572a" containerName="extract" Jul 05 13:24:25.774131 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:25.774096 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6134dcb-0886-4021-b29c-30c72b89572a" containerName="extract" Jul 05 13:24:25.774131 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:25.774107 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6134dcb-0886-4021-b29c-30c72b89572a" containerName="util" Jul 05 13:24:25.774131 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:25.774113 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6134dcb-0886-4021-b29c-30c72b89572a" containerName="util" Jul 05 13:24:25.774307 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:25.774161 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6134dcb-0886-4021-b29c-30c72b89572a" containerName="extract" Jul 05 13:24:25.777454 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:25.777436 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-z99p4" Jul 05 13:24:25.781068 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:25.781046 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 05 13:24:25.781182 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:25.781073 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 05 13:24:25.850272 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:25.850243 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-z99p4"] Jul 05 13:24:25.888371 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:25.888307 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e3bebe81-2a98-4191-a417-85735e928ad8-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-z99p4\" (UID: \"e3bebe81-2a98-4191-a417-85735e928ad8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-z99p4" Jul 05 13:24:25.888485 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:25.888356 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjxx9\" (UniqueName: \"kubernetes.io/projected/e3bebe81-2a98-4191-a417-85735e928ad8-kube-api-access-jjxx9\") pod \"cert-manager-webhook-6486ffd997-z99p4\" (UID: \"e3bebe81-2a98-4191-a417-85735e928ad8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-z99p4" Jul 05 13:24:25.989386 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:25.989351 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e3bebe81-2a98-4191-a417-85735e928ad8-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-z99p4\" (UID: \"e3bebe81-2a98-4191-a417-85735e928ad8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-z99p4" Jul 05 13:24:25.989561 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:25.989399 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjxx9\" (UniqueName: \"kubernetes.io/projected/e3bebe81-2a98-4191-a417-85735e928ad8-kube-api-access-jjxx9\") pod \"cert-manager-webhook-6486ffd997-z99p4\" (UID: \"e3bebe81-2a98-4191-a417-85735e928ad8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-z99p4" Jul 05 13:24:26.018736 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:26.018711 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e3bebe81-2a98-4191-a417-85735e928ad8-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-z99p4\" (UID: \"e3bebe81-2a98-4191-a417-85735e928ad8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-z99p4" Jul 05 13:24:26.019931 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:26.019916 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjxx9\" (UniqueName: \"kubernetes.io/projected/e3bebe81-2a98-4191-a417-85735e928ad8-kube-api-access-jjxx9\") pod \"cert-manager-webhook-6486ffd997-z99p4\" (UID: \"e3bebe81-2a98-4191-a417-85735e928ad8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-z99p4" Jul 05 13:24:26.040674 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:26.040648 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwlzcc"] Jul 05 13:24:26.042841 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:26.042827 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwlzcc" Jul 05 13:24:26.045409 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:26.045388 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 05 13:24:26.045568 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:26.045547 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 05 13:24:26.045638 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:26.045624 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-gth6t\"" Jul 05 13:24:26.072144 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:26.072112 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwlzcc"] Jul 05 13:24:26.101692 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:26.101669 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-z99p4" Jul 05 13:24:26.191458 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:26.191429 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/99d242c0-ba3a-4f68-821b-a8e36ab0d78f-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwlzcc\" (UID: \"99d242c0-ba3a-4f68-821b-a8e36ab0d78f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwlzcc" Jul 05 13:24:26.191566 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:26.191466 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/99d242c0-ba3a-4f68-821b-a8e36ab0d78f-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwlzcc\" (UID: \"99d242c0-ba3a-4f68-821b-a8e36ab0d78f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwlzcc" Jul 05 13:24:26.191566 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:26.191498 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6vjb\" (UniqueName: \"kubernetes.io/projected/99d242c0-ba3a-4f68-821b-a8e36ab0d78f-kube-api-access-b6vjb\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwlzcc\" (UID: \"99d242c0-ba3a-4f68-821b-a8e36ab0d78f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwlzcc" Jul 05 13:24:26.259966 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:26.259922 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-z99p4"] Jul 05 13:24:26.270162 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:24:26.270129 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode3bebe81_2a98_4191_a417_85735e928ad8.slice/crio-270e149d281b86c42ccbfee373c89f8fb0d11a04795d9df2b2d8bf847122374b WatchSource:0}: Error finding container 270e149d281b86c42ccbfee373c89f8fb0d11a04795d9df2b2d8bf847122374b: Status 404 returned error can't find the container with id 270e149d281b86c42ccbfee373c89f8fb0d11a04795d9df2b2d8bf847122374b Jul 05 13:24:26.292541 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:26.292513 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/99d242c0-ba3a-4f68-821b-a8e36ab0d78f-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwlzcc\" (UID: \"99d242c0-ba3a-4f68-821b-a8e36ab0d78f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwlzcc" Jul 05 13:24:26.292645 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:26.292546 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/99d242c0-ba3a-4f68-821b-a8e36ab0d78f-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwlzcc\" (UID: \"99d242c0-ba3a-4f68-821b-a8e36ab0d78f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwlzcc" Jul 05 13:24:26.292645 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:26.292568 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6vjb\" (UniqueName: \"kubernetes.io/projected/99d242c0-ba3a-4f68-821b-a8e36ab0d78f-kube-api-access-b6vjb\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwlzcc\" (UID: \"99d242c0-ba3a-4f68-821b-a8e36ab0d78f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwlzcc" Jul 05 13:24:26.292887 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:26.292865 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/99d242c0-ba3a-4f68-821b-a8e36ab0d78f-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwlzcc\" (UID: \"99d242c0-ba3a-4f68-821b-a8e36ab0d78f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwlzcc" Jul 05 13:24:26.292929 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:26.292882 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/99d242c0-ba3a-4f68-821b-a8e36ab0d78f-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwlzcc\" (UID: \"99d242c0-ba3a-4f68-821b-a8e36ab0d78f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwlzcc" Jul 05 13:24:26.328832 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:26.328808 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6vjb\" (UniqueName: \"kubernetes.io/projected/99d242c0-ba3a-4f68-821b-a8e36ab0d78f-kube-api-access-b6vjb\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwlzcc\" (UID: \"99d242c0-ba3a-4f68-821b-a8e36ab0d78f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwlzcc" Jul 05 13:24:26.351666 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:26.351647 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwlzcc" Jul 05 13:24:26.532525 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:26.532503 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwlzcc"] Jul 05 13:24:26.534252 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:24:26.534223 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99d242c0_ba3a_4f68_821b_a8e36ab0d78f.slice/crio-342716283f4bc744f5501a6aa0bdf245e56a8260d9cdd3caad27e75ea61653fc WatchSource:0}: Error finding container 342716283f4bc744f5501a6aa0bdf245e56a8260d9cdd3caad27e75ea61653fc: Status 404 returned error can't find the container with id 342716283f4bc744f5501a6aa0bdf245e56a8260d9cdd3caad27e75ea61653fc Jul 05 13:24:27.240806 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:27.240767 2573 generic.go:356] "Generic (PLEG): container finished" podID="99d242c0-ba3a-4f68-821b-a8e36ab0d78f" containerID="0fc222b402310e5b356085198bca8836a497ca448e86c22f4b1d769a91266cef" exitCode=0 Jul 05 13:24:27.241007 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:27.240851 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwlzcc" event={"ID":"99d242c0-ba3a-4f68-821b-a8e36ab0d78f","Type":"ContainerDied","Data":"0fc222b402310e5b356085198bca8836a497ca448e86c22f4b1d769a91266cef"} Jul 05 13:24:27.241007 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:27.240907 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwlzcc" event={"ID":"99d242c0-ba3a-4f68-821b-a8e36ab0d78f","Type":"ContainerStarted","Data":"342716283f4bc744f5501a6aa0bdf245e56a8260d9cdd3caad27e75ea61653fc"} Jul 05 13:24:27.242271 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:27.242239 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-z99p4" event={"ID":"e3bebe81-2a98-4191-a417-85735e928ad8","Type":"ContainerStarted","Data":"270e149d281b86c42ccbfee373c89f8fb0d11a04795d9df2b2d8bf847122374b"} Jul 05 13:24:27.589074 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:27.588982 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:24:27.749475 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:27.749426 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:24:29.791129 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:29.791055 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:24:29.890220 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:29.890189 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:24:30.258075 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:30.258039 2573 generic.go:356] "Generic (PLEG): container finished" podID="99d242c0-ba3a-4f68-821b-a8e36ab0d78f" containerID="8b3ab4dea88d461c496370167c923adc08d9205255a62d05705a5fb63fa33650" exitCode=0 Jul 05 13:24:30.258256 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:30.258117 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwlzcc" event={"ID":"99d242c0-ba3a-4f68-821b-a8e36ab0d78f","Type":"ContainerDied","Data":"8b3ab4dea88d461c496370167c923adc08d9205255a62d05705a5fb63fa33650"} Jul 05 13:24:30.259557 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:30.259536 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-z99p4" event={"ID":"e3bebe81-2a98-4191-a417-85735e928ad8","Type":"ContainerStarted","Data":"db1a5fdbe0108db878ca4920d1ea1ae2139d78b0aa257378860a1bfbe2c192af"} Jul 05 13:24:30.259635 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:30.259582 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-z99p4" Jul 05 13:24:30.326029 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:30.325990 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-z99p4" podStartSLOduration=2.081342019 podStartE2EDuration="5.3259736s" totalImagesPullingTime="3.244631581s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:24:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:24:30.323643994 +0000 UTC m=+343.219302085" watchObservedRunningTime="2026-07-05 13:24:30.3259736 +0000 UTC m=+343.221631679" Jul 05 13:24:31.265987 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:31.265933 2573 generic.go:356] "Generic (PLEG): container finished" podID="99d242c0-ba3a-4f68-821b-a8e36ab0d78f" containerID="0ac8e1686ad6365460fef982b7369e6d471e7dbec35ae5e97ce88417f8e1ff52" exitCode=0 Jul 05 13:24:31.266370 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:31.265986 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwlzcc" event={"ID":"99d242c0-ba3a-4f68-821b-a8e36ab0d78f","Type":"ContainerDied","Data":"0ac8e1686ad6365460fef982b7369e6d471e7dbec35ae5e97ce88417f8e1ff52"} Jul 05 13:24:32.392735 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:32.392711 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwlzcc" Jul 05 13:24:32.440938 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:32.440908 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/99d242c0-ba3a-4f68-821b-a8e36ab0d78f-util\") pod \"99d242c0-ba3a-4f68-821b-a8e36ab0d78f\" (UID: \"99d242c0-ba3a-4f68-821b-a8e36ab0d78f\") " Jul 05 13:24:32.441062 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:32.441038 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b6vjb\" (UniqueName: \"kubernetes.io/projected/99d242c0-ba3a-4f68-821b-a8e36ab0d78f-kube-api-access-b6vjb\") pod \"99d242c0-ba3a-4f68-821b-a8e36ab0d78f\" (UID: \"99d242c0-ba3a-4f68-821b-a8e36ab0d78f\") " Jul 05 13:24:32.441114 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:32.441096 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/99d242c0-ba3a-4f68-821b-a8e36ab0d78f-bundle\") pod \"99d242c0-ba3a-4f68-821b-a8e36ab0d78f\" (UID: \"99d242c0-ba3a-4f68-821b-a8e36ab0d78f\") " Jul 05 13:24:32.441520 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:32.441488 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99d242c0-ba3a-4f68-821b-a8e36ab0d78f-bundle" (OuterVolumeSpecName: "bundle") pod "99d242c0-ba3a-4f68-821b-a8e36ab0d78f" (UID: "99d242c0-ba3a-4f68-821b-a8e36ab0d78f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:24:32.443205 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:32.443179 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99d242c0-ba3a-4f68-821b-a8e36ab0d78f-kube-api-access-b6vjb" (OuterVolumeSpecName: "kube-api-access-b6vjb") pod "99d242c0-ba3a-4f68-821b-a8e36ab0d78f" (UID: "99d242c0-ba3a-4f68-821b-a8e36ab0d78f"). InnerVolumeSpecName "kube-api-access-b6vjb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:24:32.445125 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:32.445095 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99d242c0-ba3a-4f68-821b-a8e36ab0d78f-util" (OuterVolumeSpecName: "util") pod "99d242c0-ba3a-4f68-821b-a8e36ab0d78f" (UID: "99d242c0-ba3a-4f68-821b-a8e36ab0d78f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:24:32.542103 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:32.542031 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b6vjb\" (UniqueName: \"kubernetes.io/projected/99d242c0-ba3a-4f68-821b-a8e36ab0d78f-kube-api-access-b6vjb\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:24:32.542103 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:32.542065 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/99d242c0-ba3a-4f68-821b-a8e36ab0d78f-bundle\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:24:32.542103 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:32.542076 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/99d242c0-ba3a-4f68-821b-a8e36ab0d78f-util\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:24:33.274575 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:33.274537 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwlzcc" event={"ID":"99d242c0-ba3a-4f68-821b-a8e36ab0d78f","Type":"ContainerDied","Data":"342716283f4bc744f5501a6aa0bdf245e56a8260d9cdd3caad27e75ea61653fc"} Jul 05 13:24:33.274575 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:33.274561 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwlzcc" Jul 05 13:24:33.274575 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:33.274583 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="342716283f4bc744f5501a6aa0bdf245e56a8260d9cdd3caad27e75ea61653fc" Jul 05 13:24:35.741802 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:35.741757 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:24:36.268697 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:36.268667 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-z99p4" Jul 05 13:24:36.690354 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:36.690318 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-n4dv4"] Jul 05 13:24:36.690688 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:36.690671 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99d242c0-ba3a-4f68-821b-a8e36ab0d78f" containerName="extract" Jul 05 13:24:36.690769 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:36.690693 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="99d242c0-ba3a-4f68-821b-a8e36ab0d78f" containerName="extract" Jul 05 13:24:36.690769 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:36.690705 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99d242c0-ba3a-4f68-821b-a8e36ab0d78f" containerName="pull" Jul 05 13:24:36.690769 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:36.690712 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="99d242c0-ba3a-4f68-821b-a8e36ab0d78f" containerName="pull" Jul 05 13:24:36.690769 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:36.690735 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99d242c0-ba3a-4f68-821b-a8e36ab0d78f" containerName="util" Jul 05 13:24:36.690769 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:36.690744 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="99d242c0-ba3a-4f68-821b-a8e36ab0d78f" containerName="util" Jul 05 13:24:36.691036 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:36.690839 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="99d242c0-ba3a-4f68-821b-a8e36ab0d78f" containerName="extract" Jul 05 13:24:36.694539 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:36.694517 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-n4dv4" Jul 05 13:24:36.698979 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:36.698963 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-k49c7\"" Jul 05 13:24:36.721864 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:36.721842 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-n4dv4"] Jul 05 13:24:36.778114 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:36.778080 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgnn5\" (UniqueName: \"kubernetes.io/projected/47fec151-191b-4259-b5ff-9a2f08e54066-kube-api-access-xgnn5\") pod \"cert-manager-55b4d954d5-n4dv4\" (UID: \"47fec151-191b-4259-b5ff-9a2f08e54066\") " pod="cert-manager/cert-manager-55b4d954d5-n4dv4" Jul 05 13:24:36.778461 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:36.778149 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/47fec151-191b-4259-b5ff-9a2f08e54066-bound-sa-token\") pod \"cert-manager-55b4d954d5-n4dv4\" (UID: \"47fec151-191b-4259-b5ff-9a2f08e54066\") " pod="cert-manager/cert-manager-55b4d954d5-n4dv4" Jul 05 13:24:36.879100 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:36.879070 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgnn5\" (UniqueName: \"kubernetes.io/projected/47fec151-191b-4259-b5ff-9a2f08e54066-kube-api-access-xgnn5\") pod \"cert-manager-55b4d954d5-n4dv4\" (UID: \"47fec151-191b-4259-b5ff-9a2f08e54066\") " pod="cert-manager/cert-manager-55b4d954d5-n4dv4" Jul 05 13:24:36.879226 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:36.879125 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/47fec151-191b-4259-b5ff-9a2f08e54066-bound-sa-token\") pod \"cert-manager-55b4d954d5-n4dv4\" (UID: \"47fec151-191b-4259-b5ff-9a2f08e54066\") " pod="cert-manager/cert-manager-55b4d954d5-n4dv4" Jul 05 13:24:36.892502 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:36.892474 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/47fec151-191b-4259-b5ff-9a2f08e54066-bound-sa-token\") pod \"cert-manager-55b4d954d5-n4dv4\" (UID: \"47fec151-191b-4259-b5ff-9a2f08e54066\") " pod="cert-manager/cert-manager-55b4d954d5-n4dv4" Jul 05 13:24:36.894390 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:36.894359 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgnn5\" (UniqueName: \"kubernetes.io/projected/47fec151-191b-4259-b5ff-9a2f08e54066-kube-api-access-xgnn5\") pod \"cert-manager-55b4d954d5-n4dv4\" (UID: \"47fec151-191b-4259-b5ff-9a2f08e54066\") " pod="cert-manager/cert-manager-55b4d954d5-n4dv4" Jul 05 13:24:37.003461 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:37.003366 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-n4dv4" Jul 05 13:24:37.166049 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:24:37.166013 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod47fec151_191b_4259_b5ff_9a2f08e54066.slice/crio-680913cac2e74338ea7448d9a2fca2f553a01fe31ca74e5e5478ddbcf2278091 WatchSource:0}: Error finding container 680913cac2e74338ea7448d9a2fca2f553a01fe31ca74e5e5478ddbcf2278091: Status 404 returned error can't find the container with id 680913cac2e74338ea7448d9a2fca2f553a01fe31ca74e5e5478ddbcf2278091 Jul 05 13:24:37.171133 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:37.169737 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-n4dv4"] Jul 05 13:24:37.291282 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:37.291205 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-n4dv4" event={"ID":"47fec151-191b-4259-b5ff-9a2f08e54066","Type":"ContainerStarted","Data":"6effd79e7e3bbed730c0df7975e444cf16a86c7370d7140d1f111d891bb695db"} Jul 05 13:24:37.291282 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:37.291244 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-n4dv4" event={"ID":"47fec151-191b-4259-b5ff-9a2f08e54066","Type":"ContainerStarted","Data":"680913cac2e74338ea7448d9a2fca2f553a01fe31ca74e5e5478ddbcf2278091"} Jul 05 13:24:37.318053 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:37.318001 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-n4dv4" podStartSLOduration=1.317986319 podStartE2EDuration="1.317986319s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:24:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:24:37.316247126 +0000 UTC m=+350.211905216" watchObservedRunningTime="2026-07-05 13:24:37.317986319 +0000 UTC m=+350.213644405" Jul 05 13:24:37.727778 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:37.727742 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:24:37.821555 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:37.821521 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:24:43.745690 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:43.745611 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:24:43.845489 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:43.845457 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:24:45.402610 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:45.402578 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afllvrv"] Jul 05 13:24:45.406037 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:45.406020 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afllvrv" Jul 05 13:24:45.408435 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:45.408414 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 05 13:24:45.409240 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:45.409207 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-gth6t\"" Jul 05 13:24:45.409363 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:45.409249 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 05 13:24:45.414834 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:45.414812 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afllvrv"] Jul 05 13:24:45.442717 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:45.442683 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25cmv\" (UniqueName: \"kubernetes.io/projected/f1abc8e8-3ab9-4047-b17f-4c002c6f1275-kube-api-access-25cmv\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afllvrv\" (UID: \"f1abc8e8-3ab9-4047-b17f-4c002c6f1275\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afllvrv" Jul 05 13:24:45.442855 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:45.442732 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f1abc8e8-3ab9-4047-b17f-4c002c6f1275-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afllvrv\" (UID: \"f1abc8e8-3ab9-4047-b17f-4c002c6f1275\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afllvrv" Jul 05 13:24:45.442855 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:45.442806 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f1abc8e8-3ab9-4047-b17f-4c002c6f1275-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afllvrv\" (UID: \"f1abc8e8-3ab9-4047-b17f-4c002c6f1275\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afllvrv" Jul 05 13:24:45.544114 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:45.544055 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25cmv\" (UniqueName: \"kubernetes.io/projected/f1abc8e8-3ab9-4047-b17f-4c002c6f1275-kube-api-access-25cmv\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afllvrv\" (UID: \"f1abc8e8-3ab9-4047-b17f-4c002c6f1275\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afllvrv" Jul 05 13:24:45.544114 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:45.544125 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f1abc8e8-3ab9-4047-b17f-4c002c6f1275-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afllvrv\" (UID: \"f1abc8e8-3ab9-4047-b17f-4c002c6f1275\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afllvrv" Jul 05 13:24:45.544329 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:45.544161 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f1abc8e8-3ab9-4047-b17f-4c002c6f1275-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afllvrv\" (UID: \"f1abc8e8-3ab9-4047-b17f-4c002c6f1275\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afllvrv" Jul 05 13:24:45.544585 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:45.544564 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f1abc8e8-3ab9-4047-b17f-4c002c6f1275-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afllvrv\" (UID: \"f1abc8e8-3ab9-4047-b17f-4c002c6f1275\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afllvrv" Jul 05 13:24:45.544622 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:45.544581 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f1abc8e8-3ab9-4047-b17f-4c002c6f1275-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afllvrv\" (UID: \"f1abc8e8-3ab9-4047-b17f-4c002c6f1275\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afllvrv" Jul 05 13:24:45.552384 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:45.552360 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-25cmv\" (UniqueName: \"kubernetes.io/projected/f1abc8e8-3ab9-4047-b17f-4c002c6f1275-kube-api-access-25cmv\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afllvrv\" (UID: \"f1abc8e8-3ab9-4047-b17f-4c002c6f1275\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afllvrv" Jul 05 13:24:45.715764 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:45.715731 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afllvrv" Jul 05 13:24:45.859250 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:45.859223 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afllvrv"] Jul 05 13:24:45.861609 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:24:45.861568 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1abc8e8_3ab9_4047_b17f_4c002c6f1275.slice/crio-1d2cefa548f091ab4b38e057e840ae452b4c55cb0bf1b01ed173d4548b9555bf WatchSource:0}: Error finding container 1d2cefa548f091ab4b38e057e840ae452b4c55cb0bf1b01ed173d4548b9555bf: Status 404 returned error can't find the container with id 1d2cefa548f091ab4b38e057e840ae452b4c55cb0bf1b01ed173d4548b9555bf Jul 05 13:24:45.878009 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:45.877978 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:24:46.328042 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:46.328001 2573 generic.go:356] "Generic (PLEG): container finished" podID="f1abc8e8-3ab9-4047-b17f-4c002c6f1275" containerID="f6d190083ec4e60052d52d1e6d38b52e6223cde1bacb48abd5557ccef0c37fed" exitCode=0 Jul 05 13:24:46.328237 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:46.328071 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afllvrv" event={"ID":"f1abc8e8-3ab9-4047-b17f-4c002c6f1275","Type":"ContainerDied","Data":"f6d190083ec4e60052d52d1e6d38b52e6223cde1bacb48abd5557ccef0c37fed"} Jul 05 13:24:46.328237 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:46.328107 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afllvrv" event={"ID":"f1abc8e8-3ab9-4047-b17f-4c002c6f1275","Type":"ContainerStarted","Data":"1d2cefa548f091ab4b38e057e840ae452b4c55cb0bf1b01ed173d4548b9555bf"} Jul 05 13:24:47.333813 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:47.333777 2573 generic.go:356] "Generic (PLEG): container finished" podID="f1abc8e8-3ab9-4047-b17f-4c002c6f1275" containerID="39b18d014595298441df4b4d64c709cd58dd26fbca4d6a4e7a1d01db29b5fdc2" exitCode=0 Jul 05 13:24:47.334219 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:47.333823 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afllvrv" event={"ID":"f1abc8e8-3ab9-4047-b17f-4c002c6f1275","Type":"ContainerDied","Data":"39b18d014595298441df4b4d64c709cd58dd26fbca4d6a4e7a1d01db29b5fdc2"} Jul 05 13:24:48.340474 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:48.340435 2573 generic.go:356] "Generic (PLEG): container finished" podID="f1abc8e8-3ab9-4047-b17f-4c002c6f1275" containerID="d4475e631a05dd67f274f9641744479d1dd24eca7eaeb5786aa1cb04ca4c6b88" exitCode=0 Jul 05 13:24:48.340859 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:48.340507 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afllvrv" event={"ID":"f1abc8e8-3ab9-4047-b17f-4c002c6f1275","Type":"ContainerDied","Data":"d4475e631a05dd67f274f9641744479d1dd24eca7eaeb5786aa1cb04ca4c6b88"} Jul 05 13:24:49.475317 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:49.475295 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afllvrv" Jul 05 13:24:49.578157 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:49.578108 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f1abc8e8-3ab9-4047-b17f-4c002c6f1275-util\") pod \"f1abc8e8-3ab9-4047-b17f-4c002c6f1275\" (UID: \"f1abc8e8-3ab9-4047-b17f-4c002c6f1275\") " Jul 05 13:24:49.578157 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:49.578163 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-25cmv\" (UniqueName: \"kubernetes.io/projected/f1abc8e8-3ab9-4047-b17f-4c002c6f1275-kube-api-access-25cmv\") pod \"f1abc8e8-3ab9-4047-b17f-4c002c6f1275\" (UID: \"f1abc8e8-3ab9-4047-b17f-4c002c6f1275\") " Jul 05 13:24:49.578362 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:49.578248 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f1abc8e8-3ab9-4047-b17f-4c002c6f1275-bundle\") pod \"f1abc8e8-3ab9-4047-b17f-4c002c6f1275\" (UID: \"f1abc8e8-3ab9-4047-b17f-4c002c6f1275\") " Jul 05 13:24:49.578902 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:49.578867 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1abc8e8-3ab9-4047-b17f-4c002c6f1275-bundle" (OuterVolumeSpecName: "bundle") pod "f1abc8e8-3ab9-4047-b17f-4c002c6f1275" (UID: "f1abc8e8-3ab9-4047-b17f-4c002c6f1275"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:24:49.580149 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:49.580129 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1abc8e8-3ab9-4047-b17f-4c002c6f1275-kube-api-access-25cmv" (OuterVolumeSpecName: "kube-api-access-25cmv") pod "f1abc8e8-3ab9-4047-b17f-4c002c6f1275" (UID: "f1abc8e8-3ab9-4047-b17f-4c002c6f1275"). InnerVolumeSpecName "kube-api-access-25cmv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:24:49.586904 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:49.586866 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1abc8e8-3ab9-4047-b17f-4c002c6f1275-util" (OuterVolumeSpecName: "util") pod "f1abc8e8-3ab9-4047-b17f-4c002c6f1275" (UID: "f1abc8e8-3ab9-4047-b17f-4c002c6f1275"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:24:49.679678 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:49.679624 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f1abc8e8-3ab9-4047-b17f-4c002c6f1275-util\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:24:49.679678 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:49.679643 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-25cmv\" (UniqueName: \"kubernetes.io/projected/f1abc8e8-3ab9-4047-b17f-4c002c6f1275-kube-api-access-25cmv\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:24:49.679678 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:49.679654 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f1abc8e8-3ab9-4047-b17f-4c002c6f1275-bundle\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:24:50.350005 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:50.349944 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afllvrv" event={"ID":"f1abc8e8-3ab9-4047-b17f-4c002c6f1275","Type":"ContainerDied","Data":"1d2cefa548f091ab4b38e057e840ae452b4c55cb0bf1b01ed173d4548b9555bf"} Jul 05 13:24:50.350202 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:50.350028 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1d2cefa548f091ab4b38e057e840ae452b4c55cb0bf1b01ed173d4548b9555bf" Jul 05 13:24:50.350202 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:50.349980 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afllvrv" Jul 05 13:24:56.787697 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:56.787656 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:24:56.889488 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:24:56.889455 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:25:01.954650 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:01.954617 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:25:02.053302 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:02.053268 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:25:02.403376 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:02.403293 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:25:02.525947 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:02.525917 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:25:02.678744 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:02.678666 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6f6c4984d4-g7qxl"] Jul 05 13:25:02.679005 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:02.678989 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1abc8e8-3ab9-4047-b17f-4c002c6f1275" containerName="extract" Jul 05 13:25:02.679052 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:02.679007 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1abc8e8-3ab9-4047-b17f-4c002c6f1275" containerName="extract" Jul 05 13:25:02.679052 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:02.679020 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1abc8e8-3ab9-4047-b17f-4c002c6f1275" containerName="pull" Jul 05 13:25:02.679052 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:02.679025 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1abc8e8-3ab9-4047-b17f-4c002c6f1275" containerName="pull" Jul 05 13:25:02.679052 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:02.679041 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1abc8e8-3ab9-4047-b17f-4c002c6f1275" containerName="util" Jul 05 13:25:02.679052 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:02.679046 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1abc8e8-3ab9-4047-b17f-4c002c6f1275" containerName="util" Jul 05 13:25:02.679193 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:02.679092 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1abc8e8-3ab9-4047-b17f-4c002c6f1275" containerName="extract" Jul 05 13:25:02.681079 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:02.681064 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6f6c4984d4-g7qxl" Jul 05 13:25:02.685861 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:02.685834 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 05 13:25:02.686173 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:02.686153 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 05 13:25:02.689262 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:02.689239 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 05 13:25:02.689380 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:02.689348 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-r7q8z\"" Jul 05 13:25:02.692252 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:02.692229 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:25:02.695532 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:02.695512 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 05 13:25:02.698352 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:02.698331 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6f6c4984d4-g7qxl"] Jul 05 13:25:02.703751 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:02.703656 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 05 13:25:02.782102 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:02.782072 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ef58a174-c066-45aa-876f-9178f8ca42b7-metrics-cert\") pod \"lws-controller-manager-6f6c4984d4-g7qxl\" (UID: \"ef58a174-c066-45aa-876f-9178f8ca42b7\") " pod="openshift-lws-operator/lws-controller-manager-6f6c4984d4-g7qxl" Jul 05 13:25:02.782274 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:02.782110 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ef58a174-c066-45aa-876f-9178f8ca42b7-cert\") pod \"lws-controller-manager-6f6c4984d4-g7qxl\" (UID: \"ef58a174-c066-45aa-876f-9178f8ca42b7\") " pod="openshift-lws-operator/lws-controller-manager-6f6c4984d4-g7qxl" Jul 05 13:25:02.782274 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:02.782137 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvdf9\" (UniqueName: \"kubernetes.io/projected/ef58a174-c066-45aa-876f-9178f8ca42b7-kube-api-access-jvdf9\") pod \"lws-controller-manager-6f6c4984d4-g7qxl\" (UID: \"ef58a174-c066-45aa-876f-9178f8ca42b7\") " pod="openshift-lws-operator/lws-controller-manager-6f6c4984d4-g7qxl" Jul 05 13:25:02.782274 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:02.782222 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ef58a174-c066-45aa-876f-9178f8ca42b7-manager-config\") pod \"lws-controller-manager-6f6c4984d4-g7qxl\" (UID: \"ef58a174-c066-45aa-876f-9178f8ca42b7\") " pod="openshift-lws-operator/lws-controller-manager-6f6c4984d4-g7qxl" Jul 05 13:25:02.883013 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:02.882975 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvdf9\" (UniqueName: \"kubernetes.io/projected/ef58a174-c066-45aa-876f-9178f8ca42b7-kube-api-access-jvdf9\") pod \"lws-controller-manager-6f6c4984d4-g7qxl\" (UID: \"ef58a174-c066-45aa-876f-9178f8ca42b7\") " pod="openshift-lws-operator/lws-controller-manager-6f6c4984d4-g7qxl" Jul 05 13:25:02.883013 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:02.883020 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ef58a174-c066-45aa-876f-9178f8ca42b7-manager-config\") pod \"lws-controller-manager-6f6c4984d4-g7qxl\" (UID: \"ef58a174-c066-45aa-876f-9178f8ca42b7\") " pod="openshift-lws-operator/lws-controller-manager-6f6c4984d4-g7qxl" Jul 05 13:25:02.883234 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:02.883071 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ef58a174-c066-45aa-876f-9178f8ca42b7-metrics-cert\") pod \"lws-controller-manager-6f6c4984d4-g7qxl\" (UID: \"ef58a174-c066-45aa-876f-9178f8ca42b7\") " pod="openshift-lws-operator/lws-controller-manager-6f6c4984d4-g7qxl" Jul 05 13:25:02.883234 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:02.883093 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ef58a174-c066-45aa-876f-9178f8ca42b7-cert\") pod \"lws-controller-manager-6f6c4984d4-g7qxl\" (UID: \"ef58a174-c066-45aa-876f-9178f8ca42b7\") " pod="openshift-lws-operator/lws-controller-manager-6f6c4984d4-g7qxl" Jul 05 13:25:02.883813 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:02.883787 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ef58a174-c066-45aa-876f-9178f8ca42b7-manager-config\") pod \"lws-controller-manager-6f6c4984d4-g7qxl\" (UID: \"ef58a174-c066-45aa-876f-9178f8ca42b7\") " pod="openshift-lws-operator/lws-controller-manager-6f6c4984d4-g7qxl" Jul 05 13:25:02.885519 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:02.885491 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ef58a174-c066-45aa-876f-9178f8ca42b7-metrics-cert\") pod \"lws-controller-manager-6f6c4984d4-g7qxl\" (UID: \"ef58a174-c066-45aa-876f-9178f8ca42b7\") " pod="openshift-lws-operator/lws-controller-manager-6f6c4984d4-g7qxl" Jul 05 13:25:02.885629 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:02.885610 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ef58a174-c066-45aa-876f-9178f8ca42b7-cert\") pod \"lws-controller-manager-6f6c4984d4-g7qxl\" (UID: \"ef58a174-c066-45aa-876f-9178f8ca42b7\") " pod="openshift-lws-operator/lws-controller-manager-6f6c4984d4-g7qxl" Jul 05 13:25:02.895040 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:02.895018 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvdf9\" (UniqueName: \"kubernetes.io/projected/ef58a174-c066-45aa-876f-9178f8ca42b7-kube-api-access-jvdf9\") pod \"lws-controller-manager-6f6c4984d4-g7qxl\" (UID: \"ef58a174-c066-45aa-876f-9178f8ca42b7\") " pod="openshift-lws-operator/lws-controller-manager-6f6c4984d4-g7qxl" Jul 05 13:25:02.990001 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:02.989971 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6f6c4984d4-g7qxl" Jul 05 13:25:03.144737 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:03.144607 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6f6c4984d4-g7qxl"] Jul 05 13:25:03.147341 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:25:03.147315 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef58a174_c066_45aa_876f_9178f8ca42b7.slice/crio-c4b1e7ea08b766758f67ef4c22808952dc2c65909498d794a41d2266a933ddcc WatchSource:0}: Error finding container c4b1e7ea08b766758f67ef4c22808952dc2c65909498d794a41d2266a933ddcc: Status 404 returned error can't find the container with id c4b1e7ea08b766758f67ef4c22808952dc2c65909498d794a41d2266a933ddcc Jul 05 13:25:03.403299 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:03.403217 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6f6c4984d4-g7qxl" event={"ID":"ef58a174-c066-45aa-876f-9178f8ca42b7","Type":"ContainerStarted","Data":"c4b1e7ea08b766758f67ef4c22808952dc2c65909498d794a41d2266a933ddcc"} Jul 05 13:25:03.520160 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:03.520126 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:25:03.627068 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:03.627035 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:25:03.774466 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:03.774438 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-b4f468d94-l5zdz"] Jul 05 13:25:03.777426 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:03.777410 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-b4f468d94-l5zdz" Jul 05 13:25:03.785191 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:03.785160 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-bmwcw\"" Jul 05 13:25:03.785308 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:03.785213 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 05 13:25:03.785308 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:03.785260 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 05 13:25:03.785756 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:03.785739 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 05 13:25:03.797485 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:03.797460 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 05 13:25:03.845734 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:03.845708 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-b4f468d94-l5zdz"] Jul 05 13:25:03.891911 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:03.891875 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vccsj\" (UniqueName: \"kubernetes.io/projected/6c0a4bc3-5917-4fc7-88e6-38c057669a80-kube-api-access-vccsj\") pod \"opendatahub-operator-controller-manager-b4f468d94-l5zdz\" (UID: \"6c0a4bc3-5917-4fc7-88e6-38c057669a80\") " pod="opendatahub/opendatahub-operator-controller-manager-b4f468d94-l5zdz" Jul 05 13:25:03.892095 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:03.891928 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6c0a4bc3-5917-4fc7-88e6-38c057669a80-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b4f468d94-l5zdz\" (UID: \"6c0a4bc3-5917-4fc7-88e6-38c057669a80\") " pod="opendatahub/opendatahub-operator-controller-manager-b4f468d94-l5zdz" Jul 05 13:25:03.892095 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:03.891988 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6c0a4bc3-5917-4fc7-88e6-38c057669a80-manifests\") pod \"opendatahub-operator-controller-manager-b4f468d94-l5zdz\" (UID: \"6c0a4bc3-5917-4fc7-88e6-38c057669a80\") " pod="opendatahub/opendatahub-operator-controller-manager-b4f468d94-l5zdz" Jul 05 13:25:03.892095 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:03.892071 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6c0a4bc3-5917-4fc7-88e6-38c057669a80-tmp\") pod \"opendatahub-operator-controller-manager-b4f468d94-l5zdz\" (UID: \"6c0a4bc3-5917-4fc7-88e6-38c057669a80\") " pod="opendatahub/opendatahub-operator-controller-manager-b4f468d94-l5zdz" Jul 05 13:25:03.892230 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:03.892098 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6c0a4bc3-5917-4fc7-88e6-38c057669a80-webhook-cert\") pod \"opendatahub-operator-controller-manager-b4f468d94-l5zdz\" (UID: \"6c0a4bc3-5917-4fc7-88e6-38c057669a80\") " pod="opendatahub/opendatahub-operator-controller-manager-b4f468d94-l5zdz" Jul 05 13:25:03.993429 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:03.993389 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6c0a4bc3-5917-4fc7-88e6-38c057669a80-manifests\") pod \"opendatahub-operator-controller-manager-b4f468d94-l5zdz\" (UID: \"6c0a4bc3-5917-4fc7-88e6-38c057669a80\") " pod="opendatahub/opendatahub-operator-controller-manager-b4f468d94-l5zdz" Jul 05 13:25:03.993872 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:03.993484 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6c0a4bc3-5917-4fc7-88e6-38c057669a80-tmp\") pod \"opendatahub-operator-controller-manager-b4f468d94-l5zdz\" (UID: \"6c0a4bc3-5917-4fc7-88e6-38c057669a80\") " pod="opendatahub/opendatahub-operator-controller-manager-b4f468d94-l5zdz" Jul 05 13:25:03.993872 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:03.993517 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6c0a4bc3-5917-4fc7-88e6-38c057669a80-webhook-cert\") pod \"opendatahub-operator-controller-manager-b4f468d94-l5zdz\" (UID: \"6c0a4bc3-5917-4fc7-88e6-38c057669a80\") " pod="opendatahub/opendatahub-operator-controller-manager-b4f468d94-l5zdz" Jul 05 13:25:03.993872 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:03.993712 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vccsj\" (UniqueName: \"kubernetes.io/projected/6c0a4bc3-5917-4fc7-88e6-38c057669a80-kube-api-access-vccsj\") pod \"opendatahub-operator-controller-manager-b4f468d94-l5zdz\" (UID: \"6c0a4bc3-5917-4fc7-88e6-38c057669a80\") " pod="opendatahub/opendatahub-operator-controller-manager-b4f468d94-l5zdz" Jul 05 13:25:03.993872 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:03.993775 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6c0a4bc3-5917-4fc7-88e6-38c057669a80-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b4f468d94-l5zdz\" (UID: \"6c0a4bc3-5917-4fc7-88e6-38c057669a80\") " pod="opendatahub/opendatahub-operator-controller-manager-b4f468d94-l5zdz" Jul 05 13:25:03.993872 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:03.993832 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6c0a4bc3-5917-4fc7-88e6-38c057669a80-manifests\") pod \"opendatahub-operator-controller-manager-b4f468d94-l5zdz\" (UID: \"6c0a4bc3-5917-4fc7-88e6-38c057669a80\") " pod="opendatahub/opendatahub-operator-controller-manager-b4f468d94-l5zdz" Jul 05 13:25:03.996039 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:03.996014 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6c0a4bc3-5917-4fc7-88e6-38c057669a80-tmp\") pod \"opendatahub-operator-controller-manager-b4f468d94-l5zdz\" (UID: \"6c0a4bc3-5917-4fc7-88e6-38c057669a80\") " pod="opendatahub/opendatahub-operator-controller-manager-b4f468d94-l5zdz" Jul 05 13:25:03.996302 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:03.996278 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6c0a4bc3-5917-4fc7-88e6-38c057669a80-webhook-cert\") pod \"opendatahub-operator-controller-manager-b4f468d94-l5zdz\" (UID: \"6c0a4bc3-5917-4fc7-88e6-38c057669a80\") " pod="opendatahub/opendatahub-operator-controller-manager-b4f468d94-l5zdz" Jul 05 13:25:03.996410 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:03.996324 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6c0a4bc3-5917-4fc7-88e6-38c057669a80-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b4f468d94-l5zdz\" (UID: \"6c0a4bc3-5917-4fc7-88e6-38c057669a80\") " pod="opendatahub/opendatahub-operator-controller-manager-b4f468d94-l5zdz" Jul 05 13:25:04.017702 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:04.017666 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vccsj\" (UniqueName: \"kubernetes.io/projected/6c0a4bc3-5917-4fc7-88e6-38c057669a80-kube-api-access-vccsj\") pod \"opendatahub-operator-controller-manager-b4f468d94-l5zdz\" (UID: \"6c0a4bc3-5917-4fc7-88e6-38c057669a80\") " pod="opendatahub/opendatahub-operator-controller-manager-b4f468d94-l5zdz" Jul 05 13:25:04.087563 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:04.087472 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-b4f468d94-l5zdz" Jul 05 13:25:04.187448 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:04.187342 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:25:04.265244 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:04.265220 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-b4f468d94-l5zdz"] Jul 05 13:25:04.267474 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:25:04.267444 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c0a4bc3_5917_4fc7_88e6_38c057669a80.slice/crio-e7e160f34f43a645e81f30989e6406c4982036fdf50cc1279fb4f283b89fec2c WatchSource:0}: Error finding container e7e160f34f43a645e81f30989e6406c4982036fdf50cc1279fb4f283b89fec2c: Status 404 returned error can't find the container with id e7e160f34f43a645e81f30989e6406c4982036fdf50cc1279fb4f283b89fec2c Jul 05 13:25:04.311203 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:04.311162 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:25:04.411600 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:04.411519 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b4f468d94-l5zdz" event={"ID":"6c0a4bc3-5917-4fc7-88e6-38c057669a80","Type":"ContainerStarted","Data":"e7e160f34f43a645e81f30989e6406c4982036fdf50cc1279fb4f283b89fec2c"} Jul 05 13:25:04.414152 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:04.414115 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:25:04.526706 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:04.526673 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:25:04.620418 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:04.620384 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:25:04.735628 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:04.735588 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:25:04.820685 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:04.820623 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:25:04.940731 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:04.940656 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:25:05.420754 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:05.420545 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6f6c4984d4-g7qxl" event={"ID":"ef58a174-c066-45aa-876f-9178f8ca42b7","Type":"ContainerStarted","Data":"4f7071a756506d25e904d8ba7059c4a3b85a8ed0e31777907d31a697b3ecbcc7"} Jul 05 13:25:05.421217 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:05.420812 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6f6c4984d4-g7qxl" Jul 05 13:25:07.432056 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:07.432025 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b4f468d94-l5zdz" event={"ID":"6c0a4bc3-5917-4fc7-88e6-38c057669a80","Type":"ContainerStarted","Data":"89ee1c1fa8d777d8601fa13ca58c42a102da9af648d3f602d144c3a20245a63b"} Jul 05 13:25:07.474571 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:07.474519 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6f6c4984d4-g7qxl" podStartSLOduration=3.881698374 podStartE2EDuration="5.474503021s" totalImagesPullingTime="1.592804647s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:25:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:25:05.488556966 +0000 UTC m=+378.384215067" watchObservedRunningTime="2026-07-05 13:25:07.474503021 +0000 UTC m=+380.370161113" Jul 05 13:25:08.437903 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:08.437864 2573 generic.go:356] "Generic (PLEG): container finished" podID="6c0a4bc3-5917-4fc7-88e6-38c057669a80" containerID="89ee1c1fa8d777d8601fa13ca58c42a102da9af648d3f602d144c3a20245a63b" exitCode=0 Jul 05 13:25:08.438356 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:08.437945 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b4f468d94-l5zdz" event={"ID":"6c0a4bc3-5917-4fc7-88e6-38c057669a80","Type":"ContainerDied","Data":"89ee1c1fa8d777d8601fa13ca58c42a102da9af648d3f602d144c3a20245a63b"} Jul 05 13:25:09.445911 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:09.445873 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b4f468d94-l5zdz" event={"ID":"6c0a4bc3-5917-4fc7-88e6-38c057669a80","Type":"ContainerStarted","Data":"9970a71fff318dc83bd53ff080f44592d0dfa1367bc87581fb27ee5441e06a6f"} Jul 05 13:25:09.446416 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:09.446048 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-b4f468d94-l5zdz" Jul 05 13:25:09.489433 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:09.489379 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-b4f468d94-l5zdz" podStartSLOduration=2.318995635 podStartE2EDuration="6.489361458s" totalImagesPullingTime="3.074066981s" totalInitContainerRuntime="1.096298842s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:25:03 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:25:09.488332478 +0000 UTC m=+382.383990568" watchObservedRunningTime="2026-07-05 13:25:09.489361458 +0000 UTC m=+382.385019544" Jul 05 13:25:14.860556 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:14.860523 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:25:14.969817 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:14.969787 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:25:16.428340 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:16.428312 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6f6c4984d4-g7qxl" Jul 05 13:25:19.762591 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:19.762558 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:25:20.293463 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:20.293428 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:25:20.386111 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:20.386070 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:25:20.452610 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:20.452580 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-b4f468d94-l5zdz" Jul 05 13:25:23.310047 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:23.310012 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:25:23.432004 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:23.431938 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:25:24.275585 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:24.275553 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:25:24.438176 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:24.438127 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:25:25.036251 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:25.036215 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:25:25.188254 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:25.188220 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:25:25.239176 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:25.239135 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-67cd4f9bbf-nw66v"] Jul 05 13:25:25.248940 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:25.248913 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-67cd4f9bbf-nw66v" Jul 05 13:25:25.251908 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:25.251886 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 05 13:25:25.252912 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:25.252884 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 05 13:25:25.253023 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:25.252933 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 05 13:25:25.253066 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:25.253030 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-rpcgq\"" Jul 05 13:25:25.253221 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:25.253203 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 05 13:25:25.269842 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:25.269807 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5nqc\" (UniqueName: \"kubernetes.io/projected/2d3096e6-d86f-4547-864e-dc4633eeaf9f-kube-api-access-d5nqc\") pod \"kube-auth-proxy-67cd4f9bbf-nw66v\" (UID: \"2d3096e6-d86f-4547-864e-dc4633eeaf9f\") " pod="openshift-ingress/kube-auth-proxy-67cd4f9bbf-nw66v" Jul 05 13:25:25.270009 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:25.269872 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2d3096e6-d86f-4547-864e-dc4633eeaf9f-tmp\") pod \"kube-auth-proxy-67cd4f9bbf-nw66v\" (UID: \"2d3096e6-d86f-4547-864e-dc4633eeaf9f\") " pod="openshift-ingress/kube-auth-proxy-67cd4f9bbf-nw66v" Jul 05 13:25:25.270009 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:25.269973 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d3096e6-d86f-4547-864e-dc4633eeaf9f-tls-certs\") pod \"kube-auth-proxy-67cd4f9bbf-nw66v\" (UID: \"2d3096e6-d86f-4547-864e-dc4633eeaf9f\") " pod="openshift-ingress/kube-auth-proxy-67cd4f9bbf-nw66v" Jul 05 13:25:25.272399 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:25.272373 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-67cd4f9bbf-nw66v"] Jul 05 13:25:25.371004 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:25.370909 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d3096e6-d86f-4547-864e-dc4633eeaf9f-tls-certs\") pod \"kube-auth-proxy-67cd4f9bbf-nw66v\" (UID: \"2d3096e6-d86f-4547-864e-dc4633eeaf9f\") " pod="openshift-ingress/kube-auth-proxy-67cd4f9bbf-nw66v" Jul 05 13:25:25.371004 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:25.370976 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5nqc\" (UniqueName: \"kubernetes.io/projected/2d3096e6-d86f-4547-864e-dc4633eeaf9f-kube-api-access-d5nqc\") pod \"kube-auth-proxy-67cd4f9bbf-nw66v\" (UID: \"2d3096e6-d86f-4547-864e-dc4633eeaf9f\") " pod="openshift-ingress/kube-auth-proxy-67cd4f9bbf-nw66v" Jul 05 13:25:25.371229 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:25.371020 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2d3096e6-d86f-4547-864e-dc4633eeaf9f-tmp\") pod \"kube-auth-proxy-67cd4f9bbf-nw66v\" (UID: \"2d3096e6-d86f-4547-864e-dc4633eeaf9f\") " pod="openshift-ingress/kube-auth-proxy-67cd4f9bbf-nw66v" Jul 05 13:25:25.373321 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:25.373293 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2d3096e6-d86f-4547-864e-dc4633eeaf9f-tmp\") pod \"kube-auth-proxy-67cd4f9bbf-nw66v\" (UID: \"2d3096e6-d86f-4547-864e-dc4633eeaf9f\") " pod="openshift-ingress/kube-auth-proxy-67cd4f9bbf-nw66v" Jul 05 13:25:25.373448 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:25.373430 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d3096e6-d86f-4547-864e-dc4633eeaf9f-tls-certs\") pod \"kube-auth-proxy-67cd4f9bbf-nw66v\" (UID: \"2d3096e6-d86f-4547-864e-dc4633eeaf9f\") " pod="openshift-ingress/kube-auth-proxy-67cd4f9bbf-nw66v" Jul 05 13:25:25.404938 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:25.404909 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5nqc\" (UniqueName: \"kubernetes.io/projected/2d3096e6-d86f-4547-864e-dc4633eeaf9f-kube-api-access-d5nqc\") pod \"kube-auth-proxy-67cd4f9bbf-nw66v\" (UID: \"2d3096e6-d86f-4547-864e-dc4633eeaf9f\") " pod="openshift-ingress/kube-auth-proxy-67cd4f9bbf-nw66v" Jul 05 13:25:25.478076 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:25.478041 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:25:25.560410 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:25.560378 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-67cd4f9bbf-nw66v" Jul 05 13:25:25.648820 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:25.648749 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:25:25.719009 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:25.718975 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-67cd4f9bbf-nw66v"] Jul 05 13:25:25.724882 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:25:25.724854 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d3096e6_d86f_4547_864e_dc4633eeaf9f.slice/crio-a862f0a5fa46fbfe6e4ebb5f9aed039cc023192b4382d1af0096cd2eb73f9e45 WatchSource:0}: Error finding container a862f0a5fa46fbfe6e4ebb5f9aed039cc023192b4382d1af0096cd2eb73f9e45: Status 404 returned error can't find the container with id a862f0a5fa46fbfe6e4ebb5f9aed039cc023192b4382d1af0096cd2eb73f9e45 Jul 05 13:25:25.770041 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:25.770004 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:25:25.906688 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:25.906609 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:25:26.035640 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:26.035607 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:25:26.447930 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:26.447820 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:25:26.523541 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:26.523492 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-67cd4f9bbf-nw66v" event={"ID":"2d3096e6-d86f-4547-864e-dc4633eeaf9f","Type":"ContainerStarted","Data":"a862f0a5fa46fbfe6e4ebb5f9aed039cc023192b4382d1af0096cd2eb73f9e45"} Jul 05 13:25:26.651107 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:26.651067 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:25:27.232051 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:27.232004 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:25:27.613724 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:27.613637 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:25:27.753833 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:27.753794 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:25:27.986488 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:27.986449 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:25:28.078977 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:28.078927 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:25:28.535750 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:28.535715 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-67cd4f9bbf-nw66v" event={"ID":"2d3096e6-d86f-4547-864e-dc4633eeaf9f","Type":"ContainerStarted","Data":"19887c39c26edba5838c5f3dab8a426f39b61c316d59df3576c35ad0aeff4560"} Jul 05 13:25:29.466157 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:29.466108 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:25:33.375095 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:33.375056 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:25:33.487001 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:33.486933 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:25:35.562056 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:35.562021 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:25:35.903902 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:35.903818 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:25:36.083670 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:36.083634 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:25:39.711147 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:39.711111 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:25:39.939384 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:39.939322 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:25:40.955737 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:40.955706 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:25:41.079186 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:41.079142 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:25:41.487994 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:41.487945 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:25:41.633737 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:41.633703 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:25:41.816867 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:41.816787 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:25:41.962761 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:25:41.962726 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:26:25.848159 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:25.848077 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:26:25.996064 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:25.996033 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:26:26.161180 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:26.161091 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:26:26.794283 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:26.794204 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:26:26.991268 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:26.991236 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:26:34.768067 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:34.768013 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-67cd4f9bbf-nw66v" podStartSLOduration=67.701124775 podStartE2EDuration="1m9.767996953s" totalImagesPullingTime="2.066872178s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:25:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:25:28.566273321 +0000 UTC m=+401.461931412" watchObservedRunningTime="2026-07-05 13:26:34.767996953 +0000 UTC m=+467.663655041" Jul 05 13:26:34.768830 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:34.768814 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-857f5b4496-mhrrz"] Jul 05 13:26:34.771029 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:34.771012 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-857f5b4496-mhrrz" Jul 05 13:26:34.794121 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:34.794086 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-857f5b4496-mhrrz"] Jul 05 13:26:34.831144 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:34.831107 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/56772071-55ed-4173-8976-cab465bd6353-console-config\") pod \"console-857f5b4496-mhrrz\" (UID: \"56772071-55ed-4173-8976-cab465bd6353\") " pod="openshift-console/console-857f5b4496-mhrrz" Jul 05 13:26:34.831144 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:34.831147 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56772071-55ed-4173-8976-cab465bd6353-trusted-ca-bundle\") pod \"console-857f5b4496-mhrrz\" (UID: \"56772071-55ed-4173-8976-cab465bd6353\") " pod="openshift-console/console-857f5b4496-mhrrz" Jul 05 13:26:34.831343 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:34.831177 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/56772071-55ed-4173-8976-cab465bd6353-console-serving-cert\") pod \"console-857f5b4496-mhrrz\" (UID: \"56772071-55ed-4173-8976-cab465bd6353\") " pod="openshift-console/console-857f5b4496-mhrrz" Jul 05 13:26:34.831343 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:34.831236 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/56772071-55ed-4173-8976-cab465bd6353-service-ca\") pod \"console-857f5b4496-mhrrz\" (UID: \"56772071-55ed-4173-8976-cab465bd6353\") " pod="openshift-console/console-857f5b4496-mhrrz" Jul 05 13:26:34.831343 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:34.831276 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/56772071-55ed-4173-8976-cab465bd6353-oauth-serving-cert\") pod \"console-857f5b4496-mhrrz\" (UID: \"56772071-55ed-4173-8976-cab465bd6353\") " pod="openshift-console/console-857f5b4496-mhrrz" Jul 05 13:26:34.831343 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:34.831296 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/56772071-55ed-4173-8976-cab465bd6353-console-oauth-config\") pod \"console-857f5b4496-mhrrz\" (UID: \"56772071-55ed-4173-8976-cab465bd6353\") " pod="openshift-console/console-857f5b4496-mhrrz" Jul 05 13:26:34.831343 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:34.831321 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlq2t\" (UniqueName: \"kubernetes.io/projected/56772071-55ed-4173-8976-cab465bd6353-kube-api-access-dlq2t\") pod \"console-857f5b4496-mhrrz\" (UID: \"56772071-55ed-4173-8976-cab465bd6353\") " pod="openshift-console/console-857f5b4496-mhrrz" Jul 05 13:26:34.852869 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:34.852842 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:26:34.932437 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:34.932388 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56772071-55ed-4173-8976-cab465bd6353-trusted-ca-bundle\") pod \"console-857f5b4496-mhrrz\" (UID: \"56772071-55ed-4173-8976-cab465bd6353\") " pod="openshift-console/console-857f5b4496-mhrrz" Jul 05 13:26:34.932613 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:34.932448 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/56772071-55ed-4173-8976-cab465bd6353-console-serving-cert\") pod \"console-857f5b4496-mhrrz\" (UID: \"56772071-55ed-4173-8976-cab465bd6353\") " pod="openshift-console/console-857f5b4496-mhrrz" Jul 05 13:26:34.932613 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:34.932491 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/56772071-55ed-4173-8976-cab465bd6353-service-ca\") pod \"console-857f5b4496-mhrrz\" (UID: \"56772071-55ed-4173-8976-cab465bd6353\") " pod="openshift-console/console-857f5b4496-mhrrz" Jul 05 13:26:34.932613 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:34.932508 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/56772071-55ed-4173-8976-cab465bd6353-oauth-serving-cert\") pod \"console-857f5b4496-mhrrz\" (UID: \"56772071-55ed-4173-8976-cab465bd6353\") " pod="openshift-console/console-857f5b4496-mhrrz" Jul 05 13:26:34.932613 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:34.932526 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/56772071-55ed-4173-8976-cab465bd6353-console-oauth-config\") pod \"console-857f5b4496-mhrrz\" (UID: \"56772071-55ed-4173-8976-cab465bd6353\") " pod="openshift-console/console-857f5b4496-mhrrz" Jul 05 13:26:34.932613 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:34.932555 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlq2t\" (UniqueName: \"kubernetes.io/projected/56772071-55ed-4173-8976-cab465bd6353-kube-api-access-dlq2t\") pod \"console-857f5b4496-mhrrz\" (UID: \"56772071-55ed-4173-8976-cab465bd6353\") " pod="openshift-console/console-857f5b4496-mhrrz" Jul 05 13:26:34.932877 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:34.932654 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/56772071-55ed-4173-8976-cab465bd6353-console-config\") pod \"console-857f5b4496-mhrrz\" (UID: \"56772071-55ed-4173-8976-cab465bd6353\") " pod="openshift-console/console-857f5b4496-mhrrz" Jul 05 13:26:34.933338 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:34.933311 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/56772071-55ed-4173-8976-cab465bd6353-service-ca\") pod \"console-857f5b4496-mhrrz\" (UID: \"56772071-55ed-4173-8976-cab465bd6353\") " pod="openshift-console/console-857f5b4496-mhrrz" Jul 05 13:26:34.933467 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:34.933421 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/56772071-55ed-4173-8976-cab465bd6353-console-config\") pod \"console-857f5b4496-mhrrz\" (UID: \"56772071-55ed-4173-8976-cab465bd6353\") " pod="openshift-console/console-857f5b4496-mhrrz" Jul 05 13:26:34.933467 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:34.933427 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/56772071-55ed-4173-8976-cab465bd6353-oauth-serving-cert\") pod \"console-857f5b4496-mhrrz\" (UID: \"56772071-55ed-4173-8976-cab465bd6353\") " pod="openshift-console/console-857f5b4496-mhrrz" Jul 05 13:26:34.933640 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:34.933616 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56772071-55ed-4173-8976-cab465bd6353-trusted-ca-bundle\") pod \"console-857f5b4496-mhrrz\" (UID: \"56772071-55ed-4173-8976-cab465bd6353\") " pod="openshift-console/console-857f5b4496-mhrrz" Jul 05 13:26:34.935513 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:34.935489 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/56772071-55ed-4173-8976-cab465bd6353-console-serving-cert\") pod \"console-857f5b4496-mhrrz\" (UID: \"56772071-55ed-4173-8976-cab465bd6353\") " pod="openshift-console/console-857f5b4496-mhrrz" Jul 05 13:26:34.935597 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:34.935510 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/56772071-55ed-4173-8976-cab465bd6353-console-oauth-config\") pod \"console-857f5b4496-mhrrz\" (UID: \"56772071-55ed-4173-8976-cab465bd6353\") " pod="openshift-console/console-857f5b4496-mhrrz" Jul 05 13:26:34.945540 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:34.945516 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlq2t\" (UniqueName: \"kubernetes.io/projected/56772071-55ed-4173-8976-cab465bd6353-kube-api-access-dlq2t\") pod \"console-857f5b4496-mhrrz\" (UID: \"56772071-55ed-4173-8976-cab465bd6353\") " pod="openshift-console/console-857f5b4496-mhrrz" Jul 05 13:26:34.953880 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:34.953851 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:26:35.080280 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:35.080188 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-857f5b4496-mhrrz" Jul 05 13:26:35.228387 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:35.228362 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-857f5b4496-mhrrz"] Jul 05 13:26:35.229779 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:26:35.229752 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56772071_55ed_4173_8976_cab465bd6353.slice/crio-4ecbe19bd2778c8ead07107c7efb7239b489ff283ef5da70a9b28ef1344dd396 WatchSource:0}: Error finding container 4ecbe19bd2778c8ead07107c7efb7239b489ff283ef5da70a9b28ef1344dd396: Status 404 returned error can't find the container with id 4ecbe19bd2778c8ead07107c7efb7239b489ff283ef5da70a9b28ef1344dd396 Jul 05 13:26:35.831184 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:35.831150 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-857f5b4496-mhrrz" event={"ID":"56772071-55ed-4173-8976-cab465bd6353","Type":"ContainerStarted","Data":"a399221fa3c85bed936e2db5e69cb1bd278345b8cd9f413fbb392fdd0587db78"} Jul 05 13:26:35.831600 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:35.831197 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-857f5b4496-mhrrz" event={"ID":"56772071-55ed-4173-8976-cab465bd6353","Type":"ContainerStarted","Data":"4ecbe19bd2778c8ead07107c7efb7239b489ff283ef5da70a9b28ef1344dd396"} Jul 05 13:26:35.860893 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:35.860848 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-857f5b4496-mhrrz" podStartSLOduration=1.8608320090000001 podStartE2EDuration="1.860832009s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:26:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:26:35.859333008 +0000 UTC m=+468.754991097" watchObservedRunningTime="2026-07-05 13:26:35.860832009 +0000 UTC m=+468.756490098" Jul 05 13:26:41.739320 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:41.739286 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rdm7q"] Jul 05 13:26:41.741713 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:41.741697 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rdm7q" Jul 05 13:26:41.744542 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:41.744521 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 05 13:26:41.745455 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:41.745436 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-xlzrc\"" Jul 05 13:26:41.745541 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:41.745465 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 05 13:26:41.756569 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:41.756546 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rdm7q"] Jul 05 13:26:41.789286 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:41.789256 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b8c4ff8d-80dc-40c8-b995-234066e407ac-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rdm7q\" (UID: \"b8c4ff8d-80dc-40c8-b995-234066e407ac\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rdm7q" Jul 05 13:26:41.789422 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:41.789294 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jn22s\" (UniqueName: \"kubernetes.io/projected/b8c4ff8d-80dc-40c8-b995-234066e407ac-kube-api-access-jn22s\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rdm7q\" (UID: \"b8c4ff8d-80dc-40c8-b995-234066e407ac\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rdm7q" Jul 05 13:26:41.789422 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:41.789320 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b8c4ff8d-80dc-40c8-b995-234066e407ac-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rdm7q\" (UID: \"b8c4ff8d-80dc-40c8-b995-234066e407ac\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rdm7q" Jul 05 13:26:41.890692 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:41.890657 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b8c4ff8d-80dc-40c8-b995-234066e407ac-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rdm7q\" (UID: \"b8c4ff8d-80dc-40c8-b995-234066e407ac\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rdm7q" Jul 05 13:26:41.890868 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:41.890712 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jn22s\" (UniqueName: \"kubernetes.io/projected/b8c4ff8d-80dc-40c8-b995-234066e407ac-kube-api-access-jn22s\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rdm7q\" (UID: \"b8c4ff8d-80dc-40c8-b995-234066e407ac\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rdm7q" Jul 05 13:26:41.890868 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:41.890754 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b8c4ff8d-80dc-40c8-b995-234066e407ac-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rdm7q\" (UID: \"b8c4ff8d-80dc-40c8-b995-234066e407ac\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rdm7q" Jul 05 13:26:41.891165 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:41.891141 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b8c4ff8d-80dc-40c8-b995-234066e407ac-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rdm7q\" (UID: \"b8c4ff8d-80dc-40c8-b995-234066e407ac\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rdm7q" Jul 05 13:26:41.891224 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:41.891150 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b8c4ff8d-80dc-40c8-b995-234066e407ac-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rdm7q\" (UID: \"b8c4ff8d-80dc-40c8-b995-234066e407ac\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rdm7q" Jul 05 13:26:41.901658 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:41.901625 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jn22s\" (UniqueName: \"kubernetes.io/projected/b8c4ff8d-80dc-40c8-b995-234066e407ac-kube-api-access-jn22s\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rdm7q\" (UID: \"b8c4ff8d-80dc-40c8-b995-234066e407ac\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rdm7q" Jul 05 13:26:42.050910 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:42.050824 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rdm7q" Jul 05 13:26:42.181099 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:42.181071 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rdm7q"] Jul 05 13:26:42.182894 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:26:42.182870 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8c4ff8d_80dc_40c8_b995_234066e407ac.slice/crio-9e7149ceccaaa258a6d3946533ab14b7b395840d0afbe63e1cdc44949005cec2 WatchSource:0}: Error finding container 9e7149ceccaaa258a6d3946533ab14b7b395840d0afbe63e1cdc44949005cec2: Status 404 returned error can't find the container with id 9e7149ceccaaa258a6d3946533ab14b7b395840d0afbe63e1cdc44949005cec2 Jul 05 13:26:42.337457 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:42.337386 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ll22v"] Jul 05 13:26:42.339731 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:42.339712 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ll22v" Jul 05 13:26:42.351315 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:42.351291 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ll22v"] Jul 05 13:26:42.394903 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:42.394871 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l79qh\" (UniqueName: \"kubernetes.io/projected/c35c9430-6adb-45bc-ac83-668b51f8e611-kube-api-access-l79qh\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ll22v\" (UID: \"c35c9430-6adb-45bc-ac83-668b51f8e611\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ll22v" Jul 05 13:26:42.395101 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:42.394912 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c35c9430-6adb-45bc-ac83-668b51f8e611-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ll22v\" (UID: \"c35c9430-6adb-45bc-ac83-668b51f8e611\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ll22v" Jul 05 13:26:42.395101 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:42.394972 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c35c9430-6adb-45bc-ac83-668b51f8e611-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ll22v\" (UID: \"c35c9430-6adb-45bc-ac83-668b51f8e611\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ll22v" Jul 05 13:26:42.495603 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:42.495573 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l79qh\" (UniqueName: \"kubernetes.io/projected/c35c9430-6adb-45bc-ac83-668b51f8e611-kube-api-access-l79qh\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ll22v\" (UID: \"c35c9430-6adb-45bc-ac83-668b51f8e611\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ll22v" Jul 05 13:26:42.495744 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:42.495616 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c35c9430-6adb-45bc-ac83-668b51f8e611-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ll22v\" (UID: \"c35c9430-6adb-45bc-ac83-668b51f8e611\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ll22v" Jul 05 13:26:42.495744 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:42.495652 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c35c9430-6adb-45bc-ac83-668b51f8e611-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ll22v\" (UID: \"c35c9430-6adb-45bc-ac83-668b51f8e611\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ll22v" Jul 05 13:26:42.495980 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:42.495943 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c35c9430-6adb-45bc-ac83-668b51f8e611-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ll22v\" (UID: \"c35c9430-6adb-45bc-ac83-668b51f8e611\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ll22v" Jul 05 13:26:42.496101 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:42.496077 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c35c9430-6adb-45bc-ac83-668b51f8e611-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ll22v\" (UID: \"c35c9430-6adb-45bc-ac83-668b51f8e611\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ll22v" Jul 05 13:26:42.505096 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:42.505074 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l79qh\" (UniqueName: \"kubernetes.io/projected/c35c9430-6adb-45bc-ac83-668b51f8e611-kube-api-access-l79qh\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ll22v\" (UID: \"c35c9430-6adb-45bc-ac83-668b51f8e611\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ll22v" Jul 05 13:26:42.648378 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:42.648307 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ll22v" Jul 05 13:26:42.755316 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:42.755283 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sj8gs"] Jul 05 13:26:42.758694 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:42.758661 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sj8gs" Jul 05 13:26:42.770605 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:42.770577 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sj8gs"] Jul 05 13:26:42.789361 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:42.789159 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ll22v"] Jul 05 13:26:42.791618 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:26:42.791592 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc35c9430_6adb_45bc_ac83_668b51f8e611.slice/crio-66a9fa98595cf37657fbb5f35d8e40e81be719c4839f048e7b233450ced38c60 WatchSource:0}: Error finding container 66a9fa98595cf37657fbb5f35d8e40e81be719c4839f048e7b233450ced38c60: Status 404 returned error can't find the container with id 66a9fa98595cf37657fbb5f35d8e40e81be719c4839f048e7b233450ced38c60 Jul 05 13:26:42.798486 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:42.798466 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/25a4701d-6fa6-4415-9125-b6c8ecea61ef-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sj8gs\" (UID: \"25a4701d-6fa6-4415-9125-b6c8ecea61ef\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sj8gs" Jul 05 13:26:42.798569 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:42.798504 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nf96z\" (UniqueName: \"kubernetes.io/projected/25a4701d-6fa6-4415-9125-b6c8ecea61ef-kube-api-access-nf96z\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sj8gs\" (UID: \"25a4701d-6fa6-4415-9125-b6c8ecea61ef\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sj8gs" Jul 05 13:26:42.798569 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:42.798528 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/25a4701d-6fa6-4415-9125-b6c8ecea61ef-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sj8gs\" (UID: \"25a4701d-6fa6-4415-9125-b6c8ecea61ef\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sj8gs" Jul 05 13:26:42.864075 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:42.864046 2573 generic.go:356] "Generic (PLEG): container finished" podID="b8c4ff8d-80dc-40c8-b995-234066e407ac" containerID="50119d4ce23d8c7c35397b60253ad5d0f2f83923ef1375964b8f0469d768c6c1" exitCode=0 Jul 05 13:26:42.864204 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:42.864140 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rdm7q" event={"ID":"b8c4ff8d-80dc-40c8-b995-234066e407ac","Type":"ContainerDied","Data":"50119d4ce23d8c7c35397b60253ad5d0f2f83923ef1375964b8f0469d768c6c1"} Jul 05 13:26:42.864204 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:42.864198 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rdm7q" event={"ID":"b8c4ff8d-80dc-40c8-b995-234066e407ac","Type":"ContainerStarted","Data":"9e7149ceccaaa258a6d3946533ab14b7b395840d0afbe63e1cdc44949005cec2"} Jul 05 13:26:42.865705 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:42.865679 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ll22v" event={"ID":"c35c9430-6adb-45bc-ac83-668b51f8e611","Type":"ContainerStarted","Data":"0c82e881967b6df1a51bdbae827a9e6423f51f9f8d6e6e28e3a4e6225961ae3f"} Jul 05 13:26:42.865828 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:42.865721 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ll22v" event={"ID":"c35c9430-6adb-45bc-ac83-668b51f8e611","Type":"ContainerStarted","Data":"66a9fa98595cf37657fbb5f35d8e40e81be719c4839f048e7b233450ced38c60"} Jul 05 13:26:42.899537 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:42.899485 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nf96z\" (UniqueName: \"kubernetes.io/projected/25a4701d-6fa6-4415-9125-b6c8ecea61ef-kube-api-access-nf96z\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sj8gs\" (UID: \"25a4701d-6fa6-4415-9125-b6c8ecea61ef\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sj8gs" Jul 05 13:26:42.899537 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:42.899522 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/25a4701d-6fa6-4415-9125-b6c8ecea61ef-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sj8gs\" (UID: \"25a4701d-6fa6-4415-9125-b6c8ecea61ef\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sj8gs" Jul 05 13:26:42.899690 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:42.899618 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/25a4701d-6fa6-4415-9125-b6c8ecea61ef-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sj8gs\" (UID: \"25a4701d-6fa6-4415-9125-b6c8ecea61ef\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sj8gs" Jul 05 13:26:42.899900 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:42.899880 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/25a4701d-6fa6-4415-9125-b6c8ecea61ef-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sj8gs\" (UID: \"25a4701d-6fa6-4415-9125-b6c8ecea61ef\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sj8gs" Jul 05 13:26:42.899978 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:42.899939 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/25a4701d-6fa6-4415-9125-b6c8ecea61ef-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sj8gs\" (UID: \"25a4701d-6fa6-4415-9125-b6c8ecea61ef\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sj8gs" Jul 05 13:26:42.908794 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:42.908769 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nf96z\" (UniqueName: \"kubernetes.io/projected/25a4701d-6fa6-4415-9125-b6c8ecea61ef-kube-api-access-nf96z\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sj8gs\" (UID: \"25a4701d-6fa6-4415-9125-b6c8ecea61ef\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sj8gs" Jul 05 13:26:43.069824 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:43.069780 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sj8gs" Jul 05 13:26:43.134775 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:43.134744 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zkc22"] Jul 05 13:26:43.137706 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:43.137680 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zkc22" Jul 05 13:26:43.150349 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:43.150290 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zkc22"] Jul 05 13:26:43.202487 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:43.202454 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7b7cee0a-df79-414c-a597-f004df20451a-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zkc22\" (UID: \"7b7cee0a-df79-414c-a597-f004df20451a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zkc22" Jul 05 13:26:43.202633 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:43.202515 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7b7cee0a-df79-414c-a597-f004df20451a-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zkc22\" (UID: \"7b7cee0a-df79-414c-a597-f004df20451a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zkc22" Jul 05 13:26:43.202633 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:43.202579 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgjmt\" (UniqueName: \"kubernetes.io/projected/7b7cee0a-df79-414c-a597-f004df20451a-kube-api-access-zgjmt\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zkc22\" (UID: \"7b7cee0a-df79-414c-a597-f004df20451a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zkc22" Jul 05 13:26:43.214824 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:43.214791 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sj8gs"] Jul 05 13:26:43.239195 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:26:43.239161 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25a4701d_6fa6_4415_9125_b6c8ecea61ef.slice/crio-8a06e91a8900a09cefbaf26c17a624fd75efbc9d6eec8844b054221c1c27e538 WatchSource:0}: Error finding container 8a06e91a8900a09cefbaf26c17a624fd75efbc9d6eec8844b054221c1c27e538: Status 404 returned error can't find the container with id 8a06e91a8900a09cefbaf26c17a624fd75efbc9d6eec8844b054221c1c27e538 Jul 05 13:26:43.303563 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:43.303535 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7b7cee0a-df79-414c-a597-f004df20451a-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zkc22\" (UID: \"7b7cee0a-df79-414c-a597-f004df20451a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zkc22" Jul 05 13:26:43.303668 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:43.303634 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7b7cee0a-df79-414c-a597-f004df20451a-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zkc22\" (UID: \"7b7cee0a-df79-414c-a597-f004df20451a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zkc22" Jul 05 13:26:43.303716 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:43.303678 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgjmt\" (UniqueName: \"kubernetes.io/projected/7b7cee0a-df79-414c-a597-f004df20451a-kube-api-access-zgjmt\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zkc22\" (UID: \"7b7cee0a-df79-414c-a597-f004df20451a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zkc22" Jul 05 13:26:43.303997 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:43.303976 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7b7cee0a-df79-414c-a597-f004df20451a-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zkc22\" (UID: \"7b7cee0a-df79-414c-a597-f004df20451a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zkc22" Jul 05 13:26:43.304086 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:43.304011 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7b7cee0a-df79-414c-a597-f004df20451a-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zkc22\" (UID: \"7b7cee0a-df79-414c-a597-f004df20451a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zkc22" Jul 05 13:26:43.313688 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:43.313658 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgjmt\" (UniqueName: \"kubernetes.io/projected/7b7cee0a-df79-414c-a597-f004df20451a-kube-api-access-zgjmt\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zkc22\" (UID: \"7b7cee0a-df79-414c-a597-f004df20451a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zkc22" Jul 05 13:26:43.447968 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:43.447922 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zkc22" Jul 05 13:26:43.580629 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:43.580496 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zkc22"] Jul 05 13:26:43.583183 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:26:43.583154 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b7cee0a_df79_414c_a597_f004df20451a.slice/crio-48678f97f90aee514c0ddce0cc02b2d8bb63fb600d8c84e55d0aaea7b4566b65 WatchSource:0}: Error finding container 48678f97f90aee514c0ddce0cc02b2d8bb63fb600d8c84e55d0aaea7b4566b65: Status 404 returned error can't find the container with id 48678f97f90aee514c0ddce0cc02b2d8bb63fb600d8c84e55d0aaea7b4566b65 Jul 05 13:26:43.871575 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:43.871490 2573 generic.go:356] "Generic (PLEG): container finished" podID="b8c4ff8d-80dc-40c8-b995-234066e407ac" containerID="d7899cd13ec4095a27d033e183dcd120ea87fa5a7a79977a5dd9724189a9b847" exitCode=0 Jul 05 13:26:43.871968 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:43.871576 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rdm7q" event={"ID":"b8c4ff8d-80dc-40c8-b995-234066e407ac","Type":"ContainerDied","Data":"d7899cd13ec4095a27d033e183dcd120ea87fa5a7a79977a5dd9724189a9b847"} Jul 05 13:26:43.873134 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:43.873016 2573 generic.go:356] "Generic (PLEG): container finished" podID="25a4701d-6fa6-4415-9125-b6c8ecea61ef" containerID="6daac29133a5765fcdb218507944703192260877d7ec943a99756cd5cd4fc896" exitCode=0 Jul 05 13:26:43.873134 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:43.873085 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sj8gs" event={"ID":"25a4701d-6fa6-4415-9125-b6c8ecea61ef","Type":"ContainerDied","Data":"6daac29133a5765fcdb218507944703192260877d7ec943a99756cd5cd4fc896"} Jul 05 13:26:43.873240 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:43.873129 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sj8gs" event={"ID":"25a4701d-6fa6-4415-9125-b6c8ecea61ef","Type":"ContainerStarted","Data":"8a06e91a8900a09cefbaf26c17a624fd75efbc9d6eec8844b054221c1c27e538"} Jul 05 13:26:43.874603 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:43.874582 2573 generic.go:356] "Generic (PLEG): container finished" podID="7b7cee0a-df79-414c-a597-f004df20451a" containerID="1c3bcb87bc2b54e4b67ba555874642ddfb9b5e15862b567113af4aabc2f95133" exitCode=0 Jul 05 13:26:43.874697 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:43.874674 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zkc22" event={"ID":"7b7cee0a-df79-414c-a597-f004df20451a","Type":"ContainerDied","Data":"1c3bcb87bc2b54e4b67ba555874642ddfb9b5e15862b567113af4aabc2f95133"} Jul 05 13:26:43.874747 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:43.874715 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zkc22" event={"ID":"7b7cee0a-df79-414c-a597-f004df20451a","Type":"ContainerStarted","Data":"48678f97f90aee514c0ddce0cc02b2d8bb63fb600d8c84e55d0aaea7b4566b65"} Jul 05 13:26:43.876512 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:43.876490 2573 generic.go:356] "Generic (PLEG): container finished" podID="c35c9430-6adb-45bc-ac83-668b51f8e611" containerID="0c82e881967b6df1a51bdbae827a9e6423f51f9f8d6e6e28e3a4e6225961ae3f" exitCode=0 Jul 05 13:26:43.876621 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:43.876543 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ll22v" event={"ID":"c35c9430-6adb-45bc-ac83-668b51f8e611","Type":"ContainerDied","Data":"0c82e881967b6df1a51bdbae827a9e6423f51f9f8d6e6e28e3a4e6225961ae3f"} Jul 05 13:26:44.883358 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:44.883325 2573 generic.go:356] "Generic (PLEG): container finished" podID="7b7cee0a-df79-414c-a597-f004df20451a" containerID="cc7dd049f20f4b9284af3714838c6e1c062b588944bbe04f09355367eb93f9fe" exitCode=0 Jul 05 13:26:44.883825 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:44.883372 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zkc22" event={"ID":"7b7cee0a-df79-414c-a597-f004df20451a","Type":"ContainerDied","Data":"cc7dd049f20f4b9284af3714838c6e1c062b588944bbe04f09355367eb93f9fe"} Jul 05 13:26:44.885149 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:44.885128 2573 generic.go:356] "Generic (PLEG): container finished" podID="c35c9430-6adb-45bc-ac83-668b51f8e611" containerID="35909454929ff72abdbdaaefade5f7b63eea12fb6eea1c22a53a4cefb9b7985b" exitCode=0 Jul 05 13:26:44.885286 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:44.885261 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ll22v" event={"ID":"c35c9430-6adb-45bc-ac83-668b51f8e611","Type":"ContainerDied","Data":"35909454929ff72abdbdaaefade5f7b63eea12fb6eea1c22a53a4cefb9b7985b"} Jul 05 13:26:44.887532 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:44.887453 2573 generic.go:356] "Generic (PLEG): container finished" podID="b8c4ff8d-80dc-40c8-b995-234066e407ac" containerID="fe88545f7cfd9ceee519ed41ff2911a309fdae0b725b1c9e5cc41c49e87e6904" exitCode=0 Jul 05 13:26:44.887532 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:44.887488 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rdm7q" event={"ID":"b8c4ff8d-80dc-40c8-b995-234066e407ac","Type":"ContainerDied","Data":"fe88545f7cfd9ceee519ed41ff2911a309fdae0b725b1c9e5cc41c49e87e6904"} Jul 05 13:26:44.889629 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:44.889597 2573 generic.go:356] "Generic (PLEG): container finished" podID="25a4701d-6fa6-4415-9125-b6c8ecea61ef" containerID="208146ae31bdca99edde6ef58a36db9cb390f8187bca2d4502a1e13de0310c17" exitCode=0 Jul 05 13:26:44.889629 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:44.889629 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sj8gs" event={"ID":"25a4701d-6fa6-4415-9125-b6c8ecea61ef","Type":"ContainerDied","Data":"208146ae31bdca99edde6ef58a36db9cb390f8187bca2d4502a1e13de0310c17"} Jul 05 13:26:45.080536 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:45.080508 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-857f5b4496-mhrrz" Jul 05 13:26:45.080618 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:45.080569 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-857f5b4496-mhrrz" Jul 05 13:26:45.085927 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:45.085904 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-857f5b4496-mhrrz" Jul 05 13:26:45.901705 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:45.901671 2573 generic.go:356] "Generic (PLEG): container finished" podID="7b7cee0a-df79-414c-a597-f004df20451a" containerID="96680acdd1e656589ae14bd1ec9c574db990b076ee281ea4bc30382f5f7cfd8f" exitCode=0 Jul 05 13:26:45.902136 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:45.901748 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zkc22" event={"ID":"7b7cee0a-df79-414c-a597-f004df20451a","Type":"ContainerDied","Data":"96680acdd1e656589ae14bd1ec9c574db990b076ee281ea4bc30382f5f7cfd8f"} Jul 05 13:26:45.903679 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:45.903657 2573 generic.go:356] "Generic (PLEG): container finished" podID="c35c9430-6adb-45bc-ac83-668b51f8e611" containerID="95abb0102c771e20f0dcfabd3f9fce37e95c2f5e21f51c70a3691ea7f78d11ed" exitCode=0 Jul 05 13:26:45.903790 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:45.903735 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ll22v" event={"ID":"c35c9430-6adb-45bc-ac83-668b51f8e611","Type":"ContainerDied","Data":"95abb0102c771e20f0dcfabd3f9fce37e95c2f5e21f51c70a3691ea7f78d11ed"} Jul 05 13:26:45.905498 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:45.905476 2573 generic.go:356] "Generic (PLEG): container finished" podID="25a4701d-6fa6-4415-9125-b6c8ecea61ef" containerID="30c22e9f6e6d11762f499d5bde46286420889a18b36504adb563cb90d56c44d2" exitCode=0 Jul 05 13:26:45.905637 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:45.905525 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sj8gs" event={"ID":"25a4701d-6fa6-4415-9125-b6c8ecea61ef","Type":"ContainerDied","Data":"30c22e9f6e6d11762f499d5bde46286420889a18b36504adb563cb90d56c44d2"} Jul 05 13:26:45.909692 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:45.909670 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-857f5b4496-mhrrz" Jul 05 13:26:46.031633 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:46.031610 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rdm7q" Jul 05 13:26:46.071015 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:46.070981 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-98ccc5478-p4dhl"] Jul 05 13:26:46.125171 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:46.125139 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b8c4ff8d-80dc-40c8-b995-234066e407ac-util\") pod \"b8c4ff8d-80dc-40c8-b995-234066e407ac\" (UID: \"b8c4ff8d-80dc-40c8-b995-234066e407ac\") " Jul 05 13:26:46.125316 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:46.125207 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jn22s\" (UniqueName: \"kubernetes.io/projected/b8c4ff8d-80dc-40c8-b995-234066e407ac-kube-api-access-jn22s\") pod \"b8c4ff8d-80dc-40c8-b995-234066e407ac\" (UID: \"b8c4ff8d-80dc-40c8-b995-234066e407ac\") " Jul 05 13:26:46.125316 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:46.125225 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b8c4ff8d-80dc-40c8-b995-234066e407ac-bundle\") pod \"b8c4ff8d-80dc-40c8-b995-234066e407ac\" (UID: \"b8c4ff8d-80dc-40c8-b995-234066e407ac\") " Jul 05 13:26:46.125813 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:46.125784 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8c4ff8d-80dc-40c8-b995-234066e407ac-bundle" (OuterVolumeSpecName: "bundle") pod "b8c4ff8d-80dc-40c8-b995-234066e407ac" (UID: "b8c4ff8d-80dc-40c8-b995-234066e407ac"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:26:46.127413 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:46.127386 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8c4ff8d-80dc-40c8-b995-234066e407ac-kube-api-access-jn22s" (OuterVolumeSpecName: "kube-api-access-jn22s") pod "b8c4ff8d-80dc-40c8-b995-234066e407ac" (UID: "b8c4ff8d-80dc-40c8-b995-234066e407ac"). InnerVolumeSpecName "kube-api-access-jn22s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:26:46.130602 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:46.130582 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8c4ff8d-80dc-40c8-b995-234066e407ac-util" (OuterVolumeSpecName: "util") pod "b8c4ff8d-80dc-40c8-b995-234066e407ac" (UID: "b8c4ff8d-80dc-40c8-b995-234066e407ac"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:26:46.221677 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:46.221638 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:26:46.226757 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:46.226736 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jn22s\" (UniqueName: \"kubernetes.io/projected/b8c4ff8d-80dc-40c8-b995-234066e407ac-kube-api-access-jn22s\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:26:46.226827 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:46.226762 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b8c4ff8d-80dc-40c8-b995-234066e407ac-bundle\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:26:46.226827 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:46.226771 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b8c4ff8d-80dc-40c8-b995-234066e407ac-util\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:26:46.319973 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:46.319924 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:26:46.386849 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:46.386816 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:26:46.912463 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:46.912437 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rdm7q" Jul 05 13:26:46.912887 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:46.912488 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rdm7q" event={"ID":"b8c4ff8d-80dc-40c8-b995-234066e407ac","Type":"ContainerDied","Data":"9e7149ceccaaa258a6d3946533ab14b7b395840d0afbe63e1cdc44949005cec2"} Jul 05 13:26:46.912887 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:46.912527 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9e7149ceccaaa258a6d3946533ab14b7b395840d0afbe63e1cdc44949005cec2" Jul 05 13:26:47.086531 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:47.086499 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zkc22" Jul 05 13:26:47.089275 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:47.089255 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sj8gs" Jul 05 13:26:47.092833 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:47.092814 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ll22v" Jul 05 13:26:47.134928 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:47.134892 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nf96z\" (UniqueName: \"kubernetes.io/projected/25a4701d-6fa6-4415-9125-b6c8ecea61ef-kube-api-access-nf96z\") pod \"25a4701d-6fa6-4415-9125-b6c8ecea61ef\" (UID: \"25a4701d-6fa6-4415-9125-b6c8ecea61ef\") " Jul 05 13:26:47.135112 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:47.134968 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/25a4701d-6fa6-4415-9125-b6c8ecea61ef-bundle\") pod \"25a4701d-6fa6-4415-9125-b6c8ecea61ef\" (UID: \"25a4701d-6fa6-4415-9125-b6c8ecea61ef\") " Jul 05 13:26:47.135112 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:47.135012 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7b7cee0a-df79-414c-a597-f004df20451a-bundle\") pod \"7b7cee0a-df79-414c-a597-f004df20451a\" (UID: \"7b7cee0a-df79-414c-a597-f004df20451a\") " Jul 05 13:26:47.135112 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:47.135048 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c35c9430-6adb-45bc-ac83-668b51f8e611-util\") pod \"c35c9430-6adb-45bc-ac83-668b51f8e611\" (UID: \"c35c9430-6adb-45bc-ac83-668b51f8e611\") " Jul 05 13:26:47.135112 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:47.135075 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/25a4701d-6fa6-4415-9125-b6c8ecea61ef-util\") pod \"25a4701d-6fa6-4415-9125-b6c8ecea61ef\" (UID: \"25a4701d-6fa6-4415-9125-b6c8ecea61ef\") " Jul 05 13:26:47.135112 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:47.135111 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c35c9430-6adb-45bc-ac83-668b51f8e611-bundle\") pod \"c35c9430-6adb-45bc-ac83-668b51f8e611\" (UID: \"c35c9430-6adb-45bc-ac83-668b51f8e611\") " Jul 05 13:26:47.135348 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:47.135136 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l79qh\" (UniqueName: \"kubernetes.io/projected/c35c9430-6adb-45bc-ac83-668b51f8e611-kube-api-access-l79qh\") pod \"c35c9430-6adb-45bc-ac83-668b51f8e611\" (UID: \"c35c9430-6adb-45bc-ac83-668b51f8e611\") " Jul 05 13:26:47.135348 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:47.135161 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7b7cee0a-df79-414c-a597-f004df20451a-util\") pod \"7b7cee0a-df79-414c-a597-f004df20451a\" (UID: \"7b7cee0a-df79-414c-a597-f004df20451a\") " Jul 05 13:26:47.135348 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:47.135212 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zgjmt\" (UniqueName: \"kubernetes.io/projected/7b7cee0a-df79-414c-a597-f004df20451a-kube-api-access-zgjmt\") pod \"7b7cee0a-df79-414c-a597-f004df20451a\" (UID: \"7b7cee0a-df79-414c-a597-f004df20451a\") " Jul 05 13:26:47.136169 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:47.135538 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25a4701d-6fa6-4415-9125-b6c8ecea61ef-bundle" (OuterVolumeSpecName: "bundle") pod "25a4701d-6fa6-4415-9125-b6c8ecea61ef" (UID: "25a4701d-6fa6-4415-9125-b6c8ecea61ef"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:26:47.136169 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:47.135708 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c35c9430-6adb-45bc-ac83-668b51f8e611-bundle" (OuterVolumeSpecName: "bundle") pod "c35c9430-6adb-45bc-ac83-668b51f8e611" (UID: "c35c9430-6adb-45bc-ac83-668b51f8e611"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:26:47.136169 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:47.135790 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b7cee0a-df79-414c-a597-f004df20451a-bundle" (OuterVolumeSpecName: "bundle") pod "7b7cee0a-df79-414c-a597-f004df20451a" (UID: "7b7cee0a-df79-414c-a597-f004df20451a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:26:47.137296 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:47.137271 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25a4701d-6fa6-4415-9125-b6c8ecea61ef-kube-api-access-nf96z" (OuterVolumeSpecName: "kube-api-access-nf96z") pod "25a4701d-6fa6-4415-9125-b6c8ecea61ef" (UID: "25a4701d-6fa6-4415-9125-b6c8ecea61ef"). InnerVolumeSpecName "kube-api-access-nf96z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:26:47.137744 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:47.137726 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b7cee0a-df79-414c-a597-f004df20451a-kube-api-access-zgjmt" (OuterVolumeSpecName: "kube-api-access-zgjmt") pod "7b7cee0a-df79-414c-a597-f004df20451a" (UID: "7b7cee0a-df79-414c-a597-f004df20451a"). InnerVolumeSpecName "kube-api-access-zgjmt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:26:47.137895 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:47.137870 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c35c9430-6adb-45bc-ac83-668b51f8e611-kube-api-access-l79qh" (OuterVolumeSpecName: "kube-api-access-l79qh") pod "c35c9430-6adb-45bc-ac83-668b51f8e611" (UID: "c35c9430-6adb-45bc-ac83-668b51f8e611"). InnerVolumeSpecName "kube-api-access-l79qh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:26:47.142523 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:47.142491 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b7cee0a-df79-414c-a597-f004df20451a-util" (OuterVolumeSpecName: "util") pod "7b7cee0a-df79-414c-a597-f004df20451a" (UID: "7b7cee0a-df79-414c-a597-f004df20451a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:26:47.144309 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:47.144270 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c35c9430-6adb-45bc-ac83-668b51f8e611-util" (OuterVolumeSpecName: "util") pod "c35c9430-6adb-45bc-ac83-668b51f8e611" (UID: "c35c9430-6adb-45bc-ac83-668b51f8e611"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:26:47.144652 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:47.144633 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25a4701d-6fa6-4415-9125-b6c8ecea61ef-util" (OuterVolumeSpecName: "util") pod "25a4701d-6fa6-4415-9125-b6c8ecea61ef" (UID: "25a4701d-6fa6-4415-9125-b6c8ecea61ef"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:26:47.236308 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:47.236279 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c35c9430-6adb-45bc-ac83-668b51f8e611-util\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:26:47.236308 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:47.236306 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/25a4701d-6fa6-4415-9125-b6c8ecea61ef-util\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:26:47.236308 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:47.236316 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c35c9430-6adb-45bc-ac83-668b51f8e611-bundle\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:26:47.236538 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:47.236325 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l79qh\" (UniqueName: \"kubernetes.io/projected/c35c9430-6adb-45bc-ac83-668b51f8e611-kube-api-access-l79qh\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:26:47.236538 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:47.236335 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7b7cee0a-df79-414c-a597-f004df20451a-util\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:26:47.236538 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:47.236344 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zgjmt\" (UniqueName: \"kubernetes.io/projected/7b7cee0a-df79-414c-a597-f004df20451a-kube-api-access-zgjmt\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:26:47.236538 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:47.236352 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nf96z\" (UniqueName: \"kubernetes.io/projected/25a4701d-6fa6-4415-9125-b6c8ecea61ef-kube-api-access-nf96z\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:26:47.236538 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:47.236362 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/25a4701d-6fa6-4415-9125-b6c8ecea61ef-bundle\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:26:47.236538 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:47.236371 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7b7cee0a-df79-414c-a597-f004df20451a-bundle\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:26:47.919029 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:47.918989 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sj8gs" event={"ID":"25a4701d-6fa6-4415-9125-b6c8ecea61ef","Type":"ContainerDied","Data":"8a06e91a8900a09cefbaf26c17a624fd75efbc9d6eec8844b054221c1c27e538"} Jul 05 13:26:47.919029 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:47.919029 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sj8gs" Jul 05 13:26:47.919493 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:47.919078 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8a06e91a8900a09cefbaf26c17a624fd75efbc9d6eec8844b054221c1c27e538" Jul 05 13:26:47.920906 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:47.920880 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zkc22" event={"ID":"7b7cee0a-df79-414c-a597-f004df20451a","Type":"ContainerDied","Data":"48678f97f90aee514c0ddce0cc02b2d8bb63fb600d8c84e55d0aaea7b4566b65"} Jul 05 13:26:47.921073 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:47.920921 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="48678f97f90aee514c0ddce0cc02b2d8bb63fb600d8c84e55d0aaea7b4566b65" Jul 05 13:26:47.921177 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:47.921157 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zkc22" Jul 05 13:26:47.922925 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:47.922891 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ll22v" event={"ID":"c35c9430-6adb-45bc-ac83-668b51f8e611","Type":"ContainerDied","Data":"66a9fa98595cf37657fbb5f35d8e40e81be719c4839f048e7b233450ced38c60"} Jul 05 13:26:47.923042 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:47.922922 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ll22v" Jul 05 13:26:47.923042 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:47.922930 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="66a9fa98595cf37657fbb5f35d8e40e81be719c4839f048e7b233450ced38c60" Jul 05 13:26:50.135805 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:50.135771 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:26:54.553561 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:54.553528 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:26:55.842976 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:55.842920 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:26:57.857614 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:57.857581 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:26:58.026196 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:26:58.026163 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:27:00.568187 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:00.568150 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:27:00.852767 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:00.852682 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:27:01.416884 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:01.416849 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:27:02.577348 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:02.577313 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:27:02.694446 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:02.694412 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:27:03.347000 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:03.346946 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:27:03.385559 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:03.385527 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-j5lkx"] Jul 05 13:27:03.385832 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:03.385821 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c35c9430-6adb-45bc-ac83-668b51f8e611" containerName="pull" Jul 05 13:27:03.385870 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:03.385834 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="c35c9430-6adb-45bc-ac83-668b51f8e611" containerName="pull" Jul 05 13:27:03.385870 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:03.385843 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c35c9430-6adb-45bc-ac83-668b51f8e611" containerName="extract" Jul 05 13:27:03.385870 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:03.385849 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="c35c9430-6adb-45bc-ac83-668b51f8e611" containerName="extract" Jul 05 13:27:03.385870 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:03.385856 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b7cee0a-df79-414c-a597-f004df20451a" containerName="extract" Jul 05 13:27:03.385870 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:03.385862 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b7cee0a-df79-414c-a597-f004df20451a" containerName="extract" Jul 05 13:27:03.385870 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:03.385871 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25a4701d-6fa6-4415-9125-b6c8ecea61ef" containerName="pull" Jul 05 13:27:03.386084 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:03.385876 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="25a4701d-6fa6-4415-9125-b6c8ecea61ef" containerName="pull" Jul 05 13:27:03.386084 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:03.385884 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8c4ff8d-80dc-40c8-b995-234066e407ac" containerName="util" Jul 05 13:27:03.386084 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:03.385890 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8c4ff8d-80dc-40c8-b995-234066e407ac" containerName="util" Jul 05 13:27:03.386084 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:03.385895 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b7cee0a-df79-414c-a597-f004df20451a" containerName="pull" Jul 05 13:27:03.386084 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:03.385900 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b7cee0a-df79-414c-a597-f004df20451a" containerName="pull" Jul 05 13:27:03.386084 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:03.385908 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8c4ff8d-80dc-40c8-b995-234066e407ac" containerName="pull" Jul 05 13:27:03.386084 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:03.385913 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8c4ff8d-80dc-40c8-b995-234066e407ac" containerName="pull" Jul 05 13:27:03.386084 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:03.385923 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25a4701d-6fa6-4415-9125-b6c8ecea61ef" containerName="util" Jul 05 13:27:03.386084 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:03.385928 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="25a4701d-6fa6-4415-9125-b6c8ecea61ef" containerName="util" Jul 05 13:27:03.386084 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:03.385937 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8c4ff8d-80dc-40c8-b995-234066e407ac" containerName="extract" Jul 05 13:27:03.386084 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:03.385943 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8c4ff8d-80dc-40c8-b995-234066e407ac" containerName="extract" Jul 05 13:27:03.386084 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:03.385966 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c35c9430-6adb-45bc-ac83-668b51f8e611" containerName="util" Jul 05 13:27:03.386084 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:03.385973 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="c35c9430-6adb-45bc-ac83-668b51f8e611" containerName="util" Jul 05 13:27:03.386084 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:03.385983 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b7cee0a-df79-414c-a597-f004df20451a" containerName="util" Jul 05 13:27:03.386084 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:03.385991 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b7cee0a-df79-414c-a597-f004df20451a" containerName="util" Jul 05 13:27:03.386084 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:03.385999 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25a4701d-6fa6-4415-9125-b6c8ecea61ef" containerName="extract" Jul 05 13:27:03.386084 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:03.386006 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="25a4701d-6fa6-4415-9125-b6c8ecea61ef" containerName="extract" Jul 05 13:27:03.386084 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:03.386070 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="c35c9430-6adb-45bc-ac83-668b51f8e611" containerName="extract" Jul 05 13:27:03.386084 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:03.386080 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b7cee0a-df79-414c-a597-f004df20451a" containerName="extract" Jul 05 13:27:03.386084 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:03.386087 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8c4ff8d-80dc-40c8-b995-234066e407ac" containerName="extract" Jul 05 13:27:03.386084 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:03.386093 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="25a4701d-6fa6-4415-9125-b6c8ecea61ef" containerName="extract" Jul 05 13:27:03.389012 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:03.388993 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-j5lkx" Jul 05 13:27:03.393575 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:03.393553 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 05 13:27:03.393741 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:03.393728 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 05 13:27:03.394378 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:03.394354 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-xlzrc\"" Jul 05 13:27:03.395079 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:03.395060 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 05 13:27:03.395406 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:03.395176 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 05 13:27:03.417670 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:03.417641 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-j5lkx"] Jul 05 13:27:03.480881 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:03.480858 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c574c726-dea7-45f6-86bb-84a22216d065-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-j5lkx\" (UID: \"c574c726-dea7-45f6-86bb-84a22216d065\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-j5lkx" Jul 05 13:27:03.481043 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:03.480888 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/c574c726-dea7-45f6-86bb-84a22216d065-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-j5lkx\" (UID: \"c574c726-dea7-45f6-86bb-84a22216d065\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-j5lkx" Jul 05 13:27:03.481043 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:03.480937 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrcbz\" (UniqueName: \"kubernetes.io/projected/c574c726-dea7-45f6-86bb-84a22216d065-kube-api-access-mrcbz\") pod \"kuadrant-console-plugin-66b5774985-j5lkx\" (UID: \"c574c726-dea7-45f6-86bb-84a22216d065\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-j5lkx" Jul 05 13:27:03.581930 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:03.581898 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c574c726-dea7-45f6-86bb-84a22216d065-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-j5lkx\" (UID: \"c574c726-dea7-45f6-86bb-84a22216d065\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-j5lkx" Jul 05 13:27:03.581930 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:03.581935 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/c574c726-dea7-45f6-86bb-84a22216d065-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-j5lkx\" (UID: \"c574c726-dea7-45f6-86bb-84a22216d065\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-j5lkx" Jul 05 13:27:03.582449 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:03.581994 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrcbz\" (UniqueName: \"kubernetes.io/projected/c574c726-dea7-45f6-86bb-84a22216d065-kube-api-access-mrcbz\") pod \"kuadrant-console-plugin-66b5774985-j5lkx\" (UID: \"c574c726-dea7-45f6-86bb-84a22216d065\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-j5lkx" Jul 05 13:27:03.582552 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:03.582532 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c574c726-dea7-45f6-86bb-84a22216d065-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-j5lkx\" (UID: \"c574c726-dea7-45f6-86bb-84a22216d065\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-j5lkx" Jul 05 13:27:03.584408 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:03.584384 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/c574c726-dea7-45f6-86bb-84a22216d065-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-j5lkx\" (UID: \"c574c726-dea7-45f6-86bb-84a22216d065\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-j5lkx" Jul 05 13:27:03.619445 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:03.619389 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrcbz\" (UniqueName: \"kubernetes.io/projected/c574c726-dea7-45f6-86bb-84a22216d065-kube-api-access-mrcbz\") pod \"kuadrant-console-plugin-66b5774985-j5lkx\" (UID: \"c574c726-dea7-45f6-86bb-84a22216d065\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-j5lkx" Jul 05 13:27:03.655074 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:03.655045 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:27:03.698148 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:03.698123 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-j5lkx" Jul 05 13:27:03.838242 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:03.838219 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-j5lkx"] Jul 05 13:27:03.840289 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:27:03.840264 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc574c726_dea7_45f6_86bb_84a22216d065.slice/crio-93322bddf22c6063bc7f9035e02590faff7f49df42f3e5a3b52f831950f174fc WatchSource:0}: Error finding container 93322bddf22c6063bc7f9035e02590faff7f49df42f3e5a3b52f831950f174fc: Status 404 returned error can't find the container with id 93322bddf22c6063bc7f9035e02590faff7f49df42f3e5a3b52f831950f174fc Jul 05 13:27:03.997791 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:03.997759 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-j5lkx" event={"ID":"c574c726-dea7-45f6-86bb-84a22216d065","Type":"ContainerStarted","Data":"93322bddf22c6063bc7f9035e02590faff7f49df42f3e5a3b52f831950f174fc"} Jul 05 13:27:05.858908 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:05.858870 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:27:06.013168 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:06.013114 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:27:11.092378 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:11.092335 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-98ccc5478-p4dhl" podUID="5ffe0cee-a0a5-47ce-8c1e-8c6412844909" containerName="console" containerID="cri-o://9a26532d0fe4c251808257e2413f79338d7bca0c933f1c0ccfcc3982893070aa" gracePeriod=15 Jul 05 13:27:12.054915 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:12.054884 2573 generic.go:356] "Generic (PLEG): container finished" podID="5ffe0cee-a0a5-47ce-8c1e-8c6412844909" containerID="9a26532d0fe4c251808257e2413f79338d7bca0c933f1c0ccfcc3982893070aa" exitCode=0 Jul 05 13:27:12.055103 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:12.054940 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-98ccc5478-p4dhl" event={"ID":"5ffe0cee-a0a5-47ce-8c1e-8c6412844909","Type":"ContainerDied","Data":"9a26532d0fe4c251808257e2413f79338d7bca0c933f1c0ccfcc3982893070aa"} Jul 05 13:27:14.414067 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:14.414029 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:27:14.512208 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:14.512176 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:27:18.223759 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:18.223703 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:27:18.330152 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:18.330117 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:27:18.904143 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:18.904095 2573 patch_prober.go:28] interesting pod/console-98ccc5478-p4dhl container/console namespace/openshift-console: Readiness probe status=failure output="Get \"https://10.134.0.23:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 05 13:27:18.904329 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:18.904174 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/console-98ccc5478-p4dhl" podUID="5ffe0cee-a0a5-47ce-8c1e-8c6412844909" containerName="console" probeResult="failure" output="Get \"https://10.134.0.23:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Jul 05 13:27:23.041240 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:23.041209 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:27:23.148289 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:23.148252 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:27:23.230313 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:23.230276 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:27:25.940457 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:25.940431 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-98ccc5478-p4dhl" Jul 05 13:27:25.993324 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:25.993291 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5ffe0cee-a0a5-47ce-8c1e-8c6412844909-console-config\") pod \"5ffe0cee-a0a5-47ce-8c1e-8c6412844909\" (UID: \"5ffe0cee-a0a5-47ce-8c1e-8c6412844909\") " Jul 05 13:27:25.993494 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:25.993349 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5ffe0cee-a0a5-47ce-8c1e-8c6412844909-service-ca\") pod \"5ffe0cee-a0a5-47ce-8c1e-8c6412844909\" (UID: \"5ffe0cee-a0a5-47ce-8c1e-8c6412844909\") " Jul 05 13:27:25.993494 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:25.993420 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5ffe0cee-a0a5-47ce-8c1e-8c6412844909-console-oauth-config\") pod \"5ffe0cee-a0a5-47ce-8c1e-8c6412844909\" (UID: \"5ffe0cee-a0a5-47ce-8c1e-8c6412844909\") " Jul 05 13:27:25.993494 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:25.993454 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ffe0cee-a0a5-47ce-8c1e-8c6412844909-trusted-ca-bundle\") pod \"5ffe0cee-a0a5-47ce-8c1e-8c6412844909\" (UID: \"5ffe0cee-a0a5-47ce-8c1e-8c6412844909\") " Jul 05 13:27:25.993494 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:25.993480 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tqwmm\" (UniqueName: \"kubernetes.io/projected/5ffe0cee-a0a5-47ce-8c1e-8c6412844909-kube-api-access-tqwmm\") pod \"5ffe0cee-a0a5-47ce-8c1e-8c6412844909\" (UID: \"5ffe0cee-a0a5-47ce-8c1e-8c6412844909\") " Jul 05 13:27:25.994119 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:25.993510 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5ffe0cee-a0a5-47ce-8c1e-8c6412844909-console-serving-cert\") pod \"5ffe0cee-a0a5-47ce-8c1e-8c6412844909\" (UID: \"5ffe0cee-a0a5-47ce-8c1e-8c6412844909\") " Jul 05 13:27:25.994119 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:25.993549 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5ffe0cee-a0a5-47ce-8c1e-8c6412844909-oauth-serving-cert\") pod \"5ffe0cee-a0a5-47ce-8c1e-8c6412844909\" (UID: \"5ffe0cee-a0a5-47ce-8c1e-8c6412844909\") " Jul 05 13:27:25.994119 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:25.993729 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ffe0cee-a0a5-47ce-8c1e-8c6412844909-console-config" (OuterVolumeSpecName: "console-config") pod "5ffe0cee-a0a5-47ce-8c1e-8c6412844909" (UID: "5ffe0cee-a0a5-47ce-8c1e-8c6412844909"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:27:25.994119 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:25.993795 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ffe0cee-a0a5-47ce-8c1e-8c6412844909-service-ca" (OuterVolumeSpecName: "service-ca") pod "5ffe0cee-a0a5-47ce-8c1e-8c6412844909" (UID: "5ffe0cee-a0a5-47ce-8c1e-8c6412844909"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:27:25.994331 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:25.994128 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ffe0cee-a0a5-47ce-8c1e-8c6412844909-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "5ffe0cee-a0a5-47ce-8c1e-8c6412844909" (UID: "5ffe0cee-a0a5-47ce-8c1e-8c6412844909"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:27:25.994331 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:25.994229 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ffe0cee-a0a5-47ce-8c1e-8c6412844909-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5ffe0cee-a0a5-47ce-8c1e-8c6412844909" (UID: "5ffe0cee-a0a5-47ce-8c1e-8c6412844909"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:27:25.995712 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:25.995690 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ffe0cee-a0a5-47ce-8c1e-8c6412844909-kube-api-access-tqwmm" (OuterVolumeSpecName: "kube-api-access-tqwmm") pod "5ffe0cee-a0a5-47ce-8c1e-8c6412844909" (UID: "5ffe0cee-a0a5-47ce-8c1e-8c6412844909"). InnerVolumeSpecName "kube-api-access-tqwmm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:27:25.995985 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:25.995936 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ffe0cee-a0a5-47ce-8c1e-8c6412844909-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5ffe0cee-a0a5-47ce-8c1e-8c6412844909" (UID: "5ffe0cee-a0a5-47ce-8c1e-8c6412844909"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:27:25.996065 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:25.996017 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ffe0cee-a0a5-47ce-8c1e-8c6412844909-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5ffe0cee-a0a5-47ce-8c1e-8c6412844909" (UID: "5ffe0cee-a0a5-47ce-8c1e-8c6412844909"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:27:26.094920 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:26.094885 2573 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5ffe0cee-a0a5-47ce-8c1e-8c6412844909-service-ca\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:27:26.094920 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:26.094914 2573 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5ffe0cee-a0a5-47ce-8c1e-8c6412844909-console-oauth-config\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:27:26.094920 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:26.094924 2573 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ffe0cee-a0a5-47ce-8c1e-8c6412844909-trusted-ca-bundle\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:27:26.095178 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:26.094932 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tqwmm\" (UniqueName: \"kubernetes.io/projected/5ffe0cee-a0a5-47ce-8c1e-8c6412844909-kube-api-access-tqwmm\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:27:26.095178 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:26.094941 2573 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5ffe0cee-a0a5-47ce-8c1e-8c6412844909-console-serving-cert\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:27:26.095178 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:26.094968 2573 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5ffe0cee-a0a5-47ce-8c1e-8c6412844909-oauth-serving-cert\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:27:26.095178 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:26.094977 2573 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5ffe0cee-a0a5-47ce-8c1e-8c6412844909-console-config\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:27:26.134662 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:26.134631 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-98ccc5478-p4dhl" Jul 05 13:27:26.134662 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:26.134639 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-98ccc5478-p4dhl" event={"ID":"5ffe0cee-a0a5-47ce-8c1e-8c6412844909","Type":"ContainerDied","Data":"f3d169e5c2cbd66f3fb69439e8570d8e9873caf7c268533227af1063be6fefa2"} Jul 05 13:27:26.134849 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:26.134691 2573 scope.go:117] "RemoveContainer" containerID="9a26532d0fe4c251808257e2413f79338d7bca0c933f1c0ccfcc3982893070aa" Jul 05 13:27:26.170105 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:26.170045 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-98ccc5478-p4dhl"] Jul 05 13:27:26.173264 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:26.173242 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-98ccc5478-p4dhl"] Jul 05 13:27:27.141615 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:27.141579 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-j5lkx" event={"ID":"c574c726-dea7-45f6-86bb-84a22216d065","Type":"ContainerStarted","Data":"f9e5d02c8595d7235369eb74667b591cc5f391a40af06dba49cc01bc0fd6266f"} Jul 05 13:27:27.176375 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:27.176319 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-j5lkx" podStartSLOduration=1.301152072 podStartE2EDuration="24.176301805s" totalImagesPullingTime="22.875149733s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:27:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:27:27.175098984 +0000 UTC m=+520.070757072" watchObservedRunningTime="2026-07-05 13:27:27.176301805 +0000 UTC m=+520.071959894" Jul 05 13:27:27.277228 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:27.277196 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:27:27.377479 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:27.377442 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:27:27.735586 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:27.735555 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ffe0cee-a0a5-47ce-8c1e-8c6412844909" path="/var/lib/kubelet/pods/5ffe0cee-a0a5-47ce-8c1e-8c6412844909/volumes" Jul 05 13:27:33.479940 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:33.479882 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:27:33.568198 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:33.568156 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:27:38.178467 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:38.178432 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:27:38.287498 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:38.287462 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:27:41.447153 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:41.447066 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:27:41.939446 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:41.939413 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:27:42.977447 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:42.977411 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:27:43.073617 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:43.073584 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:27:44.195501 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:44.195468 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:27:44.283564 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:44.283529 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:27:44.391333 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:44.391305 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:27:44.684200 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:44.684166 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:27:44.684498 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:44.684481 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ffe0cee-a0a5-47ce-8c1e-8c6412844909" containerName="console" Jul 05 13:27:44.684544 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:44.684502 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ffe0cee-a0a5-47ce-8c1e-8c6412844909" containerName="console" Jul 05 13:27:44.684608 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:44.684596 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ffe0cee-a0a5-47ce-8c1e-8c6412844909" containerName="console" Jul 05 13:27:44.687601 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:44.687583 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-wzt5q" Jul 05 13:27:44.690604 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:44.690574 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 05 13:27:44.699106 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:44.699080 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:27:44.730737 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:44.730717 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:27:44.747337 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:44.747319 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2273f7be-a201-45c8-8e69-232ea0156608-config-file\") pod \"limitador-limitador-54d6d988d5-wzt5q\" (UID: \"2273f7be-a201-45c8-8e69-232ea0156608\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-wzt5q" Jul 05 13:27:44.747418 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:44.747370 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czb7h\" (UniqueName: \"kubernetes.io/projected/2273f7be-a201-45c8-8e69-232ea0156608-kube-api-access-czb7h\") pod \"limitador-limitador-54d6d988d5-wzt5q\" (UID: \"2273f7be-a201-45c8-8e69-232ea0156608\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-wzt5q" Jul 05 13:27:44.848135 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:44.848095 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czb7h\" (UniqueName: \"kubernetes.io/projected/2273f7be-a201-45c8-8e69-232ea0156608-kube-api-access-czb7h\") pod \"limitador-limitador-54d6d988d5-wzt5q\" (UID: \"2273f7be-a201-45c8-8e69-232ea0156608\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-wzt5q" Jul 05 13:27:44.848316 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:44.848167 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2273f7be-a201-45c8-8e69-232ea0156608-config-file\") pod \"limitador-limitador-54d6d988d5-wzt5q\" (UID: \"2273f7be-a201-45c8-8e69-232ea0156608\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-wzt5q" Jul 05 13:27:44.848885 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:44.848863 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2273f7be-a201-45c8-8e69-232ea0156608-config-file\") pod \"limitador-limitador-54d6d988d5-wzt5q\" (UID: \"2273f7be-a201-45c8-8e69-232ea0156608\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-wzt5q" Jul 05 13:27:44.864111 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:44.864080 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-czb7h\" (UniqueName: \"kubernetes.io/projected/2273f7be-a201-45c8-8e69-232ea0156608-kube-api-access-czb7h\") pod \"limitador-limitador-54d6d988d5-wzt5q\" (UID: \"2273f7be-a201-45c8-8e69-232ea0156608\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-wzt5q" Jul 05 13:27:44.998168 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:44.998084 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-wzt5q" Jul 05 13:27:45.084697 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:45.084666 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-pl8t7"] Jul 05 13:27:45.088276 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:45.088252 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-pl8t7" Jul 05 13:27:45.091415 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:45.091368 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-vvdx7\"" Jul 05 13:27:45.104269 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:45.104239 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-pl8t7"] Jul 05 13:27:45.133442 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:45.133414 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:27:45.144034 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:45.144014 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:27:45.146235 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:27:45.146208 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2273f7be_a201_45c8_8e69_232ea0156608.slice/crio-40ebdb1e760b40dd117dde3d6167dd6409ca951add205d3c1e3e43b4936adad5 WatchSource:0}: Error finding container 40ebdb1e760b40dd117dde3d6167dd6409ca951add205d3c1e3e43b4936adad5: Status 404 returned error can't find the container with id 40ebdb1e760b40dd117dde3d6167dd6409ca951add205d3c1e3e43b4936adad5 Jul 05 13:27:45.151878 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:45.151839 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfcjj\" (UniqueName: \"kubernetes.io/projected/18504fdb-e27c-4c0d-8cc9-243ed037831f-kube-api-access-xfcjj\") pod \"authorino-58cd955675-pl8t7\" (UID: \"18504fdb-e27c-4c0d-8cc9-243ed037831f\") " pod="kuadrant-system/authorino-58cd955675-pl8t7" Jul 05 13:27:45.193109 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:45.193079 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:27:45.225076 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:45.225044 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-wzt5q" event={"ID":"2273f7be-a201-45c8-8e69-232ea0156608","Type":"ContainerStarted","Data":"40ebdb1e760b40dd117dde3d6167dd6409ca951add205d3c1e3e43b4936adad5"} Jul 05 13:27:45.252488 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:45.252416 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfcjj\" (UniqueName: \"kubernetes.io/projected/18504fdb-e27c-4c0d-8cc9-243ed037831f-kube-api-access-xfcjj\") pod \"authorino-58cd955675-pl8t7\" (UID: \"18504fdb-e27c-4c0d-8cc9-243ed037831f\") " pod="kuadrant-system/authorino-58cd955675-pl8t7" Jul 05 13:27:45.260852 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:45.260826 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfcjj\" (UniqueName: \"kubernetes.io/projected/18504fdb-e27c-4c0d-8cc9-243ed037831f-kube-api-access-xfcjj\") pod \"authorino-58cd955675-pl8t7\" (UID: \"18504fdb-e27c-4c0d-8cc9-243ed037831f\") " pod="kuadrant-system/authorino-58cd955675-pl8t7" Jul 05 13:27:45.398969 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:45.398915 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-pl8t7" Jul 05 13:27:45.454600 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:45.454572 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-fkz5b"] Jul 05 13:27:45.458720 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:45.458699 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-fkz5b" Jul 05 13:27:45.467021 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:45.466722 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-fkz5b"] Jul 05 13:27:45.533180 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:45.533126 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-pl8t7"] Jul 05 13:27:45.536026 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:27:45.535997 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod18504fdb_e27c_4c0d_8cc9_243ed037831f.slice/crio-0b9a033dd026c3569d579b9946657604920bb1c7fd998475eca8720f5847d57d WatchSource:0}: Error finding container 0b9a033dd026c3569d579b9946657604920bb1c7fd998475eca8720f5847d57d: Status 404 returned error can't find the container with id 0b9a033dd026c3569d579b9946657604920bb1c7fd998475eca8720f5847d57d Jul 05 13:27:45.555846 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:45.555818 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbjs6\" (UniqueName: \"kubernetes.io/projected/fd1129ca-34e5-464a-8c73-48c5b911683d-kube-api-access-vbjs6\") pod \"authorino-75d556bd77-fkz5b\" (UID: \"fd1129ca-34e5-464a-8c73-48c5b911683d\") " pod="kuadrant-system/authorino-75d556bd77-fkz5b" Jul 05 13:27:45.657103 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:45.656669 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbjs6\" (UniqueName: \"kubernetes.io/projected/fd1129ca-34e5-464a-8c73-48c5b911683d-kube-api-access-vbjs6\") pod \"authorino-75d556bd77-fkz5b\" (UID: \"fd1129ca-34e5-464a-8c73-48c5b911683d\") " pod="kuadrant-system/authorino-75d556bd77-fkz5b" Jul 05 13:27:45.670726 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:45.670670 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbjs6\" (UniqueName: \"kubernetes.io/projected/fd1129ca-34e5-464a-8c73-48c5b911683d-kube-api-access-vbjs6\") pod \"authorino-75d556bd77-fkz5b\" (UID: \"fd1129ca-34e5-464a-8c73-48c5b911683d\") " pod="kuadrant-system/authorino-75d556bd77-fkz5b" Jul 05 13:27:45.770282 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:45.770023 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-fkz5b" Jul 05 13:27:45.952214 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:45.952170 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-fkz5b"] Jul 05 13:27:45.956067 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:27:45.956023 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfd1129ca_34e5_464a_8c73_48c5b911683d.slice/crio-207e839147a89bd28e88fe0adafbb0a7cf6b97f5befe90e83a8302aee4bf9a8a WatchSource:0}: Error finding container 207e839147a89bd28e88fe0adafbb0a7cf6b97f5befe90e83a8302aee4bf9a8a: Status 404 returned error can't find the container with id 207e839147a89bd28e88fe0adafbb0a7cf6b97f5befe90e83a8302aee4bf9a8a Jul 05 13:27:46.234204 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:46.234168 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-pl8t7" event={"ID":"18504fdb-e27c-4c0d-8cc9-243ed037831f","Type":"ContainerStarted","Data":"0b9a033dd026c3569d579b9946657604920bb1c7fd998475eca8720f5847d57d"} Jul 05 13:27:46.235719 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:46.235692 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-fkz5b" event={"ID":"fd1129ca-34e5-464a-8c73-48c5b911683d","Type":"ContainerStarted","Data":"207e839147a89bd28e88fe0adafbb0a7cf6b97f5befe90e83a8302aee4bf9a8a"} Jul 05 13:27:47.308382 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:47.308309 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:27:47.387121 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:47.387080 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:27:51.270756 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:51.270724 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-wzt5q" event={"ID":"2273f7be-a201-45c8-8e69-232ea0156608","Type":"ContainerStarted","Data":"aafee02f62e6fa23e1b43aac40a7d84b913d3653b63e1f409f99661e1746db72"} Jul 05 13:27:51.271232 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:51.270850 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-wzt5q" Jul 05 13:27:51.272313 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:51.272283 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-pl8t7" event={"ID":"18504fdb-e27c-4c0d-8cc9-243ed037831f","Type":"ContainerStarted","Data":"81efe387365927883fc947c6e2b9720b7bece52801a24527dc5f3379895959ab"} Jul 05 13:27:51.273604 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:51.273581 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-fkz5b" event={"ID":"fd1129ca-34e5-464a-8c73-48c5b911683d","Type":"ContainerStarted","Data":"9fff0cc4898992728bc6fa709e35281a5f9b3d582494007e0f1a4025a18e36c2"} Jul 05 13:27:51.291253 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:51.291212 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-wzt5q" podStartSLOduration=1.700339927 podStartE2EDuration="7.291199683s" totalImagesPullingTime="5.590859756s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:27:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:27:51.28977386 +0000 UTC m=+544.185431952" watchObservedRunningTime="2026-07-05 13:27:51.291199683 +0000 UTC m=+544.186857772" Jul 05 13:27:51.303658 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:51.303612 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-pl8t7" podStartSLOduration=1.110555888 podStartE2EDuration="6.303596602s" totalImagesPullingTime="5.193040714s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:27:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:27:51.303572152 +0000 UTC m=+544.199230247" watchObservedRunningTime="2026-07-05 13:27:51.303596602 +0000 UTC m=+544.199254692" Jul 05 13:27:51.318623 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:51.318572 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-fkz5b" podStartSLOduration=1.5757583670000002 podStartE2EDuration="6.318553273s" totalImagesPullingTime="4.742794906s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:27:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:27:51.316373285 +0000 UTC m=+544.212031375" watchObservedRunningTime="2026-07-05 13:27:51.318553273 +0000 UTC m=+544.214211362" Jul 05 13:27:51.341130 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:51.341097 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-pl8t7"] Jul 05 13:27:53.283266 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:53.283222 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-pl8t7" podUID="18504fdb-e27c-4c0d-8cc9-243ed037831f" containerName="authorino" containerID="cri-o://81efe387365927883fc947c6e2b9720b7bece52801a24527dc5f3379895959ab" gracePeriod=30 Jul 05 13:27:53.531213 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:53.531189 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-pl8t7" Jul 05 13:27:53.632877 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:53.632793 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xfcjj\" (UniqueName: \"kubernetes.io/projected/18504fdb-e27c-4c0d-8cc9-243ed037831f-kube-api-access-xfcjj\") pod \"18504fdb-e27c-4c0d-8cc9-243ed037831f\" (UID: \"18504fdb-e27c-4c0d-8cc9-243ed037831f\") " Jul 05 13:27:53.634824 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:53.634793 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18504fdb-e27c-4c0d-8cc9-243ed037831f-kube-api-access-xfcjj" (OuterVolumeSpecName: "kube-api-access-xfcjj") pod "18504fdb-e27c-4c0d-8cc9-243ed037831f" (UID: "18504fdb-e27c-4c0d-8cc9-243ed037831f"). InnerVolumeSpecName "kube-api-access-xfcjj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:27:53.733364 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:53.733333 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xfcjj\" (UniqueName: \"kubernetes.io/projected/18504fdb-e27c-4c0d-8cc9-243ed037831f-kube-api-access-xfcjj\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:27:54.289192 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:54.289156 2573 generic.go:356] "Generic (PLEG): container finished" podID="18504fdb-e27c-4c0d-8cc9-243ed037831f" containerID="81efe387365927883fc947c6e2b9720b7bece52801a24527dc5f3379895959ab" exitCode=0 Jul 05 13:27:54.289639 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:54.289199 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-pl8t7" Jul 05 13:27:54.289639 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:54.289224 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-pl8t7" event={"ID":"18504fdb-e27c-4c0d-8cc9-243ed037831f","Type":"ContainerDied","Data":"81efe387365927883fc947c6e2b9720b7bece52801a24527dc5f3379895959ab"} Jul 05 13:27:54.289639 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:54.289255 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-pl8t7" event={"ID":"18504fdb-e27c-4c0d-8cc9-243ed037831f","Type":"ContainerDied","Data":"0b9a033dd026c3569d579b9946657604920bb1c7fd998475eca8720f5847d57d"} Jul 05 13:27:54.289639 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:54.289272 2573 scope.go:117] "RemoveContainer" containerID="81efe387365927883fc947c6e2b9720b7bece52801a24527dc5f3379895959ab" Jul 05 13:27:54.297530 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:54.297513 2573 scope.go:117] "RemoveContainer" containerID="81efe387365927883fc947c6e2b9720b7bece52801a24527dc5f3379895959ab" Jul 05 13:27:54.297776 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:27:54.297758 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81efe387365927883fc947c6e2b9720b7bece52801a24527dc5f3379895959ab\": container with ID starting with 81efe387365927883fc947c6e2b9720b7bece52801a24527dc5f3379895959ab not found: ID does not exist" containerID="81efe387365927883fc947c6e2b9720b7bece52801a24527dc5f3379895959ab" Jul 05 13:27:54.297838 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:54.297783 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81efe387365927883fc947c6e2b9720b7bece52801a24527dc5f3379895959ab"} err="failed to get container status \"81efe387365927883fc947c6e2b9720b7bece52801a24527dc5f3379895959ab\": rpc error: code = NotFound desc = could not find container \"81efe387365927883fc947c6e2b9720b7bece52801a24527dc5f3379895959ab\": container with ID starting with 81efe387365927883fc947c6e2b9720b7bece52801a24527dc5f3379895959ab not found: ID does not exist" Jul 05 13:27:54.305431 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:54.305408 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-pl8t7"] Jul 05 13:27:54.308876 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:54.308856 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-pl8t7"] Jul 05 13:27:55.735271 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:27:55.735238 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18504fdb-e27c-4c0d-8cc9-243ed037831f" path="/var/lib/kubelet/pods/18504fdb-e27c-4c0d-8cc9-243ed037831f/volumes" Jul 05 13:28:02.280476 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:02.280447 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-wzt5q" Jul 05 13:28:05.790226 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:05.790176 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:28:06.353782 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:06.353736 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:28:06.409429 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:06.409400 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:28:17.841289 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:17.841244 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:28:18.083711 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:18.083664 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:28:18.114130 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:18.114054 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-zwscc"] Jul 05 13:28:18.114381 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:18.114369 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18504fdb-e27c-4c0d-8cc9-243ed037831f" containerName="authorino" Jul 05 13:28:18.114424 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:18.114383 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="18504fdb-e27c-4c0d-8cc9-243ed037831f" containerName="authorino" Jul 05 13:28:18.114462 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:18.114447 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="18504fdb-e27c-4c0d-8cc9-243ed037831f" containerName="authorino" Jul 05 13:28:18.123911 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:18.123880 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-zwscc" Jul 05 13:28:18.126774 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:18.126746 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-zwscc"] Jul 05 13:28:18.242759 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:18.242720 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qn8t\" (UniqueName: \"kubernetes.io/projected/78bac666-fe41-4fad-be3e-bcbbbe447891-kube-api-access-6qn8t\") pod \"authorino-5d4b67c446-zwscc\" (UID: \"78bac666-fe41-4fad-be3e-bcbbbe447891\") " pod="kuadrant-system/authorino-5d4b67c446-zwscc" Jul 05 13:28:18.284034 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:18.283996 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-zwscc"] Jul 05 13:28:18.284271 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:28:18.284250 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-6qn8t], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-zwscc" podUID="78bac666-fe41-4fad-be3e-bcbbbe447891" Jul 05 13:28:18.314683 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:18.314648 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-76877994-s2vfq"] Jul 05 13:28:18.320704 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:18.320682 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-s2vfq" Jul 05 13:28:18.323198 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:18.323176 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 05 13:28:18.328452 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:18.328432 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76877994-s2vfq"] Jul 05 13:28:18.343730 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:18.343704 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qn8t\" (UniqueName: \"kubernetes.io/projected/78bac666-fe41-4fad-be3e-bcbbbe447891-kube-api-access-6qn8t\") pod \"authorino-5d4b67c446-zwscc\" (UID: \"78bac666-fe41-4fad-be3e-bcbbbe447891\") " pod="kuadrant-system/authorino-5d4b67c446-zwscc" Jul 05 13:28:18.343867 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:18.343745 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a9006f6a-b3fd-4b4a-a027-36749afa3d60-tls-cert\") pod \"authorino-76877994-s2vfq\" (UID: \"a9006f6a-b3fd-4b4a-a027-36749afa3d60\") " pod="kuadrant-system/authorino-76877994-s2vfq" Jul 05 13:28:18.343867 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:18.343765 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhnkp\" (UniqueName: \"kubernetes.io/projected/a9006f6a-b3fd-4b4a-a027-36749afa3d60-kube-api-access-lhnkp\") pod \"authorino-76877994-s2vfq\" (UID: \"a9006f6a-b3fd-4b4a-a027-36749afa3d60\") " pod="kuadrant-system/authorino-76877994-s2vfq" Jul 05 13:28:18.353029 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:18.353003 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qn8t\" (UniqueName: \"kubernetes.io/projected/78bac666-fe41-4fad-be3e-bcbbbe447891-kube-api-access-6qn8t\") pod \"authorino-5d4b67c446-zwscc\" (UID: \"78bac666-fe41-4fad-be3e-bcbbbe447891\") " pod="kuadrant-system/authorino-5d4b67c446-zwscc" Jul 05 13:28:18.390444 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:18.390384 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-s2vfq"] Jul 05 13:28:18.390635 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:28:18.390617 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-lhnkp tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-76877994-s2vfq" podUID="a9006f6a-b3fd-4b4a-a027-36749afa3d60" Jul 05 13:28:18.406457 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:18.406433 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-s2vfq" Jul 05 13:28:18.406566 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:18.406434 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-zwscc" Jul 05 13:28:18.420981 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:18.420938 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5554dc9854-8rtzw"] Jul 05 13:28:18.443579 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:18.443547 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-s2vfq" Jul 05 13:28:18.444441 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:18.444419 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a9006f6a-b3fd-4b4a-a027-36749afa3d60-tls-cert\") pod \"authorino-76877994-s2vfq\" (UID: \"a9006f6a-b3fd-4b4a-a027-36749afa3d60\") " pod="kuadrant-system/authorino-76877994-s2vfq" Jul 05 13:28:18.444506 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:18.444452 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhnkp\" (UniqueName: \"kubernetes.io/projected/a9006f6a-b3fd-4b4a-a027-36749afa3d60-kube-api-access-lhnkp\") pod \"authorino-76877994-s2vfq\" (UID: \"a9006f6a-b3fd-4b4a-a027-36749afa3d60\") " pod="kuadrant-system/authorino-76877994-s2vfq" Jul 05 13:28:18.447105 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:18.447082 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a9006f6a-b3fd-4b4a-a027-36749afa3d60-tls-cert\") pod \"authorino-76877994-s2vfq\" (UID: \"a9006f6a-b3fd-4b4a-a027-36749afa3d60\") " pod="kuadrant-system/authorino-76877994-s2vfq" Jul 05 13:28:18.451417 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:18.451394 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5554dc9854-8rtzw"] Jul 05 13:28:18.451513 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:18.451472 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5554dc9854-8rtzw" Jul 05 13:28:18.451637 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:18.451615 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhnkp\" (UniqueName: \"kubernetes.io/projected/a9006f6a-b3fd-4b4a-a027-36749afa3d60-kube-api-access-lhnkp\") pod \"authorino-76877994-s2vfq\" (UID: \"a9006f6a-b3fd-4b4a-a027-36749afa3d60\") " pod="kuadrant-system/authorino-76877994-s2vfq" Jul 05 13:28:18.451802 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:18.451787 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-zwscc" Jul 05 13:28:18.545118 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:18.545094 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a9006f6a-b3fd-4b4a-a027-36749afa3d60-tls-cert\") pod \"a9006f6a-b3fd-4b4a-a027-36749afa3d60\" (UID: \"a9006f6a-b3fd-4b4a-a027-36749afa3d60\") " Jul 05 13:28:18.545236 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:18.545161 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lhnkp\" (UniqueName: \"kubernetes.io/projected/a9006f6a-b3fd-4b4a-a027-36749afa3d60-kube-api-access-lhnkp\") pod \"a9006f6a-b3fd-4b4a-a027-36749afa3d60\" (UID: \"a9006f6a-b3fd-4b4a-a027-36749afa3d60\") " Jul 05 13:28:18.545236 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:18.545212 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6qn8t\" (UniqueName: \"kubernetes.io/projected/78bac666-fe41-4fad-be3e-bcbbbe447891-kube-api-access-6qn8t\") pod \"78bac666-fe41-4fad-be3e-bcbbbe447891\" (UID: \"78bac666-fe41-4fad-be3e-bcbbbe447891\") " Jul 05 13:28:18.545364 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:18.545345 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1b206266-6d09-425a-b979-2daa7fb1f516-tls-cert\") pod \"authorino-5554dc9854-8rtzw\" (UID: \"1b206266-6d09-425a-b979-2daa7fb1f516\") " pod="kuadrant-system/authorino-5554dc9854-8rtzw" Jul 05 13:28:18.545416 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:18.545402 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbhzk\" (UniqueName: \"kubernetes.io/projected/1b206266-6d09-425a-b979-2daa7fb1f516-kube-api-access-gbhzk\") pod \"authorino-5554dc9854-8rtzw\" (UID: \"1b206266-6d09-425a-b979-2daa7fb1f516\") " pod="kuadrant-system/authorino-5554dc9854-8rtzw" Jul 05 13:28:18.547219 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:18.547191 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9006f6a-b3fd-4b4a-a027-36749afa3d60-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "a9006f6a-b3fd-4b4a-a027-36749afa3d60" (UID: "a9006f6a-b3fd-4b4a-a027-36749afa3d60"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:28:18.547317 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:18.547224 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78bac666-fe41-4fad-be3e-bcbbbe447891-kube-api-access-6qn8t" (OuterVolumeSpecName: "kube-api-access-6qn8t") pod "78bac666-fe41-4fad-be3e-bcbbbe447891" (UID: "78bac666-fe41-4fad-be3e-bcbbbe447891"). InnerVolumeSpecName "kube-api-access-6qn8t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:28:18.547317 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:18.547265 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9006f6a-b3fd-4b4a-a027-36749afa3d60-kube-api-access-lhnkp" (OuterVolumeSpecName: "kube-api-access-lhnkp") pod "a9006f6a-b3fd-4b4a-a027-36749afa3d60" (UID: "a9006f6a-b3fd-4b4a-a027-36749afa3d60"). InnerVolumeSpecName "kube-api-access-lhnkp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:28:18.646703 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:18.646624 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1b206266-6d09-425a-b979-2daa7fb1f516-tls-cert\") pod \"authorino-5554dc9854-8rtzw\" (UID: \"1b206266-6d09-425a-b979-2daa7fb1f516\") " pod="kuadrant-system/authorino-5554dc9854-8rtzw" Jul 05 13:28:18.646703 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:18.646669 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbhzk\" (UniqueName: \"kubernetes.io/projected/1b206266-6d09-425a-b979-2daa7fb1f516-kube-api-access-gbhzk\") pod \"authorino-5554dc9854-8rtzw\" (UID: \"1b206266-6d09-425a-b979-2daa7fb1f516\") " pod="kuadrant-system/authorino-5554dc9854-8rtzw" Jul 05 13:28:18.646703 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:18.646699 2573 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a9006f6a-b3fd-4b4a-a027-36749afa3d60-tls-cert\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:28:18.646926 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:18.646715 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lhnkp\" (UniqueName: \"kubernetes.io/projected/a9006f6a-b3fd-4b4a-a027-36749afa3d60-kube-api-access-lhnkp\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:28:18.646926 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:18.646725 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6qn8t\" (UniqueName: \"kubernetes.io/projected/78bac666-fe41-4fad-be3e-bcbbbe447891-kube-api-access-6qn8t\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:28:18.649227 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:18.649197 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1b206266-6d09-425a-b979-2daa7fb1f516-tls-cert\") pod \"authorino-5554dc9854-8rtzw\" (UID: \"1b206266-6d09-425a-b979-2daa7fb1f516\") " pod="kuadrant-system/authorino-5554dc9854-8rtzw" Jul 05 13:28:18.655626 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:18.655606 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbhzk\" (UniqueName: \"kubernetes.io/projected/1b206266-6d09-425a-b979-2daa7fb1f516-kube-api-access-gbhzk\") pod \"authorino-5554dc9854-8rtzw\" (UID: \"1b206266-6d09-425a-b979-2daa7fb1f516\") " pod="kuadrant-system/authorino-5554dc9854-8rtzw" Jul 05 13:28:18.769008 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:18.768984 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5554dc9854-8rtzw" Jul 05 13:28:18.895568 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:18.895541 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5554dc9854-8rtzw"] Jul 05 13:28:18.897620 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:28:18.897563 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1b206266_6d09_425a_b979_2daa7fb1f516.slice/crio-04cc9f531cca30eb30433534ec65611fc1d8f78e03383281d548af458681930c WatchSource:0}: Error finding container 04cc9f531cca30eb30433534ec65611fc1d8f78e03383281d548af458681930c: Status 404 returned error can't find the container with id 04cc9f531cca30eb30433534ec65611fc1d8f78e03383281d548af458681930c Jul 05 13:28:19.413095 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:19.413051 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5554dc9854-8rtzw" event={"ID":"1b206266-6d09-425a-b979-2daa7fb1f516","Type":"ContainerStarted","Data":"4a253b9357470c0435f13aa45787e8f8fad8bdfe5530ebc7f6b70e0f26b0b28e"} Jul 05 13:28:19.413095 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:19.413080 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-s2vfq" Jul 05 13:28:19.413305 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:19.413104 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5554dc9854-8rtzw" event={"ID":"1b206266-6d09-425a-b979-2daa7fb1f516","Type":"ContainerStarted","Data":"04cc9f531cca30eb30433534ec65611fc1d8f78e03383281d548af458681930c"} Jul 05 13:28:19.413305 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:19.413064 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-zwscc" Jul 05 13:28:19.431714 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:19.431636 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5554dc9854-8rtzw" podStartSLOduration=1.150869552 podStartE2EDuration="1.431621692s" totalImagesPullingTime="280.75214ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:28:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:28:19.42963182 +0000 UTC m=+572.325289910" watchObservedRunningTime="2026-07-05 13:28:19.431621692 +0000 UTC m=+572.327279781" Jul 05 13:28:19.453816 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:19.453790 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-zwscc"] Jul 05 13:28:19.458422 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:19.458399 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-fkz5b"] Jul 05 13:28:19.458917 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:19.458726 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-fkz5b" podUID="fd1129ca-34e5-464a-8c73-48c5b911683d" containerName="authorino" containerID="cri-o://9fff0cc4898992728bc6fa709e35281a5f9b3d582494007e0f1a4025a18e36c2" gracePeriod=30 Jul 05 13:28:19.461618 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:19.461591 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-zwscc"] Jul 05 13:28:19.485495 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:19.485464 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-s2vfq"] Jul 05 13:28:19.487476 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:19.487455 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-76877994-s2vfq"] Jul 05 13:28:19.719776 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:19.719750 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-fkz5b" Jul 05 13:28:19.736309 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:19.736281 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78bac666-fe41-4fad-be3e-bcbbbe447891" path="/var/lib/kubelet/pods/78bac666-fe41-4fad-be3e-bcbbbe447891/volumes" Jul 05 13:28:19.736531 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:19.736519 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9006f6a-b3fd-4b4a-a027-36749afa3d60" path="/var/lib/kubelet/pods/a9006f6a-b3fd-4b4a-a027-36749afa3d60/volumes" Jul 05 13:28:19.758559 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:19.758520 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vbjs6\" (UniqueName: \"kubernetes.io/projected/fd1129ca-34e5-464a-8c73-48c5b911683d-kube-api-access-vbjs6\") pod \"fd1129ca-34e5-464a-8c73-48c5b911683d\" (UID: \"fd1129ca-34e5-464a-8c73-48c5b911683d\") " Jul 05 13:28:19.760540 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:19.760514 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd1129ca-34e5-464a-8c73-48c5b911683d-kube-api-access-vbjs6" (OuterVolumeSpecName: "kube-api-access-vbjs6") pod "fd1129ca-34e5-464a-8c73-48c5b911683d" (UID: "fd1129ca-34e5-464a-8c73-48c5b911683d"). InnerVolumeSpecName "kube-api-access-vbjs6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:28:19.860128 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:19.860088 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vbjs6\" (UniqueName: \"kubernetes.io/projected/fd1129ca-34e5-464a-8c73-48c5b911683d-kube-api-access-vbjs6\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:28:20.419474 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:20.419437 2573 generic.go:356] "Generic (PLEG): container finished" podID="fd1129ca-34e5-464a-8c73-48c5b911683d" containerID="9fff0cc4898992728bc6fa709e35281a5f9b3d582494007e0f1a4025a18e36c2" exitCode=0 Jul 05 13:28:20.419897 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:20.419482 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-fkz5b" Jul 05 13:28:20.419897 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:20.419529 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-fkz5b" event={"ID":"fd1129ca-34e5-464a-8c73-48c5b911683d","Type":"ContainerDied","Data":"9fff0cc4898992728bc6fa709e35281a5f9b3d582494007e0f1a4025a18e36c2"} Jul 05 13:28:20.419897 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:20.419570 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-fkz5b" event={"ID":"fd1129ca-34e5-464a-8c73-48c5b911683d","Type":"ContainerDied","Data":"207e839147a89bd28e88fe0adafbb0a7cf6b97f5befe90e83a8302aee4bf9a8a"} Jul 05 13:28:20.419897 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:20.419594 2573 scope.go:117] "RemoveContainer" containerID="9fff0cc4898992728bc6fa709e35281a5f9b3d582494007e0f1a4025a18e36c2" Jul 05 13:28:20.431444 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:20.431426 2573 scope.go:117] "RemoveContainer" containerID="9fff0cc4898992728bc6fa709e35281a5f9b3d582494007e0f1a4025a18e36c2" Jul 05 13:28:20.431825 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:28:20.431794 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9fff0cc4898992728bc6fa709e35281a5f9b3d582494007e0f1a4025a18e36c2\": container with ID starting with 9fff0cc4898992728bc6fa709e35281a5f9b3d582494007e0f1a4025a18e36c2 not found: ID does not exist" containerID="9fff0cc4898992728bc6fa709e35281a5f9b3d582494007e0f1a4025a18e36c2" Jul 05 13:28:20.431883 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:20.431835 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fff0cc4898992728bc6fa709e35281a5f9b3d582494007e0f1a4025a18e36c2"} err="failed to get container status \"9fff0cc4898992728bc6fa709e35281a5f9b3d582494007e0f1a4025a18e36c2\": rpc error: code = NotFound desc = could not find container \"9fff0cc4898992728bc6fa709e35281a5f9b3d582494007e0f1a4025a18e36c2\": container with ID starting with 9fff0cc4898992728bc6fa709e35281a5f9b3d582494007e0f1a4025a18e36c2 not found: ID does not exist" Jul 05 13:28:20.446879 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:20.446854 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-fkz5b"] Jul 05 13:28:20.454503 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:20.451533 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-fkz5b"] Jul 05 13:28:21.655071 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:21.655037 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:28:21.735590 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:21.735562 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd1129ca-34e5-464a-8c73-48c5b911683d" path="/var/lib/kubelet/pods/fd1129ca-34e5-464a-8c73-48c5b911683d/volumes" Jul 05 13:28:21.755390 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:21.755364 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:28:24.075946 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:24.075912 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:28:24.142396 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:24.142359 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:28:24.200283 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:24.200252 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:28:24.317237 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:24.317203 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:28:26.520660 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:26.520633 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:28:26.632898 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:26.632865 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:28:57.129554 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:57.129523 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:28:58.183463 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:58.183427 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:28:58.306902 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:28:58.306858 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:29:06.587672 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:29:06.587635 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:29:06.683804 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:29:06.683770 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:29:11.852096 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:29:11.852012 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:29:12.166603 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:29:12.166516 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:29:37.955960 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:29:37.955928 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:29:38.960134 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:29:38.960097 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:29:40.283471 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:29:40.283434 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:29:40.365081 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:29:40.365047 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:29:50.140618 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:29:50.140582 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:29:51.251397 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:29:51.251349 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:30:00.169428 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:00.169388 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29720970-dmn4z"] Jul 05 13:30:00.169901 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:00.169702 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd1129ca-34e5-464a-8c73-48c5b911683d" containerName="authorino" Jul 05 13:30:00.169901 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:00.169712 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd1129ca-34e5-464a-8c73-48c5b911683d" containerName="authorino" Jul 05 13:30:00.169901 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:00.169769 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd1129ca-34e5-464a-8c73-48c5b911683d" containerName="authorino" Jul 05 13:30:00.172782 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:00.172762 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29720970-dmn4z" Jul 05 13:30:00.175704 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:00.175685 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-t2n54\"" Jul 05 13:30:00.187277 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:00.187249 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29720970-dmn4z"] Jul 05 13:30:00.294981 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:00.294913 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mr92n\" (UniqueName: \"kubernetes.io/projected/7551d605-e483-4238-81f3-b0fd5ee6a226-kube-api-access-mr92n\") pod \"maas-api-key-cleanup-29720970-dmn4z\" (UID: \"7551d605-e483-4238-81f3-b0fd5ee6a226\") " pod="opendatahub/maas-api-key-cleanup-29720970-dmn4z" Jul 05 13:30:00.396297 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:00.396254 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mr92n\" (UniqueName: \"kubernetes.io/projected/7551d605-e483-4238-81f3-b0fd5ee6a226-kube-api-access-mr92n\") pod \"maas-api-key-cleanup-29720970-dmn4z\" (UID: \"7551d605-e483-4238-81f3-b0fd5ee6a226\") " pod="opendatahub/maas-api-key-cleanup-29720970-dmn4z" Jul 05 13:30:00.409736 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:00.409696 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mr92n\" (UniqueName: \"kubernetes.io/projected/7551d605-e483-4238-81f3-b0fd5ee6a226-kube-api-access-mr92n\") pod \"maas-api-key-cleanup-29720970-dmn4z\" (UID: \"7551d605-e483-4238-81f3-b0fd5ee6a226\") " pod="opendatahub/maas-api-key-cleanup-29720970-dmn4z" Jul 05 13:30:00.483298 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:00.483260 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29720970-dmn4z" Jul 05 13:30:00.821476 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:00.821454 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29720970-dmn4z"] Jul 05 13:30:00.823825 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:30:00.823800 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7551d605_e483_4238_81f3_b0fd5ee6a226.slice/crio-e83666871375173d2b63ccbc9f402efbacb0af34a9180c9df06c72b95ac02279 WatchSource:0}: Error finding container e83666871375173d2b63ccbc9f402efbacb0af34a9180c9df06c72b95ac02279: Status 404 returned error can't find the container with id e83666871375173d2b63ccbc9f402efbacb0af34a9180c9df06c72b95ac02279 Jul 05 13:30:00.826014 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:00.825993 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 05 13:30:00.891761 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:00.891730 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29720970-dmn4z" event={"ID":"7551d605-e483-4238-81f3-b0fd5ee6a226","Type":"ContainerStarted","Data":"e83666871375173d2b63ccbc9f402efbacb0af34a9180c9df06c72b95ac02279"} Jul 05 13:30:03.909291 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:03.909253 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29720970-dmn4z" event={"ID":"7551d605-e483-4238-81f3-b0fd5ee6a226","Type":"ContainerStarted","Data":"b99d021707d246cef9d7728ea3da6b316cb7ad2fee5d5bd72c303b9420da2743"} Jul 05 13:30:03.926883 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:03.926828 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29720970-dmn4z" podStartSLOduration=1.652520708 podStartE2EDuration="3.926811112s" totalImagesPullingTime="2.274290404s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:30:03.924765898 +0000 UTC m=+676.820423988" watchObservedRunningTime="2026-07-05 13:30:03.926811112 +0000 UTC m=+676.822469200" Jul 05 13:30:04.915271 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:04.915237 2573 generic.go:356] "Generic (PLEG): container finished" podID="7551d605-e483-4238-81f3-b0fd5ee6a226" containerID="b99d021707d246cef9d7728ea3da6b316cb7ad2fee5d5bd72c303b9420da2743" exitCode=0 Jul 05 13:30:04.915664 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:04.915320 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29720970-dmn4z" event={"ID":"7551d605-e483-4238-81f3-b0fd5ee6a226","Type":"ContainerDied","Data":"b99d021707d246cef9d7728ea3da6b316cb7ad2fee5d5bd72c303b9420da2743"} Jul 05 13:30:06.050072 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:06.050050 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29720970-dmn4z" Jul 05 13:30:06.149679 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:06.149640 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mr92n\" (UniqueName: \"kubernetes.io/projected/7551d605-e483-4238-81f3-b0fd5ee6a226-kube-api-access-mr92n\") pod \"7551d605-e483-4238-81f3-b0fd5ee6a226\" (UID: \"7551d605-e483-4238-81f3-b0fd5ee6a226\") " Jul 05 13:30:06.151835 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:06.151809 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7551d605-e483-4238-81f3-b0fd5ee6a226-kube-api-access-mr92n" (OuterVolumeSpecName: "kube-api-access-mr92n") pod "7551d605-e483-4238-81f3-b0fd5ee6a226" (UID: "7551d605-e483-4238-81f3-b0fd5ee6a226"). InnerVolumeSpecName "kube-api-access-mr92n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:30:06.251667 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:06.251616 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mr92n\" (UniqueName: \"kubernetes.io/projected/7551d605-e483-4238-81f3-b0fd5ee6a226-kube-api-access-mr92n\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:30:06.925582 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:06.925554 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29720970-dmn4z" Jul 05 13:30:06.925582 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:06.925572 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29720970-dmn4z" event={"ID":"7551d605-e483-4238-81f3-b0fd5ee6a226","Type":"ContainerDied","Data":"e83666871375173d2b63ccbc9f402efbacb0af34a9180c9df06c72b95ac02279"} Jul 05 13:30:06.925792 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:06.925618 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e83666871375173d2b63ccbc9f402efbacb0af34a9180c9df06c72b95ac02279" Jul 05 13:30:07.866782 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:07.866743 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:30:09.452067 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:09.452030 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:30:20.944986 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:20.944938 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-sl6tv"] Jul 05 13:30:20.945442 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:20.945418 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7551d605-e483-4238-81f3-b0fd5ee6a226" containerName="cleanup" Jul 05 13:30:20.945516 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:20.945449 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="7551d605-e483-4238-81f3-b0fd5ee6a226" containerName="cleanup" Jul 05 13:30:20.945592 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:20.945580 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="7551d605-e483-4238-81f3-b0fd5ee6a226" containerName="cleanup" Jul 05 13:30:20.948671 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:20.948649 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sl6tv" Jul 05 13:30:20.951333 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:20.951313 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 05 13:30:20.952262 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:20.952243 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-5hzdx\"" Jul 05 13:30:20.952342 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:20.952243 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 05 13:30:20.952342 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:20.952251 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 05 13:30:20.960921 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:20.960900 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-sl6tv"] Jul 05 13:30:20.974207 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:20.974185 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:30:21.080226 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:21.080185 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3de7428c-db34-439c-92b2-a6cd6e2336b4-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sl6tv\" (UID: \"3de7428c-db34-439c-92b2-a6cd6e2336b4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sl6tv" Jul 05 13:30:21.080410 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:21.080261 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3de7428c-db34-439c-92b2-a6cd6e2336b4-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sl6tv\" (UID: \"3de7428c-db34-439c-92b2-a6cd6e2336b4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sl6tv" Jul 05 13:30:21.080410 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:21.080309 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3de7428c-db34-439c-92b2-a6cd6e2336b4-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sl6tv\" (UID: \"3de7428c-db34-439c-92b2-a6cd6e2336b4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sl6tv" Jul 05 13:30:21.080410 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:21.080346 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d475p\" (UniqueName: \"kubernetes.io/projected/3de7428c-db34-439c-92b2-a6cd6e2336b4-kube-api-access-d475p\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sl6tv\" (UID: \"3de7428c-db34-439c-92b2-a6cd6e2336b4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sl6tv" Jul 05 13:30:21.080410 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:21.080373 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3de7428c-db34-439c-92b2-a6cd6e2336b4-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sl6tv\" (UID: \"3de7428c-db34-439c-92b2-a6cd6e2336b4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sl6tv" Jul 05 13:30:21.080410 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:21.080394 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3de7428c-db34-439c-92b2-a6cd6e2336b4-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sl6tv\" (UID: \"3de7428c-db34-439c-92b2-a6cd6e2336b4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sl6tv" Jul 05 13:30:21.181583 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:21.181545 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3de7428c-db34-439c-92b2-a6cd6e2336b4-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sl6tv\" (UID: \"3de7428c-db34-439c-92b2-a6cd6e2336b4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sl6tv" Jul 05 13:30:21.181785 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:21.181599 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3de7428c-db34-439c-92b2-a6cd6e2336b4-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sl6tv\" (UID: \"3de7428c-db34-439c-92b2-a6cd6e2336b4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sl6tv" Jul 05 13:30:21.181785 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:21.181646 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3de7428c-db34-439c-92b2-a6cd6e2336b4-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sl6tv\" (UID: \"3de7428c-db34-439c-92b2-a6cd6e2336b4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sl6tv" Jul 05 13:30:21.181785 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:21.181665 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d475p\" (UniqueName: \"kubernetes.io/projected/3de7428c-db34-439c-92b2-a6cd6e2336b4-kube-api-access-d475p\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sl6tv\" (UID: \"3de7428c-db34-439c-92b2-a6cd6e2336b4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sl6tv" Jul 05 13:30:21.181785 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:21.181697 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3de7428c-db34-439c-92b2-a6cd6e2336b4-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sl6tv\" (UID: \"3de7428c-db34-439c-92b2-a6cd6e2336b4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sl6tv" Jul 05 13:30:21.182066 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:21.181916 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3de7428c-db34-439c-92b2-a6cd6e2336b4-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sl6tv\" (UID: \"3de7428c-db34-439c-92b2-a6cd6e2336b4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sl6tv" Jul 05 13:30:21.182122 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:21.182065 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3de7428c-db34-439c-92b2-a6cd6e2336b4-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sl6tv\" (UID: \"3de7428c-db34-439c-92b2-a6cd6e2336b4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sl6tv" Jul 05 13:30:21.182122 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:21.182096 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3de7428c-db34-439c-92b2-a6cd6e2336b4-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sl6tv\" (UID: \"3de7428c-db34-439c-92b2-a6cd6e2336b4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sl6tv" Jul 05 13:30:21.182194 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:21.182167 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3de7428c-db34-439c-92b2-a6cd6e2336b4-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sl6tv\" (UID: \"3de7428c-db34-439c-92b2-a6cd6e2336b4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sl6tv" Jul 05 13:30:21.183976 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:21.183930 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3de7428c-db34-439c-92b2-a6cd6e2336b4-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sl6tv\" (UID: \"3de7428c-db34-439c-92b2-a6cd6e2336b4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sl6tv" Jul 05 13:30:21.184249 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:21.184231 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3de7428c-db34-439c-92b2-a6cd6e2336b4-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sl6tv\" (UID: \"3de7428c-db34-439c-92b2-a6cd6e2336b4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sl6tv" Jul 05 13:30:21.188993 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:21.188973 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d475p\" (UniqueName: \"kubernetes.io/projected/3de7428c-db34-439c-92b2-a6cd6e2336b4-kube-api-access-d475p\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sl6tv\" (UID: \"3de7428c-db34-439c-92b2-a6cd6e2336b4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sl6tv" Jul 05 13:30:21.258831 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:21.258754 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sl6tv" Jul 05 13:30:21.389875 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:21.389848 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-sl6tv"] Jul 05 13:30:21.391554 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:30:21.391522 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3de7428c_db34_439c_92b2_a6cd6e2336b4.slice/crio-257d6c27099faaf3728c02528ddc3f309fdfa8320a069f9ac44b7322a0439284 WatchSource:0}: Error finding container 257d6c27099faaf3728c02528ddc3f309fdfa8320a069f9ac44b7322a0439284: Status 404 returned error can't find the container with id 257d6c27099faaf3728c02528ddc3f309fdfa8320a069f9ac44b7322a0439284 Jul 05 13:30:21.999073 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:21.998986 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sl6tv" event={"ID":"3de7428c-db34-439c-92b2-a6cd6e2336b4","Type":"ContainerStarted","Data":"257d6c27099faaf3728c02528ddc3f309fdfa8320a069f9ac44b7322a0439284"} Jul 05 13:30:22.851235 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:22.851199 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:30:23.006410 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:23.006371 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sl6tv" event={"ID":"3de7428c-db34-439c-92b2-a6cd6e2336b4","Type":"ContainerStarted","Data":"d2254592d557e9efb7e0b8e7089c94882b378e0c0911739505418bbd20175a7a"} Jul 05 13:30:23.026062 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:23.025994 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sl6tv" podStartSLOduration=2.038765094 podStartE2EDuration="3.025973504s" totalImagesPullingTime="987.20841ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:30:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:30:23.02508353 +0000 UTC m=+695.920741619" watchObservedRunningTime="2026-07-05 13:30:23.025973504 +0000 UTC m=+695.921631585" Jul 05 13:30:28.468868 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:28.468834 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-69fnx"] Jul 05 13:30:28.472323 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:28.472306 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-69fnx" Jul 05 13:30:28.477024 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:28.477003 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 05 13:30:28.489242 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:28.489217 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-69fnx"] Jul 05 13:30:28.546550 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:28.546522 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/930ebf4c-3588-40da-9e0a-eb8e6902317f-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-69fnx\" (UID: \"930ebf4c-3588-40da-9e0a-eb8e6902317f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-69fnx" Jul 05 13:30:28.546672 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:28.546559 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckgph\" (UniqueName: \"kubernetes.io/projected/930ebf4c-3588-40da-9e0a-eb8e6902317f-kube-api-access-ckgph\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-69fnx\" (UID: \"930ebf4c-3588-40da-9e0a-eb8e6902317f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-69fnx" Jul 05 13:30:28.546672 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:28.546588 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/930ebf4c-3588-40da-9e0a-eb8e6902317f-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-69fnx\" (UID: \"930ebf4c-3588-40da-9e0a-eb8e6902317f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-69fnx" Jul 05 13:30:28.546672 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:28.546650 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/930ebf4c-3588-40da-9e0a-eb8e6902317f-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-69fnx\" (UID: \"930ebf4c-3588-40da-9e0a-eb8e6902317f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-69fnx" Jul 05 13:30:28.546773 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:28.546696 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/930ebf4c-3588-40da-9e0a-eb8e6902317f-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-69fnx\" (UID: \"930ebf4c-3588-40da-9e0a-eb8e6902317f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-69fnx" Jul 05 13:30:28.546773 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:28.546750 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/930ebf4c-3588-40da-9e0a-eb8e6902317f-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-69fnx\" (UID: \"930ebf4c-3588-40da-9e0a-eb8e6902317f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-69fnx" Jul 05 13:30:28.646762 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:28.646724 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:30:28.647560 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:28.647535 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/930ebf4c-3588-40da-9e0a-eb8e6902317f-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-69fnx\" (UID: \"930ebf4c-3588-40da-9e0a-eb8e6902317f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-69fnx" Jul 05 13:30:28.647694 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:28.647568 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckgph\" (UniqueName: \"kubernetes.io/projected/930ebf4c-3588-40da-9e0a-eb8e6902317f-kube-api-access-ckgph\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-69fnx\" (UID: \"930ebf4c-3588-40da-9e0a-eb8e6902317f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-69fnx" Jul 05 13:30:28.647694 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:28.647598 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/930ebf4c-3588-40da-9e0a-eb8e6902317f-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-69fnx\" (UID: \"930ebf4c-3588-40da-9e0a-eb8e6902317f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-69fnx" Jul 05 13:30:28.647694 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:28.647622 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/930ebf4c-3588-40da-9e0a-eb8e6902317f-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-69fnx\" (UID: \"930ebf4c-3588-40da-9e0a-eb8e6902317f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-69fnx" Jul 05 13:30:28.647694 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:28.647646 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/930ebf4c-3588-40da-9e0a-eb8e6902317f-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-69fnx\" (UID: \"930ebf4c-3588-40da-9e0a-eb8e6902317f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-69fnx" Jul 05 13:30:28.647694 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:28.647688 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/930ebf4c-3588-40da-9e0a-eb8e6902317f-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-69fnx\" (UID: \"930ebf4c-3588-40da-9e0a-eb8e6902317f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-69fnx" Jul 05 13:30:28.648105 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:28.648072 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/930ebf4c-3588-40da-9e0a-eb8e6902317f-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-69fnx\" (UID: \"930ebf4c-3588-40da-9e0a-eb8e6902317f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-69fnx" Jul 05 13:30:28.648105 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:28.648094 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/930ebf4c-3588-40da-9e0a-eb8e6902317f-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-69fnx\" (UID: \"930ebf4c-3588-40da-9e0a-eb8e6902317f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-69fnx" Jul 05 13:30:28.648262 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:28.648132 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/930ebf4c-3588-40da-9e0a-eb8e6902317f-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-69fnx\" (UID: \"930ebf4c-3588-40da-9e0a-eb8e6902317f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-69fnx" Jul 05 13:30:28.649804 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:28.649783 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/930ebf4c-3588-40da-9e0a-eb8e6902317f-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-69fnx\" (UID: \"930ebf4c-3588-40da-9e0a-eb8e6902317f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-69fnx" Jul 05 13:30:28.650164 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:28.650147 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/930ebf4c-3588-40da-9e0a-eb8e6902317f-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-69fnx\" (UID: \"930ebf4c-3588-40da-9e0a-eb8e6902317f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-69fnx" Jul 05 13:30:28.655501 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:28.655482 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckgph\" (UniqueName: \"kubernetes.io/projected/930ebf4c-3588-40da-9e0a-eb8e6902317f-kube-api-access-ckgph\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-69fnx\" (UID: \"930ebf4c-3588-40da-9e0a-eb8e6902317f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-69fnx" Jul 05 13:30:28.780735 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:28.780650 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-69fnx" Jul 05 13:30:28.917016 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:28.916985 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-69fnx"] Jul 05 13:30:28.918588 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:30:28.918559 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod930ebf4c_3588_40da_9e0a_eb8e6902317f.slice/crio-6bc1ee988bc04a93fe3834e7897b7062712bafd7000436749cfa6fe17d2804af WatchSource:0}: Error finding container 6bc1ee988bc04a93fe3834e7897b7062712bafd7000436749cfa6fe17d2804af: Status 404 returned error can't find the container with id 6bc1ee988bc04a93fe3834e7897b7062712bafd7000436749cfa6fe17d2804af Jul 05 13:30:29.036440 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:29.036359 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-69fnx" event={"ID":"930ebf4c-3588-40da-9e0a-eb8e6902317f","Type":"ContainerStarted","Data":"6bc1ee988bc04a93fe3834e7897b7062712bafd7000436749cfa6fe17d2804af"} Jul 05 13:30:30.042910 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:30.042872 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-69fnx" event={"ID":"930ebf4c-3588-40da-9e0a-eb8e6902317f","Type":"ContainerStarted","Data":"e55bce6e226d44d76895963c31d977b8558d41225bbb3ef64b9b0b7c9f4dd081"} Jul 05 13:30:30.066928 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:30.066880 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-69fnx" podStartSLOduration=1.9128965070000001 podStartE2EDuration="2.066865708s" totalImagesPullingTime="153.969201ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:30:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:30:30.064641231 +0000 UTC m=+702.960299320" watchObservedRunningTime="2026-07-05 13:30:30.066865708 +0000 UTC m=+702.962523797" Jul 05 13:30:31.259760 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:31.259727 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sl6tv" Jul 05 13:30:31.260258 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:31.260012 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sl6tv" Jul 05 13:30:31.272434 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:31.272410 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sl6tv" Jul 05 13:30:31.847923 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:31.847888 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:30:32.065722 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:32.065693 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sl6tv" Jul 05 13:30:38.780973 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:38.780921 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-69fnx" Jul 05 13:30:38.781362 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:38.780986 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-69fnx" Jul 05 13:30:38.793515 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:38.793488 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-69fnx" Jul 05 13:30:39.101174 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:39.101090 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-69fnx" Jul 05 13:30:39.929025 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:39.928972 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:30:40.030731 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:30:40.030692 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:31:31.225840 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:31:31.225802 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6fc7fd77f-4bqc2"] Jul 05 13:31:31.229145 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:31:31.229127 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6fc7fd77f-4bqc2" Jul 05 13:31:31.236643 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:31:31.236616 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6fc7fd77f-4bqc2"] Jul 05 13:31:31.395373 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:31:31.395331 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69mgj\" (UniqueName: \"kubernetes.io/projected/19474780-521c-42be-904f-5d3d59bd53bf-kube-api-access-69mgj\") pod \"authorino-6fc7fd77f-4bqc2\" (UID: \"19474780-521c-42be-904f-5d3d59bd53bf\") " pod="kuadrant-system/authorino-6fc7fd77f-4bqc2" Jul 05 13:31:31.395570 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:31:31.395382 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/19474780-521c-42be-904f-5d3d59bd53bf-tls-cert\") pod \"authorino-6fc7fd77f-4bqc2\" (UID: \"19474780-521c-42be-904f-5d3d59bd53bf\") " pod="kuadrant-system/authorino-6fc7fd77f-4bqc2" Jul 05 13:31:31.496840 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:31:31.496737 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/19474780-521c-42be-904f-5d3d59bd53bf-tls-cert\") pod \"authorino-6fc7fd77f-4bqc2\" (UID: \"19474780-521c-42be-904f-5d3d59bd53bf\") " pod="kuadrant-system/authorino-6fc7fd77f-4bqc2" Jul 05 13:31:31.497010 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:31:31.496867 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69mgj\" (UniqueName: \"kubernetes.io/projected/19474780-521c-42be-904f-5d3d59bd53bf-kube-api-access-69mgj\") pod \"authorino-6fc7fd77f-4bqc2\" (UID: \"19474780-521c-42be-904f-5d3d59bd53bf\") " pod="kuadrant-system/authorino-6fc7fd77f-4bqc2" Jul 05 13:31:31.499209 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:31:31.499187 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/19474780-521c-42be-904f-5d3d59bd53bf-tls-cert\") pod \"authorino-6fc7fd77f-4bqc2\" (UID: \"19474780-521c-42be-904f-5d3d59bd53bf\") " pod="kuadrant-system/authorino-6fc7fd77f-4bqc2" Jul 05 13:31:31.504399 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:31:31.504374 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-69mgj\" (UniqueName: \"kubernetes.io/projected/19474780-521c-42be-904f-5d3d59bd53bf-kube-api-access-69mgj\") pod \"authorino-6fc7fd77f-4bqc2\" (UID: \"19474780-521c-42be-904f-5d3d59bd53bf\") " pod="kuadrant-system/authorino-6fc7fd77f-4bqc2" Jul 05 13:31:31.539469 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:31:31.539426 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6fc7fd77f-4bqc2" Jul 05 13:31:31.648817 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:31:31.648777 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:31:31.674768 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:31:31.674739 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6fc7fd77f-4bqc2"] Jul 05 13:31:31.676686 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:31:31.676657 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod19474780_521c_42be_904f_5d3d59bd53bf.slice/crio-cb6a2fa2bfeb0afc23e0b039d3062aff68c6a8efd149f6c0f0565672314f432d WatchSource:0}: Error finding container cb6a2fa2bfeb0afc23e0b039d3062aff68c6a8efd149f6c0f0565672314f432d: Status 404 returned error can't find the container with id cb6a2fa2bfeb0afc23e0b039d3062aff68c6a8efd149f6c0f0565672314f432d Jul 05 13:31:31.747742 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:31:31.747644 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:31:32.354791 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:31:32.354706 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6fc7fd77f-4bqc2" event={"ID":"19474780-521c-42be-904f-5d3d59bd53bf","Type":"ContainerStarted","Data":"ba135c5feb7be8691fb08be9a39eacf4fb86a3d06d0afe63c854b0e5567bd1ac"} Jul 05 13:31:32.354791 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:31:32.354751 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6fc7fd77f-4bqc2" event={"ID":"19474780-521c-42be-904f-5d3d59bd53bf","Type":"ContainerStarted","Data":"cb6a2fa2bfeb0afc23e0b039d3062aff68c6a8efd149f6c0f0565672314f432d"} Jul 05 13:31:32.385806 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:31:32.385730 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6fc7fd77f-4bqc2" podStartSLOduration=1.04849682 podStartE2EDuration="1.385710074s" totalImagesPullingTime="337.213254ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:31:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:31:32.384479588 +0000 UTC m=+765.280137678" watchObservedRunningTime="2026-07-05 13:31:32.385710074 +0000 UTC m=+765.281368163" Jul 05 13:31:32.435637 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:31:32.435557 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5554dc9854-8rtzw"] Jul 05 13:31:32.435828 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:31:32.435771 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5554dc9854-8rtzw" podUID="1b206266-6d09-425a-b979-2daa7fb1f516" containerName="authorino" containerID="cri-o://4a253b9357470c0435f13aa45787e8f8fad8bdfe5530ebc7f6b70e0f26b0b28e" gracePeriod=30 Jul 05 13:31:32.701385 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:31:32.701352 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5554dc9854-8rtzw" Jul 05 13:31:32.809845 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:31:32.809807 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1b206266-6d09-425a-b979-2daa7fb1f516-tls-cert\") pod \"1b206266-6d09-425a-b979-2daa7fb1f516\" (UID: \"1b206266-6d09-425a-b979-2daa7fb1f516\") " Jul 05 13:31:32.810067 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:31:32.809883 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gbhzk\" (UniqueName: \"kubernetes.io/projected/1b206266-6d09-425a-b979-2daa7fb1f516-kube-api-access-gbhzk\") pod \"1b206266-6d09-425a-b979-2daa7fb1f516\" (UID: \"1b206266-6d09-425a-b979-2daa7fb1f516\") " Jul 05 13:31:32.813073 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:31:32.813036 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b206266-6d09-425a-b979-2daa7fb1f516-kube-api-access-gbhzk" (OuterVolumeSpecName: "kube-api-access-gbhzk") pod "1b206266-6d09-425a-b979-2daa7fb1f516" (UID: "1b206266-6d09-425a-b979-2daa7fb1f516"). InnerVolumeSpecName "kube-api-access-gbhzk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:31:32.828465 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:31:32.828435 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b206266-6d09-425a-b979-2daa7fb1f516-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "1b206266-6d09-425a-b979-2daa7fb1f516" (UID: "1b206266-6d09-425a-b979-2daa7fb1f516"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:31:32.911234 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:31:32.911142 2573 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1b206266-6d09-425a-b979-2daa7fb1f516-tls-cert\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:31:32.911234 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:31:32.911182 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gbhzk\" (UniqueName: \"kubernetes.io/projected/1b206266-6d09-425a-b979-2daa7fb1f516-kube-api-access-gbhzk\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:31:32.930791 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:31:32.930752 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:31:32.998594 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:31:32.998545 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:31:33.361732 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:31:33.361699 2573 generic.go:356] "Generic (PLEG): container finished" podID="1b206266-6d09-425a-b979-2daa7fb1f516" containerID="4a253b9357470c0435f13aa45787e8f8fad8bdfe5530ebc7f6b70e0f26b0b28e" exitCode=0 Jul 05 13:31:33.362179 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:31:33.361761 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5554dc9854-8rtzw" Jul 05 13:31:33.362179 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:31:33.361788 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5554dc9854-8rtzw" event={"ID":"1b206266-6d09-425a-b979-2daa7fb1f516","Type":"ContainerDied","Data":"4a253b9357470c0435f13aa45787e8f8fad8bdfe5530ebc7f6b70e0f26b0b28e"} Jul 05 13:31:33.362179 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:31:33.361832 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5554dc9854-8rtzw" event={"ID":"1b206266-6d09-425a-b979-2daa7fb1f516","Type":"ContainerDied","Data":"04cc9f531cca30eb30433534ec65611fc1d8f78e03383281d548af458681930c"} Jul 05 13:31:33.362179 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:31:33.361852 2573 scope.go:117] "RemoveContainer" containerID="4a253b9357470c0435f13aa45787e8f8fad8bdfe5530ebc7f6b70e0f26b0b28e" Jul 05 13:31:33.370663 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:31:33.370639 2573 scope.go:117] "RemoveContainer" containerID="4a253b9357470c0435f13aa45787e8f8fad8bdfe5530ebc7f6b70e0f26b0b28e" Jul 05 13:31:33.370993 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:31:33.370968 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a253b9357470c0435f13aa45787e8f8fad8bdfe5530ebc7f6b70e0f26b0b28e\": container with ID starting with 4a253b9357470c0435f13aa45787e8f8fad8bdfe5530ebc7f6b70e0f26b0b28e not found: ID does not exist" containerID="4a253b9357470c0435f13aa45787e8f8fad8bdfe5530ebc7f6b70e0f26b0b28e" Jul 05 13:31:33.371071 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:31:33.371003 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a253b9357470c0435f13aa45787e8f8fad8bdfe5530ebc7f6b70e0f26b0b28e"} err="failed to get container status \"4a253b9357470c0435f13aa45787e8f8fad8bdfe5530ebc7f6b70e0f26b0b28e\": rpc error: code = NotFound desc = could not find container \"4a253b9357470c0435f13aa45787e8f8fad8bdfe5530ebc7f6b70e0f26b0b28e\": container with ID starting with 4a253b9357470c0435f13aa45787e8f8fad8bdfe5530ebc7f6b70e0f26b0b28e not found: ID does not exist" Jul 05 13:31:33.392770 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:31:33.392736 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5554dc9854-8rtzw"] Jul 05 13:31:33.396601 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:31:33.396577 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5554dc9854-8rtzw"] Jul 05 13:31:33.737143 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:31:33.736740 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b206266-6d09-425a-b979-2daa7fb1f516" path="/var/lib/kubelet/pods/1b206266-6d09-425a-b979-2daa7fb1f516/volumes" Jul 05 13:31:46.124622 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:31:46.124583 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:32:01.460400 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:32:01.460360 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:32:14.454366 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:32:14.454279 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:32:22.131299 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:32:22.131255 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:32:23.555368 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:32:23.555328 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:32:32.347096 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:32:32.347053 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:32:43.151667 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:32:43.151629 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:32:45.214322 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:32:45.214285 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:32:46.702086 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:32:46.702054 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:33:09.856168 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:33:09.856128 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:33:16.547171 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:33:16.547133 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:33:16.648518 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:33:16.648481 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:33:17.879615 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:33:17.879580 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:33:20.843309 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:33:20.843268 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:33:26.043127 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:33:26.043085 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:33:26.142457 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:33:26.142428 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:33:26.263671 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:33:26.263637 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:33:32.258312 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:33:32.258270 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:33:32.322398 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:33:32.322362 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:33:32.419604 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:33:32.419567 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:33:32.506994 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:33:32.506943 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:33:32.626369 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:33:32.626273 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:33:38.685656 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:33:38.685621 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:33:38.748777 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:33:38.748746 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:33:55.248046 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:33:55.248003 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:34:05.252779 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:34:05.252746 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:34:13.561562 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:34:13.561526 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:34:22.752881 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:34:22.752840 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:34:33.767820 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:34:33.767777 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:34:45.550520 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:34:45.550481 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:34:56.250049 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:34:56.250012 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:35:05.847113 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:35:05.847079 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:35:28.864620 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:35:28.864586 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:35:37.662827 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:35:37.662792 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:36:00.752380 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:36:00.752336 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:36:11.650361 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:36:11.650323 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:36:28.451605 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:36:28.451568 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:36:38.157107 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:36:38.157070 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:36:48.351587 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:36:48.351505 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:36:49.835664 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:36:49.835627 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:36:58.036496 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:36:58.036459 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:37:07.087368 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:37:07.087332 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:38:45.966180 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:38:45.966148 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:38:54.246700 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:38:54.246658 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:39:05.553306 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:39:05.553273 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:39:14.052435 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:39:14.052391 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:39:25.247598 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:39:25.247561 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:39:34.247944 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:39:34.247902 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:39:46.245121 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:39:46.245046 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:39:54.960828 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:39:54.960791 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:40:05.452445 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:40:05.452407 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:40:17.312485 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:40:17.312448 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:40:26.147704 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:40:26.147664 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:40:27.418498 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:40:27.418457 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:40:36.950144 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:40:36.950110 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:40:59.060125 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:40:59.060090 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:41:08.554793 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:41:08.554758 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:41:17.949624 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:41:17.949532 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:41:30.549476 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:41:30.549436 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:41:46.053438 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:41:46.053390 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:41:57.503175 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:41:57.503132 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:41:57.544103 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:41:57.544074 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:41:58.583450 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:41:58.583381 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:42:06.748979 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:42:06.748933 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:42:16.847562 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:42:16.847528 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:42:26.547270 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:42:26.547234 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:42:35.648066 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:42:35.648027 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:42:48.659179 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:42:48.659090 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:42:58.059763 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:42:58.059729 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:43:07.455404 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:43:07.455365 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:43:18.657622 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:43:18.657582 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:43:30.898588 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:43:30.898552 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:43:39.547860 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:43:39.547828 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:43:53.374760 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:43:53.374725 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:44:04.683191 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:44:04.683150 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:44:17.359329 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:44:17.359251 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:44:23.467218 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:44:23.467176 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:44:23.555543 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:44:23.555504 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:44:30.143832 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:44:30.143793 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:44:30.251281 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:44:30.251241 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:44:30.426265 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:44:30.426177 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:44:30.488040 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:44:30.488007 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:44:31.582690 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:44:31.582653 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:44:31.675212 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:44:31.675174 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:44:31.818349 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:44:31.818311 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:44:31.988334 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:44:31.988302 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:44:32.265755 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:44:32.265672 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:44:32.367072 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:44:32.367035 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:44:34.265663 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:44:34.265626 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:44:36.360637 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:44:36.360600 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:44:36.592196 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:44:36.592154 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:44:36.773391 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:44:36.773353 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:44:36.875001 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:44:36.874969 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:44:37.012402 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:44:37.012371 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:44:38.544011 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:44:38.543978 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:44:43.146980 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:44:43.146917 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:44:43.829049 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:44:43.829013 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:44:43.940212 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:44:43.940166 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:44:44.109128 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:44:44.109036 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:44:44.204701 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:44:44.204663 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:44:45.268934 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:44:45.268900 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:44:45.366521 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:44:45.366480 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:44:45.430594 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:44:45.430557 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:44:45.630207 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:44:45.630119 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:44:46.112794 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:44:46.112762 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:44:46.216790 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:44:46.216758 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:44:47.381930 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:44:47.381902 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:44:50.363696 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:44:50.363661 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:44:50.823005 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:44:50.822936 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:44:56.470376 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:44:56.470327 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:44:56.571828 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:44:56.571794 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:44:58.971249 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:44:58.971217 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:45:02.844904 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:45:02.844865 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:45:02.952006 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:45:02.951968 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:45:03.072919 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:45:03.072880 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:45:03.164992 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:45:03.164883 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:45:04.164130 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:45:04.164094 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:45:04.241526 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:45:04.241494 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:45:04.315480 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:45:04.315451 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:45:04.465424 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:45:04.465395 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:45:04.884843 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:45:04.884765 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:45:04.949588 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:45:04.949547 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:45:06.481339 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:45:06.481283 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:45:07.051787 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:45:07.051753 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:45:08.599444 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:45:08.599403 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:45:08.790840 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:45:08.790798 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:45:08.888608 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:45:08.888521 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:45:09.021609 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:45:09.021579 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:45:10.543174 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:45:10.543131 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:45:15.051315 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:45:15.051281 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:45:15.125131 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:45:15.125099 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:45:15.597345 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:45:15.597309 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:45:15.660775 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:45:15.660741 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:45:17.181589 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:45:17.181553 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:45:19.474523 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:45:19.474492 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:45:23.436482 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:45:23.436445 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:45:23.499865 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:45:23.499828 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:45:23.571275 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:45:23.571234 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:45:23.675598 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:45:23.675562 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:45:23.771093 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:45:23.771004 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:45:24.138467 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:45:24.138373 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:45:50.139138 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:45:50.139106 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:45:56.640841 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:45:56.640801 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:45:56.704320 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:45:56.704281 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:45:56.798429 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:45:56.798389 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:45:56.888347 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:45:56.888312 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:46:04.070359 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:46:04.070323 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:46:12.741070 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:46:12.741031 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:46:19.330904 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:46:19.330866 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:46:19.435466 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:46:19.435425 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:46:19.535079 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:46:19.535046 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:46:30.110429 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:46:30.110389 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:46:30.933256 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:46:30.933218 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:46:37.478019 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:46:37.477978 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:46:37.540048 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:46:37.540007 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:46:37.625510 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:46:37.625476 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:46:37.731618 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:46:37.731506 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:46:37.835935 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:46:37.835900 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:46:38.634851 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:46:38.634816 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:46:38.730352 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:46:38.730319 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:46:39.754824 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:46:39.754793 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:46:40.373489 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:46:40.373460 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:46:46.275056 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:46:46.275015 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:46:46.333700 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:46:46.333669 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:46:47.001444 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:46:47.001407 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:46:47.375242 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:46:47.375164 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:46:54.271891 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:46:54.271860 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:46:54.370248 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:46:54.370211 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:46:54.507476 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:46:54.507444 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:46:54.836559 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:46:54.836525 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:46:55.076785 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:46:55.076751 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:46:55.314774 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:46:55.314742 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:46:57.675759 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:46:57.675726 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:46:57.751335 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:46:57.751299 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:46:57.928111 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:46:57.928021 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:46:58.299154 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:46:58.299122 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:46:58.570587 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:46:58.570496 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:46:59.837686 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:46:59.837648 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:02.062198 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:02.062168 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:02.672696 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:02.672661 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:02.784139 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:02.784102 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:03.114806 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:03.114763 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:03.170469 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:03.170436 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:47:09.723017 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:09.722916 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:09.790551 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:09.790507 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:10.159641 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:10.159544 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:10.517767 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:10.517723 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:10.576590 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:10.576559 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:13.559347 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:13.559311 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:47:16.767799 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:16.767740 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:16.872198 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:16.872161 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:17.081567 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:17.081469 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:17.362667 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:17.362569 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:17.465802 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:17.465768 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:17.573157 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:17.573111 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:17.671532 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:17.671441 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:24.484679 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:24.484641 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:24.576824 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:24.576787 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:24.670454 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:24.670421 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:24.848107 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:24.848015 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:31.051143 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:31.051112 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:31.147502 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:31.147464 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:37.852834 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:37.852788 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:38.289406 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:38.289368 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:38.394753 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:38.394701 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:41.853707 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:41.853675 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:41.968260 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:41.968230 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:42.039564 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:42.039529 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:42.275042 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:42.275008 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:42.678804 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:42.678722 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:42.789620 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:42.789583 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:46.355674 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:46.355640 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:46.656135 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:46.656049 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:47:54.072842 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:54.072807 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-3536bbfe/e2e-lifecycle-model-3536bbfe-kserve-bf6ccc5bc-mzr8f"] Jul 05 13:47:54.073226 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:54.073162 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b206266-6d09-425a-b979-2daa7fb1f516" containerName="authorino" Jul 05 13:47:54.073226 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:54.073174 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b206266-6d09-425a-b979-2daa7fb1f516" containerName="authorino" Jul 05 13:47:54.073298 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:54.073233 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b206266-6d09-425a-b979-2daa7fb1f516" containerName="authorino" Jul 05 13:47:54.076114 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:54.076098 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-3536bbfe/e2e-lifecycle-model-3536bbfe-kserve-bf6ccc5bc-mzr8f" Jul 05 13:47:54.080196 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:54.080175 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-3536bbfe\"/\"default-dockercfg-d7rsg\"" Jul 05 13:47:54.080331 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:54.080199 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-3536bbfe\"/\"openshift-service-ca.crt\"" Jul 05 13:47:54.080331 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:54.080176 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-3536bbfe\"/\"e2e-lifecycle-model-3536bbfe-kserve-self-signed-certs\"" Jul 05 13:47:54.080331 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:54.080175 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-3536bbfe\"/\"kube-root-ca.crt\"" Jul 05 13:47:54.089901 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:54.089876 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-3536bbfe/e2e-lifecycle-model-3536bbfe-kserve-bf6ccc5bc-mzr8f"] Jul 05 13:47:54.115006 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:54.114975 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:54.207317 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:54.201940 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff-home\") pod \"e2e-lifecycle-model-3536bbfe-kserve-bf6ccc5bc-mzr8f\" (UID: \"52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff\") " pod="ai-tenant-e2e-mt-3536bbfe/e2e-lifecycle-model-3536bbfe-kserve-bf6ccc5bc-mzr8f" Jul 05 13:47:54.207317 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:54.202070 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff-model-cache\") pod \"e2e-lifecycle-model-3536bbfe-kserve-bf6ccc5bc-mzr8f\" (UID: \"52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff\") " pod="ai-tenant-e2e-mt-3536bbfe/e2e-lifecycle-model-3536bbfe-kserve-bf6ccc5bc-mzr8f" Jul 05 13:47:54.207317 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:54.202119 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff-tls-certs\") pod \"e2e-lifecycle-model-3536bbfe-kserve-bf6ccc5bc-mzr8f\" (UID: \"52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff\") " pod="ai-tenant-e2e-mt-3536bbfe/e2e-lifecycle-model-3536bbfe-kserve-bf6ccc5bc-mzr8f" Jul 05 13:47:54.207317 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:54.202171 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6k29n\" (UniqueName: \"kubernetes.io/projected/52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff-kube-api-access-6k29n\") pod \"e2e-lifecycle-model-3536bbfe-kserve-bf6ccc5bc-mzr8f\" (UID: \"52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff\") " pod="ai-tenant-e2e-mt-3536bbfe/e2e-lifecycle-model-3536bbfe-kserve-bf6ccc5bc-mzr8f" Jul 05 13:47:54.207317 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:54.202251 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff-tmp-dir\") pod \"e2e-lifecycle-model-3536bbfe-kserve-bf6ccc5bc-mzr8f\" (UID: \"52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff\") " pod="ai-tenant-e2e-mt-3536bbfe/e2e-lifecycle-model-3536bbfe-kserve-bf6ccc5bc-mzr8f" Jul 05 13:47:54.207317 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:54.202285 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff-dshm\") pod \"e2e-lifecycle-model-3536bbfe-kserve-bf6ccc5bc-mzr8f\" (UID: \"52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff\") " pod="ai-tenant-e2e-mt-3536bbfe/e2e-lifecycle-model-3536bbfe-kserve-bf6ccc5bc-mzr8f" Jul 05 13:47:54.303118 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:54.303085 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff-home\") pod \"e2e-lifecycle-model-3536bbfe-kserve-bf6ccc5bc-mzr8f\" (UID: \"52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff\") " pod="ai-tenant-e2e-mt-3536bbfe/e2e-lifecycle-model-3536bbfe-kserve-bf6ccc5bc-mzr8f" Jul 05 13:47:54.303308 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:54.303128 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff-model-cache\") pod \"e2e-lifecycle-model-3536bbfe-kserve-bf6ccc5bc-mzr8f\" (UID: \"52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff\") " pod="ai-tenant-e2e-mt-3536bbfe/e2e-lifecycle-model-3536bbfe-kserve-bf6ccc5bc-mzr8f" Jul 05 13:47:54.303308 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:54.303148 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff-tls-certs\") pod \"e2e-lifecycle-model-3536bbfe-kserve-bf6ccc5bc-mzr8f\" (UID: \"52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff\") " pod="ai-tenant-e2e-mt-3536bbfe/e2e-lifecycle-model-3536bbfe-kserve-bf6ccc5bc-mzr8f" Jul 05 13:47:54.303308 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:54.303189 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6k29n\" (UniqueName: \"kubernetes.io/projected/52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff-kube-api-access-6k29n\") pod \"e2e-lifecycle-model-3536bbfe-kserve-bf6ccc5bc-mzr8f\" (UID: \"52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff\") " pod="ai-tenant-e2e-mt-3536bbfe/e2e-lifecycle-model-3536bbfe-kserve-bf6ccc5bc-mzr8f" Jul 05 13:47:54.303308 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:54.303235 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff-tmp-dir\") pod \"e2e-lifecycle-model-3536bbfe-kserve-bf6ccc5bc-mzr8f\" (UID: \"52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff\") " pod="ai-tenant-e2e-mt-3536bbfe/e2e-lifecycle-model-3536bbfe-kserve-bf6ccc5bc-mzr8f" Jul 05 13:47:54.303308 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:54.303266 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff-dshm\") pod \"e2e-lifecycle-model-3536bbfe-kserve-bf6ccc5bc-mzr8f\" (UID: \"52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff\") " pod="ai-tenant-e2e-mt-3536bbfe/e2e-lifecycle-model-3536bbfe-kserve-bf6ccc5bc-mzr8f" Jul 05 13:47:54.303581 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:54.303513 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff-home\") pod \"e2e-lifecycle-model-3536bbfe-kserve-bf6ccc5bc-mzr8f\" (UID: \"52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff\") " pod="ai-tenant-e2e-mt-3536bbfe/e2e-lifecycle-model-3536bbfe-kserve-bf6ccc5bc-mzr8f" Jul 05 13:47:54.303581 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:54.303535 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff-tmp-dir\") pod \"e2e-lifecycle-model-3536bbfe-kserve-bf6ccc5bc-mzr8f\" (UID: \"52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff\") " pod="ai-tenant-e2e-mt-3536bbfe/e2e-lifecycle-model-3536bbfe-kserve-bf6ccc5bc-mzr8f" Jul 05 13:47:54.303677 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:54.303648 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff-model-cache\") pod \"e2e-lifecycle-model-3536bbfe-kserve-bf6ccc5bc-mzr8f\" (UID: \"52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff\") " pod="ai-tenant-e2e-mt-3536bbfe/e2e-lifecycle-model-3536bbfe-kserve-bf6ccc5bc-mzr8f" Jul 05 13:47:54.305504 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:54.305483 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff-dshm\") pod \"e2e-lifecycle-model-3536bbfe-kserve-bf6ccc5bc-mzr8f\" (UID: \"52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff\") " pod="ai-tenant-e2e-mt-3536bbfe/e2e-lifecycle-model-3536bbfe-kserve-bf6ccc5bc-mzr8f" Jul 05 13:47:54.305729 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:54.305708 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff-tls-certs\") pod \"e2e-lifecycle-model-3536bbfe-kserve-bf6ccc5bc-mzr8f\" (UID: \"52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff\") " pod="ai-tenant-e2e-mt-3536bbfe/e2e-lifecycle-model-3536bbfe-kserve-bf6ccc5bc-mzr8f" Jul 05 13:47:54.315262 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:54.315238 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6k29n\" (UniqueName: \"kubernetes.io/projected/52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff-kube-api-access-6k29n\") pod \"e2e-lifecycle-model-3536bbfe-kserve-bf6ccc5bc-mzr8f\" (UID: \"52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff\") " pod="ai-tenant-e2e-mt-3536bbfe/e2e-lifecycle-model-3536bbfe-kserve-bf6ccc5bc-mzr8f" Jul 05 13:47:54.384943 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:54.384854 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-3536bbfe/e2e-lifecycle-model-3536bbfe-kserve-bf6ccc5bc-mzr8f" Jul 05 13:47:54.520214 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:54.520190 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-3536bbfe/e2e-lifecycle-model-3536bbfe-kserve-bf6ccc5bc-mzr8f"] Jul 05 13:47:54.522316 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:47:54.522288 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod52b17577_dae7_4bd6_ba8b_9b6efcb2a7ff.slice/crio-9da1cb2604e57190f03ec508165a59eecbd6be670ad46558ff2a90c6dc4b0f9b WatchSource:0}: Error finding container 9da1cb2604e57190f03ec508165a59eecbd6be670ad46558ff2a90c6dc4b0f9b: Status 404 returned error can't find the container with id 9da1cb2604e57190f03ec508165a59eecbd6be670ad46558ff2a90c6dc4b0f9b Jul 05 13:47:55.264106 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:55.264063 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-3536bbfe/e2e-lifecycle-model-3536bbfe-kserve-bf6ccc5bc-mzr8f" event={"ID":"52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff","Type":"ContainerStarted","Data":"b5c34e04b0733201f4a7638091009204c24d61c43bd52ac1c14b3bd62c8dbe29"} Jul 05 13:47:55.264565 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:55.264115 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-3536bbfe/e2e-lifecycle-model-3536bbfe-kserve-bf6ccc5bc-mzr8f" event={"ID":"52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff","Type":"ContainerStarted","Data":"9da1cb2604e57190f03ec508165a59eecbd6be670ad46558ff2a90c6dc4b0f9b"} Jul 05 13:47:55.289828 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:55.289776 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-3536bbfe/e2e-lifecycle-model-3536bbfe-kserve-bf6ccc5bc-mzr8f" podStartSLOduration=1.289761843 podStartE2EDuration="1.289761843s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:47:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:47:55.287853083 +0000 UTC m=+1748.183511173" watchObservedRunningTime="2026-07-05 13:47:55.289761843 +0000 UTC m=+1748.185419948" Jul 05 13:47:57.261111 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:47:57.261074 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:48:04.385777 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:48:04.385741 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-3536bbfe/e2e-lifecycle-model-3536bbfe-kserve-bf6ccc5bc-mzr8f" Jul 05 13:48:04.386277 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:48:04.385794 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-3536bbfe/e2e-lifecycle-model-3536bbfe-kserve-bf6ccc5bc-mzr8f" Jul 05 13:48:04.398379 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:48:04.398358 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-3536bbfe/e2e-lifecycle-model-3536bbfe-kserve-bf6ccc5bc-mzr8f" Jul 05 13:48:05.323519 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:48:05.323486 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-3536bbfe/e2e-lifecycle-model-3536bbfe-kserve-bf6ccc5bc-mzr8f" Jul 05 13:48:08.726028 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:48:08.725976 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:48:08.827777 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:48:08.827741 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:48:11.960846 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:48:11.960812 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:48:13.885350 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:48:13.885316 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:48:14.389385 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:48:14.389352 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:48:14.490683 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:48:14.490640 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:48:14.678975 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:48:14.678869 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:48:19.992674 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:48:19.992630 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:48:24.959258 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:48:24.959220 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:48:24.981030 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:48:24.980998 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-3536bbfe/e2e-lifecycle-model-3536bbfe-kserve-bf6ccc5bc-mzr8f"] Jul 05 13:48:25.079702 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:48:25.079669 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:48:40.000584 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:48:40.000473 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-3536bbfe/e2e-lifecycle-model-3536bbfe-kserve-bf6ccc5bc-mzr8f" podUID="52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff" containerName="main" containerID="cri-o://b5c34e04b0733201f4a7638091009204c24d61c43bd52ac1c14b3bd62c8dbe29" gracePeriod=45 Jul 05 13:48:40.255377 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:48:40.255316 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-3536bbfe/e2e-lifecycle-model-3536bbfe-kserve-bf6ccc5bc-mzr8f" Jul 05 13:48:40.285653 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:48:40.285618 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff-model-cache\") pod \"52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff\" (UID: \"52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff\") " Jul 05 13:48:40.285822 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:48:40.285668 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff-tmp-dir\") pod \"52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff\" (UID: \"52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff\") " Jul 05 13:48:40.285822 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:48:40.285695 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6k29n\" (UniqueName: \"kubernetes.io/projected/52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff-kube-api-access-6k29n\") pod \"52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff\" (UID: \"52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff\") " Jul 05 13:48:40.285822 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:48:40.285763 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff-dshm\") pod \"52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff\" (UID: \"52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff\") " Jul 05 13:48:40.285822 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:48:40.285785 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff-tls-certs\") pod \"52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff\" (UID: \"52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff\") " Jul 05 13:48:40.285822 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:48:40.285806 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff-home\") pod \"52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff\" (UID: \"52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff\") " Jul 05 13:48:40.286130 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:48:40.285908 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff-model-cache" (OuterVolumeSpecName: "model-cache") pod "52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff" (UID: "52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:48:40.286130 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:48:40.285919 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff" (UID: "52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:48:40.286248 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:48:40.286144 2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff-model-cache\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:48:40.286248 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:48:40.286151 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff-home" (OuterVolumeSpecName: "home") pod "52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff" (UID: "52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:48:40.286248 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:48:40.286177 2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff-tmp-dir\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:48:40.288230 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:48:40.288195 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff-dshm" (OuterVolumeSpecName: "dshm") pod "52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff" (UID: "52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:48:40.288393 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:48:40.288368 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff" (UID: "52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:48:40.288460 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:48:40.288385 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff-kube-api-access-6k29n" (OuterVolumeSpecName: "kube-api-access-6k29n") pod "52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff" (UID: "52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff"). InnerVolumeSpecName "kube-api-access-6k29n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:48:40.387054 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:48:40.387018 2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff-dshm\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:48:40.387054 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:48:40.387046 2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff-tls-certs\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:48:40.387054 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:48:40.387057 2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff-home\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:48:40.387277 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:48:40.387066 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6k29n\" (UniqueName: \"kubernetes.io/projected/52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff-kube-api-access-6k29n\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:48:40.487660 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:48:40.487624 2573 generic.go:356] "Generic (PLEG): container finished" podID="52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff" containerID="b5c34e04b0733201f4a7638091009204c24d61c43bd52ac1c14b3bd62c8dbe29" exitCode=0 Jul 05 13:48:40.487816 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:48:40.487666 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-3536bbfe/e2e-lifecycle-model-3536bbfe-kserve-bf6ccc5bc-mzr8f" Jul 05 13:48:40.487816 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:48:40.487713 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-3536bbfe/e2e-lifecycle-model-3536bbfe-kserve-bf6ccc5bc-mzr8f" event={"ID":"52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff","Type":"ContainerDied","Data":"b5c34e04b0733201f4a7638091009204c24d61c43bd52ac1c14b3bd62c8dbe29"} Jul 05 13:48:40.487816 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:48:40.487752 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-3536bbfe/e2e-lifecycle-model-3536bbfe-kserve-bf6ccc5bc-mzr8f" event={"ID":"52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff","Type":"ContainerDied","Data":"9da1cb2604e57190f03ec508165a59eecbd6be670ad46558ff2a90c6dc4b0f9b"} Jul 05 13:48:40.487816 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:48:40.487768 2573 scope.go:117] "RemoveContainer" containerID="b5c34e04b0733201f4a7638091009204c24d61c43bd52ac1c14b3bd62c8dbe29" Jul 05 13:48:40.496568 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:48:40.496552 2573 scope.go:117] "RemoveContainer" containerID="b5c34e04b0733201f4a7638091009204c24d61c43bd52ac1c14b3bd62c8dbe29" Jul 05 13:48:40.496829 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:48:40.496812 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5c34e04b0733201f4a7638091009204c24d61c43bd52ac1c14b3bd62c8dbe29\": container with ID starting with b5c34e04b0733201f4a7638091009204c24d61c43bd52ac1c14b3bd62c8dbe29 not found: ID does not exist" containerID="b5c34e04b0733201f4a7638091009204c24d61c43bd52ac1c14b3bd62c8dbe29" Jul 05 13:48:40.496880 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:48:40.496838 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5c34e04b0733201f4a7638091009204c24d61c43bd52ac1c14b3bd62c8dbe29"} err="failed to get container status \"b5c34e04b0733201f4a7638091009204c24d61c43bd52ac1c14b3bd62c8dbe29\": rpc error: code = NotFound desc = could not find container \"b5c34e04b0733201f4a7638091009204c24d61c43bd52ac1c14b3bd62c8dbe29\": container with ID starting with b5c34e04b0733201f4a7638091009204c24d61c43bd52ac1c14b3bd62c8dbe29 not found: ID does not exist" Jul 05 13:48:40.519751 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:48:40.519695 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-3536bbfe/e2e-lifecycle-model-3536bbfe-kserve-bf6ccc5bc-mzr8f"] Jul 05 13:48:40.523403 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:48:40.523371 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-3536bbfe/e2e-lifecycle-model-3536bbfe-kserve-bf6ccc5bc-mzr8f"] Jul 05 13:48:41.735183 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:48:41.735148 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff" path="/var/lib/kubelet/pods/52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff/volumes" Jul 05 13:49:01.453878 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:49:01.453812 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:49:02.335769 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:49:02.335715 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:49:03.024735 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:49:03.024686 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:49:04.866380 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:49:04.866332 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:49:09.819574 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:49:09.819545 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:49:09.915237 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:49:09.915200 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:49:10.130345 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:49:10.130252 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:49:10.341560 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:49:10.341505 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:49:11.379604 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:49:11.379569 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:49:11.476492 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:49:11.476465 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:49:11.590989 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:49:11.590936 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:49:12.392884 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:49:12.392844 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:49:12.501455 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:49:12.501418 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:49:13.869715 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:49:13.869683 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:49:16.202601 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:49:16.202569 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:49:21.515492 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:49:21.515457 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:49:21.715708 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:49:21.715673 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:49:22.216076 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:49:22.216037 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:49:29.286017 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:49:29.285980 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:49:29.884283 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:49:29.884248 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:49:29.995007 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:49:29.994975 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:49:30.129882 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:49:30.129850 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:49:40.691159 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:49:40.691123 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:49:40.986774 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:49:40.986680 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:50:01.981586 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:01.981544 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:50:08.828421 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:08.828380 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:50:08.948748 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:08.948701 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:50:09.183590 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:09.183553 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:50:09.588802 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:09.588677 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:50:09.694296 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:09.694266 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:50:12.732263 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:12.732221 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:50:12.854564 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:12.854528 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:50:13.072043 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:13.071923 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:50:13.289546 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:13.289501 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:50:13.819505 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:13.819470 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:50:13.917602 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:13.917561 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:50:17.270214 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:17.270176 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:50:17.637965 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:17.637862 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:50:17.996652 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:17.996615 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:50:18.093539 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:18.093502 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:50:20.460791 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:20.460755 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:50:20.796761 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:20.796669 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:50:20.898330 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:20.898296 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:50:20.992209 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:20.992171 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:50:21.130660 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:21.130569 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:50:21.517242 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:21.517206 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:50:21.623524 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:21.623493 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:50:25.229076 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:25.229038 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:50:29.582037 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:29.582002 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-76289d/test-model-76289d-kserve-77cc54845-f4gwq"] Jul 05 13:50:29.582441 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:29.582332 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff" containerName="main" Jul 05 13:50:29.582441 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:29.582344 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff" containerName="main" Jul 05 13:50:29.582441 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:29.582416 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="52b17577-dae7-4bd6-ba8b-9b6efcb2a7ff" containerName="main" Jul 05 13:50:29.585099 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:29.585080 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-76289d/test-model-76289d-kserve-77cc54845-f4gwq" Jul 05 13:50:29.589153 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:29.589129 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-76289d\"/\"kube-root-ca.crt\"" Jul 05 13:50:29.589288 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:29.589182 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-76289d\"/\"test-model-76289d-kserve-self-signed-certs\"" Jul 05 13:50:29.589348 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:29.589326 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-76289d\"/\"default-dockercfg-5cpng\"" Jul 05 13:50:29.589417 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:29.589403 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-76289d\"/\"openshift-service-ca.crt\"" Jul 05 13:50:29.605307 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:29.605283 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-76289d/test-model-76289d-kserve-77cc54845-f4gwq"] Jul 05 13:50:29.627414 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:29.627374 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:50:29.731470 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:29.731437 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e57524f6-9b1e-453d-af77-e40e7afd59b8-model-cache\") pod \"test-model-76289d-kserve-77cc54845-f4gwq\" (UID: \"e57524f6-9b1e-453d-af77-e40e7afd59b8\") " pod="ai-tenant-e2e-inf-a-76289d/test-model-76289d-kserve-77cc54845-f4gwq" Jul 05 13:50:29.731635 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:29.731477 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e57524f6-9b1e-453d-af77-e40e7afd59b8-tls-certs\") pod \"test-model-76289d-kserve-77cc54845-f4gwq\" (UID: \"e57524f6-9b1e-453d-af77-e40e7afd59b8\") " pod="ai-tenant-e2e-inf-a-76289d/test-model-76289d-kserve-77cc54845-f4gwq" Jul 05 13:50:29.731635 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:29.731509 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e57524f6-9b1e-453d-af77-e40e7afd59b8-dshm\") pod \"test-model-76289d-kserve-77cc54845-f4gwq\" (UID: \"e57524f6-9b1e-453d-af77-e40e7afd59b8\") " pod="ai-tenant-e2e-inf-a-76289d/test-model-76289d-kserve-77cc54845-f4gwq" Jul 05 13:50:29.731635 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:29.731525 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e57524f6-9b1e-453d-af77-e40e7afd59b8-tmp-dir\") pod \"test-model-76289d-kserve-77cc54845-f4gwq\" (UID: \"e57524f6-9b1e-453d-af77-e40e7afd59b8\") " pod="ai-tenant-e2e-inf-a-76289d/test-model-76289d-kserve-77cc54845-f4gwq" Jul 05 13:50:29.731635 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:29.731621 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlc2s\" (UniqueName: \"kubernetes.io/projected/e57524f6-9b1e-453d-af77-e40e7afd59b8-kube-api-access-wlc2s\") pod \"test-model-76289d-kserve-77cc54845-f4gwq\" (UID: \"e57524f6-9b1e-453d-af77-e40e7afd59b8\") " pod="ai-tenant-e2e-inf-a-76289d/test-model-76289d-kserve-77cc54845-f4gwq" Jul 05 13:50:29.731776 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:29.731658 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e57524f6-9b1e-453d-af77-e40e7afd59b8-home\") pod \"test-model-76289d-kserve-77cc54845-f4gwq\" (UID: \"e57524f6-9b1e-453d-af77-e40e7afd59b8\") " pod="ai-tenant-e2e-inf-a-76289d/test-model-76289d-kserve-77cc54845-f4gwq" Jul 05 13:50:29.832825 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:29.832733 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e57524f6-9b1e-453d-af77-e40e7afd59b8-home\") pod \"test-model-76289d-kserve-77cc54845-f4gwq\" (UID: \"e57524f6-9b1e-453d-af77-e40e7afd59b8\") " pod="ai-tenant-e2e-inf-a-76289d/test-model-76289d-kserve-77cc54845-f4gwq" Jul 05 13:50:29.832825 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:29.832788 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e57524f6-9b1e-453d-af77-e40e7afd59b8-model-cache\") pod \"test-model-76289d-kserve-77cc54845-f4gwq\" (UID: \"e57524f6-9b1e-453d-af77-e40e7afd59b8\") " pod="ai-tenant-e2e-inf-a-76289d/test-model-76289d-kserve-77cc54845-f4gwq" Jul 05 13:50:29.832825 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:29.832814 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e57524f6-9b1e-453d-af77-e40e7afd59b8-tls-certs\") pod \"test-model-76289d-kserve-77cc54845-f4gwq\" (UID: \"e57524f6-9b1e-453d-af77-e40e7afd59b8\") " pod="ai-tenant-e2e-inf-a-76289d/test-model-76289d-kserve-77cc54845-f4gwq" Jul 05 13:50:29.833155 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:29.832848 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e57524f6-9b1e-453d-af77-e40e7afd59b8-dshm\") pod \"test-model-76289d-kserve-77cc54845-f4gwq\" (UID: \"e57524f6-9b1e-453d-af77-e40e7afd59b8\") " pod="ai-tenant-e2e-inf-a-76289d/test-model-76289d-kserve-77cc54845-f4gwq" Jul 05 13:50:29.833155 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:29.832865 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e57524f6-9b1e-453d-af77-e40e7afd59b8-tmp-dir\") pod \"test-model-76289d-kserve-77cc54845-f4gwq\" (UID: \"e57524f6-9b1e-453d-af77-e40e7afd59b8\") " pod="ai-tenant-e2e-inf-a-76289d/test-model-76289d-kserve-77cc54845-f4gwq" Jul 05 13:50:29.833155 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:29.832930 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlc2s\" (UniqueName: \"kubernetes.io/projected/e57524f6-9b1e-453d-af77-e40e7afd59b8-kube-api-access-wlc2s\") pod \"test-model-76289d-kserve-77cc54845-f4gwq\" (UID: \"e57524f6-9b1e-453d-af77-e40e7afd59b8\") " pod="ai-tenant-e2e-inf-a-76289d/test-model-76289d-kserve-77cc54845-f4gwq" Jul 05 13:50:29.833322 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:29.833209 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e57524f6-9b1e-453d-af77-e40e7afd59b8-home\") pod \"test-model-76289d-kserve-77cc54845-f4gwq\" (UID: \"e57524f6-9b1e-453d-af77-e40e7afd59b8\") " pod="ai-tenant-e2e-inf-a-76289d/test-model-76289d-kserve-77cc54845-f4gwq" Jul 05 13:50:29.833322 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:29.833253 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e57524f6-9b1e-453d-af77-e40e7afd59b8-model-cache\") pod \"test-model-76289d-kserve-77cc54845-f4gwq\" (UID: \"e57524f6-9b1e-453d-af77-e40e7afd59b8\") " pod="ai-tenant-e2e-inf-a-76289d/test-model-76289d-kserve-77cc54845-f4gwq" Jul 05 13:50:29.833416 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:29.833320 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e57524f6-9b1e-453d-af77-e40e7afd59b8-tmp-dir\") pod \"test-model-76289d-kserve-77cc54845-f4gwq\" (UID: \"e57524f6-9b1e-453d-af77-e40e7afd59b8\") " pod="ai-tenant-e2e-inf-a-76289d/test-model-76289d-kserve-77cc54845-f4gwq" Jul 05 13:50:29.835187 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:29.835162 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e57524f6-9b1e-453d-af77-e40e7afd59b8-dshm\") pod \"test-model-76289d-kserve-77cc54845-f4gwq\" (UID: \"e57524f6-9b1e-453d-af77-e40e7afd59b8\") " pod="ai-tenant-e2e-inf-a-76289d/test-model-76289d-kserve-77cc54845-f4gwq" Jul 05 13:50:29.835332 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:29.835315 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e57524f6-9b1e-453d-af77-e40e7afd59b8-tls-certs\") pod \"test-model-76289d-kserve-77cc54845-f4gwq\" (UID: \"e57524f6-9b1e-453d-af77-e40e7afd59b8\") " pod="ai-tenant-e2e-inf-a-76289d/test-model-76289d-kserve-77cc54845-f4gwq" Jul 05 13:50:29.844517 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:29.844491 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlc2s\" (UniqueName: \"kubernetes.io/projected/e57524f6-9b1e-453d-af77-e40e7afd59b8-kube-api-access-wlc2s\") pod \"test-model-76289d-kserve-77cc54845-f4gwq\" (UID: \"e57524f6-9b1e-453d-af77-e40e7afd59b8\") " pod="ai-tenant-e2e-inf-a-76289d/test-model-76289d-kserve-77cc54845-f4gwq" Jul 05 13:50:29.893622 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:29.893589 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-76289d/test-model-76289d-kserve-77cc54845-f4gwq" Jul 05 13:50:30.034402 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:30.034381 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-76289d/test-model-76289d-kserve-77cc54845-f4gwq"] Jul 05 13:50:30.035968 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:50:30.035925 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode57524f6_9b1e_453d_af77_e40e7afd59b8.slice/crio-c2c6ede9d62ee4482bc148e82c1475d390a8335ef3d29128b8672775dd7285fd WatchSource:0}: Error finding container c2c6ede9d62ee4482bc148e82c1475d390a8335ef3d29128b8672775dd7285fd: Status 404 returned error can't find the container with id c2c6ede9d62ee4482bc148e82c1475d390a8335ef3d29128b8672775dd7285fd Jul 05 13:50:31.023239 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:31.023203 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-76289d/test-model-76289d-kserve-77cc54845-f4gwq" event={"ID":"e57524f6-9b1e-453d-af77-e40e7afd59b8","Type":"ContainerStarted","Data":"b9952be236dd20f4a0899ed318f8b8c49236528d41c8e3c30263575614d28cd0"} Jul 05 13:50:31.023239 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:31.023245 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-76289d/test-model-76289d-kserve-77cc54845-f4gwq" event={"ID":"e57524f6-9b1e-453d-af77-e40e7afd59b8","Type":"ContainerStarted","Data":"c2c6ede9d62ee4482bc148e82c1475d390a8335ef3d29128b8672775dd7285fd"} Jul 05 13:50:31.049659 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:31.049611 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-76289d/test-model-76289d-kserve-77cc54845-f4gwq" podStartSLOduration=2.049595154 podStartE2EDuration="2.049595154s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:50:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:50:31.047876753 +0000 UTC m=+1903.943534839" watchObservedRunningTime="2026-07-05 13:50:31.049595154 +0000 UTC m=+1903.945253276" Jul 05 13:50:34.644671 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:34.644628 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:50:34.745291 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:34.745257 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:50:34.971836 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:34.971797 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:50:39.894541 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:39.894507 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-76289d/test-model-76289d-kserve-77cc54845-f4gwq" Jul 05 13:50:39.894938 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:39.894558 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-76289d/test-model-76289d-kserve-77cc54845-f4gwq" Jul 05 13:50:39.907288 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:39.907267 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-76289d/test-model-76289d-kserve-77cc54845-f4gwq" Jul 05 13:50:40.085289 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:40.085261 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-76289d/test-model-76289d-kserve-77cc54845-f4gwq" Jul 05 13:50:42.397531 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:42.397493 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:50:42.498652 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:42.498618 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:50:46.476106 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:50:46.476068 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:51:14.071293 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:14.071259 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-4b5487/test-model-4b5487-kserve-5b8cd5bd5c-rsmtb"] Jul 05 13:51:14.074600 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:14.074583 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-4b5487/test-model-4b5487-kserve-5b8cd5bd5c-rsmtb" Jul 05 13:51:14.077857 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:14.077830 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-4b5487\"/\"kube-root-ca.crt\"" Jul 05 13:51:14.078001 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:14.077921 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-4b5487\"/\"openshift-service-ca.crt\"" Jul 05 13:51:14.078001 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:14.077926 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-4b5487\"/\"test-model-4b5487-kserve-self-signed-certs\"" Jul 05 13:51:14.078001 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:14.077989 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-4b5487\"/\"default-dockercfg-br2x8\"" Jul 05 13:51:14.084007 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:14.083984 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-4b5487/test-model-4b5487-kserve-5b8cd5bd5c-rsmtb"] Jul 05 13:51:14.111297 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:14.111258 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1352c381-9193-4370-96c6-cea79a756bd7-tmp-dir\") pod \"test-model-4b5487-kserve-5b8cd5bd5c-rsmtb\" (UID: \"1352c381-9193-4370-96c6-cea79a756bd7\") " pod="ai-tenant-e2e-inf-b-4b5487/test-model-4b5487-kserve-5b8cd5bd5c-rsmtb" Jul 05 13:51:14.111475 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:14.111314 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1352c381-9193-4370-96c6-cea79a756bd7-tls-certs\") pod \"test-model-4b5487-kserve-5b8cd5bd5c-rsmtb\" (UID: \"1352c381-9193-4370-96c6-cea79a756bd7\") " pod="ai-tenant-e2e-inf-b-4b5487/test-model-4b5487-kserve-5b8cd5bd5c-rsmtb" Jul 05 13:51:14.111475 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:14.111405 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1352c381-9193-4370-96c6-cea79a756bd7-model-cache\") pod \"test-model-4b5487-kserve-5b8cd5bd5c-rsmtb\" (UID: \"1352c381-9193-4370-96c6-cea79a756bd7\") " pod="ai-tenant-e2e-inf-b-4b5487/test-model-4b5487-kserve-5b8cd5bd5c-rsmtb" Jul 05 13:51:14.111475 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:14.111453 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1352c381-9193-4370-96c6-cea79a756bd7-dshm\") pod \"test-model-4b5487-kserve-5b8cd5bd5c-rsmtb\" (UID: \"1352c381-9193-4370-96c6-cea79a756bd7\") " pod="ai-tenant-e2e-inf-b-4b5487/test-model-4b5487-kserve-5b8cd5bd5c-rsmtb" Jul 05 13:51:14.111475 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:14.111472 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1352c381-9193-4370-96c6-cea79a756bd7-home\") pod \"test-model-4b5487-kserve-5b8cd5bd5c-rsmtb\" (UID: \"1352c381-9193-4370-96c6-cea79a756bd7\") " pod="ai-tenant-e2e-inf-b-4b5487/test-model-4b5487-kserve-5b8cd5bd5c-rsmtb" Jul 05 13:51:14.111714 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:14.111487 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltzrn\" (UniqueName: \"kubernetes.io/projected/1352c381-9193-4370-96c6-cea79a756bd7-kube-api-access-ltzrn\") pod \"test-model-4b5487-kserve-5b8cd5bd5c-rsmtb\" (UID: \"1352c381-9193-4370-96c6-cea79a756bd7\") " pod="ai-tenant-e2e-inf-b-4b5487/test-model-4b5487-kserve-5b8cd5bd5c-rsmtb" Jul 05 13:51:14.151084 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:14.151049 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:51:14.212305 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:14.212278 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1352c381-9193-4370-96c6-cea79a756bd7-model-cache\") pod \"test-model-4b5487-kserve-5b8cd5bd5c-rsmtb\" (UID: \"1352c381-9193-4370-96c6-cea79a756bd7\") " pod="ai-tenant-e2e-inf-b-4b5487/test-model-4b5487-kserve-5b8cd5bd5c-rsmtb" Jul 05 13:51:14.212465 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:14.212325 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1352c381-9193-4370-96c6-cea79a756bd7-dshm\") pod \"test-model-4b5487-kserve-5b8cd5bd5c-rsmtb\" (UID: \"1352c381-9193-4370-96c6-cea79a756bd7\") " pod="ai-tenant-e2e-inf-b-4b5487/test-model-4b5487-kserve-5b8cd5bd5c-rsmtb" Jul 05 13:51:14.212465 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:14.212438 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1352c381-9193-4370-96c6-cea79a756bd7-home\") pod \"test-model-4b5487-kserve-5b8cd5bd5c-rsmtb\" (UID: \"1352c381-9193-4370-96c6-cea79a756bd7\") " pod="ai-tenant-e2e-inf-b-4b5487/test-model-4b5487-kserve-5b8cd5bd5c-rsmtb" Jul 05 13:51:14.212544 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:14.212468 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltzrn\" (UniqueName: \"kubernetes.io/projected/1352c381-9193-4370-96c6-cea79a756bd7-kube-api-access-ltzrn\") pod \"test-model-4b5487-kserve-5b8cd5bd5c-rsmtb\" (UID: \"1352c381-9193-4370-96c6-cea79a756bd7\") " pod="ai-tenant-e2e-inf-b-4b5487/test-model-4b5487-kserve-5b8cd5bd5c-rsmtb" Jul 05 13:51:14.212696 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:14.212673 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1352c381-9193-4370-96c6-cea79a756bd7-tmp-dir\") pod \"test-model-4b5487-kserve-5b8cd5bd5c-rsmtb\" (UID: \"1352c381-9193-4370-96c6-cea79a756bd7\") " pod="ai-tenant-e2e-inf-b-4b5487/test-model-4b5487-kserve-5b8cd5bd5c-rsmtb" Jul 05 13:51:14.212760 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:14.212693 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1352c381-9193-4370-96c6-cea79a756bd7-model-cache\") pod \"test-model-4b5487-kserve-5b8cd5bd5c-rsmtb\" (UID: \"1352c381-9193-4370-96c6-cea79a756bd7\") " pod="ai-tenant-e2e-inf-b-4b5487/test-model-4b5487-kserve-5b8cd5bd5c-rsmtb" Jul 05 13:51:14.212760 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:14.212729 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1352c381-9193-4370-96c6-cea79a756bd7-tls-certs\") pod \"test-model-4b5487-kserve-5b8cd5bd5c-rsmtb\" (UID: \"1352c381-9193-4370-96c6-cea79a756bd7\") " pod="ai-tenant-e2e-inf-b-4b5487/test-model-4b5487-kserve-5b8cd5bd5c-rsmtb" Jul 05 13:51:14.212760 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:14.212750 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1352c381-9193-4370-96c6-cea79a756bd7-home\") pod \"test-model-4b5487-kserve-5b8cd5bd5c-rsmtb\" (UID: \"1352c381-9193-4370-96c6-cea79a756bd7\") " pod="ai-tenant-e2e-inf-b-4b5487/test-model-4b5487-kserve-5b8cd5bd5c-rsmtb" Jul 05 13:51:14.212919 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:14.212903 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1352c381-9193-4370-96c6-cea79a756bd7-tmp-dir\") pod \"test-model-4b5487-kserve-5b8cd5bd5c-rsmtb\" (UID: \"1352c381-9193-4370-96c6-cea79a756bd7\") " pod="ai-tenant-e2e-inf-b-4b5487/test-model-4b5487-kserve-5b8cd5bd5c-rsmtb" Jul 05 13:51:14.214563 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:14.214544 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1352c381-9193-4370-96c6-cea79a756bd7-dshm\") pod \"test-model-4b5487-kserve-5b8cd5bd5c-rsmtb\" (UID: \"1352c381-9193-4370-96c6-cea79a756bd7\") " pod="ai-tenant-e2e-inf-b-4b5487/test-model-4b5487-kserve-5b8cd5bd5c-rsmtb" Jul 05 13:51:14.215060 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:14.215043 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1352c381-9193-4370-96c6-cea79a756bd7-tls-certs\") pod \"test-model-4b5487-kserve-5b8cd5bd5c-rsmtb\" (UID: \"1352c381-9193-4370-96c6-cea79a756bd7\") " pod="ai-tenant-e2e-inf-b-4b5487/test-model-4b5487-kserve-5b8cd5bd5c-rsmtb" Jul 05 13:51:14.221721 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:14.221698 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltzrn\" (UniqueName: \"kubernetes.io/projected/1352c381-9193-4370-96c6-cea79a756bd7-kube-api-access-ltzrn\") pod \"test-model-4b5487-kserve-5b8cd5bd5c-rsmtb\" (UID: \"1352c381-9193-4370-96c6-cea79a756bd7\") " pod="ai-tenant-e2e-inf-b-4b5487/test-model-4b5487-kserve-5b8cd5bd5c-rsmtb" Jul 05 13:51:14.384308 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:14.384217 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-4b5487/test-model-4b5487-kserve-5b8cd5bd5c-rsmtb" Jul 05 13:51:14.519130 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:14.519082 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-4b5487/test-model-4b5487-kserve-5b8cd5bd5c-rsmtb"] Jul 05 13:51:14.520046 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:51:14.520022 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1352c381_9193_4370_96c6_cea79a756bd7.slice/crio-c8a4032571a0360538918bb16fe9de10611205437f9094b62e95bd80f57ec402 WatchSource:0}: Error finding container c8a4032571a0360538918bb16fe9de10611205437f9094b62e95bd80f57ec402: Status 404 returned error can't find the container with id c8a4032571a0360538918bb16fe9de10611205437f9094b62e95bd80f57ec402 Jul 05 13:51:15.243023 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:15.242986 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-4b5487/test-model-4b5487-kserve-5b8cd5bd5c-rsmtb" event={"ID":"1352c381-9193-4370-96c6-cea79a756bd7","Type":"ContainerStarted","Data":"e6475abaeb913cd1d30718cf8e3c03b982413b165324f1c501fc406d708ea57f"} Jul 05 13:51:15.243393 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:15.243034 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-4b5487/test-model-4b5487-kserve-5b8cd5bd5c-rsmtb" event={"ID":"1352c381-9193-4370-96c6-cea79a756bd7","Type":"ContainerStarted","Data":"c8a4032571a0360538918bb16fe9de10611205437f9094b62e95bd80f57ec402"} Jul 05 13:51:15.263189 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:15.263138 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-4b5487/test-model-4b5487-kserve-5b8cd5bd5c-rsmtb" podStartSLOduration=1.26312009 podStartE2EDuration="1.26312009s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:51:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:51:15.261271529 +0000 UTC m=+1948.156929619" watchObservedRunningTime="2026-07-05 13:51:15.26312009 +0000 UTC m=+1948.158778178" Jul 05 13:51:17.756562 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:17.756527 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:51:24.384494 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:24.384436 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-4b5487/test-model-4b5487-kserve-5b8cd5bd5c-rsmtb" Jul 05 13:51:24.384494 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:24.384503 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-4b5487/test-model-4b5487-kserve-5b8cd5bd5c-rsmtb" Jul 05 13:51:24.397143 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:24.397114 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-4b5487/test-model-4b5487-kserve-5b8cd5bd5c-rsmtb" Jul 05 13:51:25.306742 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:25.306716 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-4b5487/test-model-4b5487-kserve-5b8cd5bd5c-rsmtb" Jul 05 13:51:31.184621 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:31.184582 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:51:34.175668 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:34.175632 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-76289d/test-model-76289d-kserve-77cc54845-f4gwq"] Jul 05 13:51:34.212410 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:34.212382 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:51:35.257008 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:35.256970 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:51:35.257439 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:35.257414 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-4b5487/test-model-4b5487-kserve-5b8cd5bd5c-rsmtb"] Jul 05 13:51:35.397429 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:35.397395 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:51:35.766375 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:35.766330 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:51:35.887060 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:35.887028 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:51:35.987068 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:35.987032 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:51:36.171417 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:36.171330 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:51:38.213338 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:38.213303 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:51:40.846560 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:40.846473 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:51:45.244189 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:45.244156 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:51:45.334491 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:45.334440 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:51:46.976438 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:46.976405 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:51:49.195284 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:49.195243 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-76289d/test-model-76289d-kserve-77cc54845-f4gwq" podUID="e57524f6-9b1e-453d-af77-e40e7afd59b8" containerName="main" containerID="cri-o://b9952be236dd20f4a0899ed318f8b8c49236528d41c8e3c30263575614d28cd0" gracePeriod=45 Jul 05 13:51:49.422227 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:49.422190 2573 generic.go:356] "Generic (PLEG): container finished" podID="e57524f6-9b1e-453d-af77-e40e7afd59b8" containerID="b9952be236dd20f4a0899ed318f8b8c49236528d41c8e3c30263575614d28cd0" exitCode=0 Jul 05 13:51:49.422377 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:49.422268 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-76289d/test-model-76289d-kserve-77cc54845-f4gwq" event={"ID":"e57524f6-9b1e-453d-af77-e40e7afd59b8","Type":"ContainerDied","Data":"b9952be236dd20f4a0899ed318f8b8c49236528d41c8e3c30263575614d28cd0"} Jul 05 13:51:49.422377 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:49.422322 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-76289d/test-model-76289d-kserve-77cc54845-f4gwq" event={"ID":"e57524f6-9b1e-453d-af77-e40e7afd59b8","Type":"ContainerDied","Data":"c2c6ede9d62ee4482bc148e82c1475d390a8335ef3d29128b8672775dd7285fd"} Jul 05 13:51:49.422377 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:49.422338 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c2c6ede9d62ee4482bc148e82c1475d390a8335ef3d29128b8672775dd7285fd" Jul 05 13:51:49.436268 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:49.436243 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-76289d/test-model-76289d-kserve-77cc54845-f4gwq" Jul 05 13:51:49.630576 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:49.630498 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e57524f6-9b1e-453d-af77-e40e7afd59b8-dshm\") pod \"e57524f6-9b1e-453d-af77-e40e7afd59b8\" (UID: \"e57524f6-9b1e-453d-af77-e40e7afd59b8\") " Jul 05 13:51:49.630576 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:49.630536 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wlc2s\" (UniqueName: \"kubernetes.io/projected/e57524f6-9b1e-453d-af77-e40e7afd59b8-kube-api-access-wlc2s\") pod \"e57524f6-9b1e-453d-af77-e40e7afd59b8\" (UID: \"e57524f6-9b1e-453d-af77-e40e7afd59b8\") " Jul 05 13:51:49.630576 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:49.630560 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e57524f6-9b1e-453d-af77-e40e7afd59b8-home\") pod \"e57524f6-9b1e-453d-af77-e40e7afd59b8\" (UID: \"e57524f6-9b1e-453d-af77-e40e7afd59b8\") " Jul 05 13:51:49.630813 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:49.630625 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e57524f6-9b1e-453d-af77-e40e7afd59b8-model-cache\") pod \"e57524f6-9b1e-453d-af77-e40e7afd59b8\" (UID: \"e57524f6-9b1e-453d-af77-e40e7afd59b8\") " Jul 05 13:51:49.630813 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:49.630707 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e57524f6-9b1e-453d-af77-e40e7afd59b8-tls-certs\") pod \"e57524f6-9b1e-453d-af77-e40e7afd59b8\" (UID: \"e57524f6-9b1e-453d-af77-e40e7afd59b8\") " Jul 05 13:51:49.630813 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:49.630752 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e57524f6-9b1e-453d-af77-e40e7afd59b8-tmp-dir\") pod \"e57524f6-9b1e-453d-af77-e40e7afd59b8\" (UID: \"e57524f6-9b1e-453d-af77-e40e7afd59b8\") " Jul 05 13:51:49.631021 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:49.630870 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e57524f6-9b1e-453d-af77-e40e7afd59b8-home" (OuterVolumeSpecName: "home") pod "e57524f6-9b1e-453d-af77-e40e7afd59b8" (UID: "e57524f6-9b1e-453d-af77-e40e7afd59b8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:51:49.631021 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:49.630947 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e57524f6-9b1e-453d-af77-e40e7afd59b8-model-cache" (OuterVolumeSpecName: "model-cache") pod "e57524f6-9b1e-453d-af77-e40e7afd59b8" (UID: "e57524f6-9b1e-453d-af77-e40e7afd59b8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:51:49.631147 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:49.631093 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e57524f6-9b1e-453d-af77-e40e7afd59b8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e57524f6-9b1e-453d-af77-e40e7afd59b8" (UID: "e57524f6-9b1e-453d-af77-e40e7afd59b8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:51:49.631147 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:49.631114 2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e57524f6-9b1e-453d-af77-e40e7afd59b8-home\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:51:49.631147 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:49.631131 2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e57524f6-9b1e-453d-af77-e40e7afd59b8-model-cache\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:51:49.632813 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:49.632780 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e57524f6-9b1e-453d-af77-e40e7afd59b8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e57524f6-9b1e-453d-af77-e40e7afd59b8" (UID: "e57524f6-9b1e-453d-af77-e40e7afd59b8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:51:49.632940 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:49.632837 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e57524f6-9b1e-453d-af77-e40e7afd59b8-dshm" (OuterVolumeSpecName: "dshm") pod "e57524f6-9b1e-453d-af77-e40e7afd59b8" (UID: "e57524f6-9b1e-453d-af77-e40e7afd59b8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:51:49.632940 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:49.632858 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e57524f6-9b1e-453d-af77-e40e7afd59b8-kube-api-access-wlc2s" (OuterVolumeSpecName: "kube-api-access-wlc2s") pod "e57524f6-9b1e-453d-af77-e40e7afd59b8" (UID: "e57524f6-9b1e-453d-af77-e40e7afd59b8"). InnerVolumeSpecName "kube-api-access-wlc2s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:51:49.731675 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:49.731639 2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e57524f6-9b1e-453d-af77-e40e7afd59b8-tmp-dir\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:51:49.731675 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:49.731676 2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e57524f6-9b1e-453d-af77-e40e7afd59b8-dshm\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:51:49.731847 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:49.731690 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wlc2s\" (UniqueName: \"kubernetes.io/projected/e57524f6-9b1e-453d-af77-e40e7afd59b8-kube-api-access-wlc2s\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:51:49.731847 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:49.731707 2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e57524f6-9b1e-453d-af77-e40e7afd59b8-tls-certs\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:51:50.277326 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:50.277290 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-4b5487/test-model-4b5487-kserve-5b8cd5bd5c-rsmtb" podUID="1352c381-9193-4370-96c6-cea79a756bd7" containerName="main" containerID="cri-o://e6475abaeb913cd1d30718cf8e3c03b982413b165324f1c501fc406d708ea57f" gracePeriod=45 Jul 05 13:51:50.429979 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:50.429934 2573 generic.go:356] "Generic (PLEG): container finished" podID="1352c381-9193-4370-96c6-cea79a756bd7" containerID="e6475abaeb913cd1d30718cf8e3c03b982413b165324f1c501fc406d708ea57f" exitCode=0 Jul 05 13:51:50.430159 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:50.430012 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-4b5487/test-model-4b5487-kserve-5b8cd5bd5c-rsmtb" event={"ID":"1352c381-9193-4370-96c6-cea79a756bd7","Type":"ContainerDied","Data":"e6475abaeb913cd1d30718cf8e3c03b982413b165324f1c501fc406d708ea57f"} Jul 05 13:51:50.430159 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:50.430122 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-76289d/test-model-76289d-kserve-77cc54845-f4gwq" Jul 05 13:51:50.445822 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:50.445793 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-76289d/test-model-76289d-kserve-77cc54845-f4gwq"] Jul 05 13:51:50.450445 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:50.450418 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-76289d/test-model-76289d-kserve-77cc54845-f4gwq"] Jul 05 13:51:50.534506 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:50.534442 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-4b5487/test-model-4b5487-kserve-5b8cd5bd5c-rsmtb" Jul 05 13:51:50.537489 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:50.537466 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1352c381-9193-4370-96c6-cea79a756bd7-tmp-dir\") pod \"1352c381-9193-4370-96c6-cea79a756bd7\" (UID: \"1352c381-9193-4370-96c6-cea79a756bd7\") " Jul 05 13:51:50.537611 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:50.537502 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1352c381-9193-4370-96c6-cea79a756bd7-tls-certs\") pod \"1352c381-9193-4370-96c6-cea79a756bd7\" (UID: \"1352c381-9193-4370-96c6-cea79a756bd7\") " Jul 05 13:51:50.537611 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:50.537544 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1352c381-9193-4370-96c6-cea79a756bd7-dshm\") pod \"1352c381-9193-4370-96c6-cea79a756bd7\" (UID: \"1352c381-9193-4370-96c6-cea79a756bd7\") " Jul 05 13:51:50.537611 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:50.537563 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1352c381-9193-4370-96c6-cea79a756bd7-model-cache\") pod \"1352c381-9193-4370-96c6-cea79a756bd7\" (UID: \"1352c381-9193-4370-96c6-cea79a756bd7\") " Jul 05 13:51:50.537611 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:50.537610 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ltzrn\" (UniqueName: \"kubernetes.io/projected/1352c381-9193-4370-96c6-cea79a756bd7-kube-api-access-ltzrn\") pod \"1352c381-9193-4370-96c6-cea79a756bd7\" (UID: \"1352c381-9193-4370-96c6-cea79a756bd7\") " Jul 05 13:51:50.537823 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:50.537643 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1352c381-9193-4370-96c6-cea79a756bd7-home\") pod \"1352c381-9193-4370-96c6-cea79a756bd7\" (UID: \"1352c381-9193-4370-96c6-cea79a756bd7\") " Jul 05 13:51:50.537823 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:50.537746 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1352c381-9193-4370-96c6-cea79a756bd7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1352c381-9193-4370-96c6-cea79a756bd7" (UID: "1352c381-9193-4370-96c6-cea79a756bd7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:51:50.537937 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:50.537913 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1352c381-9193-4370-96c6-cea79a756bd7-model-cache" (OuterVolumeSpecName: "model-cache") pod "1352c381-9193-4370-96c6-cea79a756bd7" (UID: "1352c381-9193-4370-96c6-cea79a756bd7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:51:50.538013 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:50.537973 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1352c381-9193-4370-96c6-cea79a756bd7-home" (OuterVolumeSpecName: "home") pod "1352c381-9193-4370-96c6-cea79a756bd7" (UID: "1352c381-9193-4370-96c6-cea79a756bd7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:51:50.538072 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:50.538013 2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1352c381-9193-4370-96c6-cea79a756bd7-tmp-dir\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:51:50.539780 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:50.539754 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1352c381-9193-4370-96c6-cea79a756bd7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1352c381-9193-4370-96c6-cea79a756bd7" (UID: "1352c381-9193-4370-96c6-cea79a756bd7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:51:50.539780 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:50.539776 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1352c381-9193-4370-96c6-cea79a756bd7-dshm" (OuterVolumeSpecName: "dshm") pod "1352c381-9193-4370-96c6-cea79a756bd7" (UID: "1352c381-9193-4370-96c6-cea79a756bd7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:51:50.539943 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:50.539801 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1352c381-9193-4370-96c6-cea79a756bd7-kube-api-access-ltzrn" (OuterVolumeSpecName: "kube-api-access-ltzrn") pod "1352c381-9193-4370-96c6-cea79a756bd7" (UID: "1352c381-9193-4370-96c6-cea79a756bd7"). InnerVolumeSpecName "kube-api-access-ltzrn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:51:50.638393 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:50.638360 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ltzrn\" (UniqueName: \"kubernetes.io/projected/1352c381-9193-4370-96c6-cea79a756bd7-kube-api-access-ltzrn\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:51:50.638393 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:50.638387 2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1352c381-9193-4370-96c6-cea79a756bd7-home\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:51:50.638393 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:50.638397 2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1352c381-9193-4370-96c6-cea79a756bd7-tls-certs\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:51:50.638393 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:50.638405 2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1352c381-9193-4370-96c6-cea79a756bd7-dshm\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:51:50.638660 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:50.638412 2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1352c381-9193-4370-96c6-cea79a756bd7-model-cache\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:51:51.436664 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:51.436628 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-4b5487/test-model-4b5487-kserve-5b8cd5bd5c-rsmtb" event={"ID":"1352c381-9193-4370-96c6-cea79a756bd7","Type":"ContainerDied","Data":"c8a4032571a0360538918bb16fe9de10611205437f9094b62e95bd80f57ec402"} Jul 05 13:51:51.436664 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:51.436656 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-4b5487/test-model-4b5487-kserve-5b8cd5bd5c-rsmtb" Jul 05 13:51:51.437158 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:51.436681 2573 scope.go:117] "RemoveContainer" containerID="e6475abaeb913cd1d30718cf8e3c03b982413b165324f1c501fc406d708ea57f" Jul 05 13:51:51.458063 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:51.458026 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-4b5487/test-model-4b5487-kserve-5b8cd5bd5c-rsmtb"] Jul 05 13:51:51.461989 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:51.461966 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-4b5487/test-model-4b5487-kserve-5b8cd5bd5c-rsmtb"] Jul 05 13:51:51.736157 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:51.736080 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1352c381-9193-4370-96c6-cea79a756bd7" path="/var/lib/kubelet/pods/1352c381-9193-4370-96c6-cea79a756bd7/volumes" Jul 05 13:51:51.736490 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:51:51.736476 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e57524f6-9b1e-453d-af77-e40e7afd59b8" path="/var/lib/kubelet/pods/e57524f6-9b1e-453d-af77-e40e7afd59b8/volumes" Jul 05 13:52:17.911269 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:52:17.911218 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:52:18.997609 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:52:18.997570 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:52:19.093409 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:52:19.093371 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:52:19.196081 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:52:19.196040 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:52:19.458590 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:52:19.458545 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:52:22.811985 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:52:22.811940 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:52:25.753514 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:52:25.753473 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:52:25.823555 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:52:25.823508 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:52:26.032826 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:52:26.032748 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:52:26.244793 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:52:26.244760 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:52:26.568662 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:52:26.568632 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:52:26.893249 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:52:26.893170 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:52:27.194398 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:52:27.194367 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:52:29.568514 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:52:29.568465 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:52:29.666686 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:52:29.666651 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:52:29.761056 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:52:29.761023 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:52:29.933871 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:52:29.933839 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:52:30.359773 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:52:30.359658 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:52:30.459798 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:52:30.459764 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:52:33.999969 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:52:33.999919 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:52:34.480838 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:52:34.480796 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:52:34.843703 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:52:34.843611 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:52:34.965451 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:52:34.965416 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:52:36.960104 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:52:36.960068 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:52:37.590477 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:52:37.590440 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:52:37.673328 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:52:37.673287 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:52:37.751861 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:52:37.751831 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:52:38.121585 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:52:38.121550 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:52:38.481678 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:52:38.481643 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:52:38.578013 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:52:38.577978 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:52:42.038726 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:52:42.038683 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:52:50.388600 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:52:50.388557 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:52:50.861458 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:52:50.861423 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:52:51.131043 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:52:51.130939 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:52:51.230674 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:52:51.230639 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:52:51.423164 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:52:51.423076 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:52:52.657102 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:52:52.657068 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:52:57.481020 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:52:57.480981 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:52:57.584042 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:52:57.584011 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:52:57.755994 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:52:57.755889 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:52:57.983306 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:52:57.983264 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:52:58.155756 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:52:58.155662 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:52:58.549332 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:52:58.549295 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:52:58.651698 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:52:58.651665 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:52:58.748598 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:52:58.748564 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:52:58.881014 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:52:58.880921 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:53:04.543501 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:04.543462 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:53:04.826449 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:04.826359 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:53:04.924058 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:04.924023 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:53:05.093728 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:05.093647 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:53:05.307198 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:05.307163 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:53:05.630376 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:05.630348 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:53:06.020709 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:06.020655 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:53:06.085501 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:06.085462 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:53:08.897547 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:08.897512 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:53:09.057628 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:09.057592 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:53:09.155854 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:09.155767 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:53:09.468296 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:09.468251 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:53:09.702585 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:09.702552 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:53:09.801746 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:09.801660 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:53:10.581090 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:10.581054 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:53:10.684378 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:10.684343 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:53:11.316305 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:11.316265 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:53:11.461718 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:11.461681 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:53:13.367522 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:13.367487 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:53:13.700879 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:13.700841 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:53:14.035651 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:14.035567 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:53:14.121498 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:14.121465 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:53:16.897999 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:16.897940 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:53:16.979664 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:16.979622 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:53:17.069553 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:17.069522 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:53:17.243516 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:17.243485 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:53:17.367687 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:17.367648 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:53:17.696723 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:17.696692 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:53:17.797479 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:17.797438 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:53:21.400843 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:21.400799 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:53:23.249510 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:23.249476 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-6096ec/auth-test-model-6096ec-kserve-6dccb5cb5f-tp2gl"] Jul 05 13:53:23.250008 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:23.249989 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1352c381-9193-4370-96c6-cea79a756bd7" containerName="main" Jul 05 13:53:23.250091 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:23.250012 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="1352c381-9193-4370-96c6-cea79a756bd7" containerName="main" Jul 05 13:53:23.250091 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:23.250033 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e57524f6-9b1e-453d-af77-e40e7afd59b8" containerName="main" Jul 05 13:53:23.250091 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:23.250043 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="e57524f6-9b1e-453d-af77-e40e7afd59b8" containerName="main" Jul 05 13:53:23.250255 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:23.250140 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="e57524f6-9b1e-453d-af77-e40e7afd59b8" containerName="main" Jul 05 13:53:23.250255 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:23.250156 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="1352c381-9193-4370-96c6-cea79a756bd7" containerName="main" Jul 05 13:53:23.253110 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:23.253090 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6096ec/auth-test-model-6096ec-kserve-6dccb5cb5f-tp2gl" Jul 05 13:53:23.257641 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:23.257610 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-6096ec\"/\"kube-root-ca.crt\"" Jul 05 13:53:23.257641 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:23.257631 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-6096ec\"/\"auth-test-model-6096ec-kserve-self-signed-certs\"" Jul 05 13:53:23.257792 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:23.257625 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-6096ec\"/\"default-dockercfg-djr8f\"" Jul 05 13:53:23.257792 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:23.257680 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-6096ec\"/\"openshift-service-ca.crt\"" Jul 05 13:53:23.263058 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:23.263037 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-6096ec/auth-test-model-6096ec-kserve-6dccb5cb5f-tp2gl"] Jul 05 13:53:23.282593 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:23.282565 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:53:23.323245 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:23.323215 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9gdm\" (UniqueName: \"kubernetes.io/projected/c48e9d86-812e-4d89-a425-0ff804be530e-kube-api-access-x9gdm\") pod \"auth-test-model-6096ec-kserve-6dccb5cb5f-tp2gl\" (UID: \"c48e9d86-812e-4d89-a425-0ff804be530e\") " pod="ai-tenant-e2e-shared-a-6096ec/auth-test-model-6096ec-kserve-6dccb5cb5f-tp2gl" Jul 05 13:53:23.323378 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:23.323263 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c48e9d86-812e-4d89-a425-0ff804be530e-home\") pod \"auth-test-model-6096ec-kserve-6dccb5cb5f-tp2gl\" (UID: \"c48e9d86-812e-4d89-a425-0ff804be530e\") " pod="ai-tenant-e2e-shared-a-6096ec/auth-test-model-6096ec-kserve-6dccb5cb5f-tp2gl" Jul 05 13:53:23.323378 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:23.323317 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c48e9d86-812e-4d89-a425-0ff804be530e-tmp-dir\") pod \"auth-test-model-6096ec-kserve-6dccb5cb5f-tp2gl\" (UID: \"c48e9d86-812e-4d89-a425-0ff804be530e\") " pod="ai-tenant-e2e-shared-a-6096ec/auth-test-model-6096ec-kserve-6dccb5cb5f-tp2gl" Jul 05 13:53:23.323465 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:23.323388 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c48e9d86-812e-4d89-a425-0ff804be530e-dshm\") pod \"auth-test-model-6096ec-kserve-6dccb5cb5f-tp2gl\" (UID: \"c48e9d86-812e-4d89-a425-0ff804be530e\") " pod="ai-tenant-e2e-shared-a-6096ec/auth-test-model-6096ec-kserve-6dccb5cb5f-tp2gl" Jul 05 13:53:23.323465 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:23.323405 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c48e9d86-812e-4d89-a425-0ff804be530e-model-cache\") pod \"auth-test-model-6096ec-kserve-6dccb5cb5f-tp2gl\" (UID: \"c48e9d86-812e-4d89-a425-0ff804be530e\") " pod="ai-tenant-e2e-shared-a-6096ec/auth-test-model-6096ec-kserve-6dccb5cb5f-tp2gl" Jul 05 13:53:23.323537 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:23.323447 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c48e9d86-812e-4d89-a425-0ff804be530e-tls-certs\") pod \"auth-test-model-6096ec-kserve-6dccb5cb5f-tp2gl\" (UID: \"c48e9d86-812e-4d89-a425-0ff804be530e\") " pod="ai-tenant-e2e-shared-a-6096ec/auth-test-model-6096ec-kserve-6dccb5cb5f-tp2gl" Jul 05 13:53:23.424399 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:23.424365 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9gdm\" (UniqueName: \"kubernetes.io/projected/c48e9d86-812e-4d89-a425-0ff804be530e-kube-api-access-x9gdm\") pod \"auth-test-model-6096ec-kserve-6dccb5cb5f-tp2gl\" (UID: \"c48e9d86-812e-4d89-a425-0ff804be530e\") " pod="ai-tenant-e2e-shared-a-6096ec/auth-test-model-6096ec-kserve-6dccb5cb5f-tp2gl" Jul 05 13:53:23.424544 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:23.424407 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c48e9d86-812e-4d89-a425-0ff804be530e-home\") pod \"auth-test-model-6096ec-kserve-6dccb5cb5f-tp2gl\" (UID: \"c48e9d86-812e-4d89-a425-0ff804be530e\") " pod="ai-tenant-e2e-shared-a-6096ec/auth-test-model-6096ec-kserve-6dccb5cb5f-tp2gl" Jul 05 13:53:23.424544 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:23.424518 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c48e9d86-812e-4d89-a425-0ff804be530e-tmp-dir\") pod \"auth-test-model-6096ec-kserve-6dccb5cb5f-tp2gl\" (UID: \"c48e9d86-812e-4d89-a425-0ff804be530e\") " pod="ai-tenant-e2e-shared-a-6096ec/auth-test-model-6096ec-kserve-6dccb5cb5f-tp2gl" Jul 05 13:53:23.424671 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:23.424584 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c48e9d86-812e-4d89-a425-0ff804be530e-dshm\") pod \"auth-test-model-6096ec-kserve-6dccb5cb5f-tp2gl\" (UID: \"c48e9d86-812e-4d89-a425-0ff804be530e\") " pod="ai-tenant-e2e-shared-a-6096ec/auth-test-model-6096ec-kserve-6dccb5cb5f-tp2gl" Jul 05 13:53:23.424671 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:23.424600 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c48e9d86-812e-4d89-a425-0ff804be530e-model-cache\") pod \"auth-test-model-6096ec-kserve-6dccb5cb5f-tp2gl\" (UID: \"c48e9d86-812e-4d89-a425-0ff804be530e\") " pod="ai-tenant-e2e-shared-a-6096ec/auth-test-model-6096ec-kserve-6dccb5cb5f-tp2gl" Jul 05 13:53:23.424671 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:23.424617 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c48e9d86-812e-4d89-a425-0ff804be530e-tls-certs\") pod \"auth-test-model-6096ec-kserve-6dccb5cb5f-tp2gl\" (UID: \"c48e9d86-812e-4d89-a425-0ff804be530e\") " pod="ai-tenant-e2e-shared-a-6096ec/auth-test-model-6096ec-kserve-6dccb5cb5f-tp2gl" Jul 05 13:53:23.424858 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:23.424838 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c48e9d86-812e-4d89-a425-0ff804be530e-home\") pod \"auth-test-model-6096ec-kserve-6dccb5cb5f-tp2gl\" (UID: \"c48e9d86-812e-4d89-a425-0ff804be530e\") " pod="ai-tenant-e2e-shared-a-6096ec/auth-test-model-6096ec-kserve-6dccb5cb5f-tp2gl" Jul 05 13:53:23.424920 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:23.424884 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c48e9d86-812e-4d89-a425-0ff804be530e-tmp-dir\") pod \"auth-test-model-6096ec-kserve-6dccb5cb5f-tp2gl\" (UID: \"c48e9d86-812e-4d89-a425-0ff804be530e\") " pod="ai-tenant-e2e-shared-a-6096ec/auth-test-model-6096ec-kserve-6dccb5cb5f-tp2gl" Jul 05 13:53:23.424996 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:23.424929 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c48e9d86-812e-4d89-a425-0ff804be530e-model-cache\") pod \"auth-test-model-6096ec-kserve-6dccb5cb5f-tp2gl\" (UID: \"c48e9d86-812e-4d89-a425-0ff804be530e\") " pod="ai-tenant-e2e-shared-a-6096ec/auth-test-model-6096ec-kserve-6dccb5cb5f-tp2gl" Jul 05 13:53:23.426895 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:23.426863 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c48e9d86-812e-4d89-a425-0ff804be530e-dshm\") pod \"auth-test-model-6096ec-kserve-6dccb5cb5f-tp2gl\" (UID: \"c48e9d86-812e-4d89-a425-0ff804be530e\") " pod="ai-tenant-e2e-shared-a-6096ec/auth-test-model-6096ec-kserve-6dccb5cb5f-tp2gl" Jul 05 13:53:23.427092 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:23.427077 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c48e9d86-812e-4d89-a425-0ff804be530e-tls-certs\") pod \"auth-test-model-6096ec-kserve-6dccb5cb5f-tp2gl\" (UID: \"c48e9d86-812e-4d89-a425-0ff804be530e\") " pod="ai-tenant-e2e-shared-a-6096ec/auth-test-model-6096ec-kserve-6dccb5cb5f-tp2gl" Jul 05 13:53:23.432992 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:23.432947 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9gdm\" (UniqueName: \"kubernetes.io/projected/c48e9d86-812e-4d89-a425-0ff804be530e-kube-api-access-x9gdm\") pod \"auth-test-model-6096ec-kserve-6dccb5cb5f-tp2gl\" (UID: \"c48e9d86-812e-4d89-a425-0ff804be530e\") " pod="ai-tenant-e2e-shared-a-6096ec/auth-test-model-6096ec-kserve-6dccb5cb5f-tp2gl" Jul 05 13:53:23.563511 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:23.563415 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6096ec/auth-test-model-6096ec-kserve-6dccb5cb5f-tp2gl" Jul 05 13:53:23.693863 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:23.693836 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-6096ec/auth-test-model-6096ec-kserve-6dccb5cb5f-tp2gl"] Jul 05 13:53:23.695825 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:53:23.695796 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc48e9d86_812e_4d89_a425_0ff804be530e.slice/crio-f1fa2e53e87d3eb141841d8aa4af5798580a8dbe9e45ffe109803102da5fdcf0 WatchSource:0}: Error finding container f1fa2e53e87d3eb141841d8aa4af5798580a8dbe9e45ffe109803102da5fdcf0: Status 404 returned error can't find the container with id f1fa2e53e87d3eb141841d8aa4af5798580a8dbe9e45ffe109803102da5fdcf0 Jul 05 13:53:23.901359 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:23.901264 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6096ec/auth-test-model-6096ec-kserve-6dccb5cb5f-tp2gl" event={"ID":"c48e9d86-812e-4d89-a425-0ff804be530e","Type":"ContainerStarted","Data":"b41a674aa303ea416c1059aa7b9a0e1b9db5b0791b654a20aa0bb755e8920545"} Jul 05 13:53:23.901359 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:23.901318 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6096ec/auth-test-model-6096ec-kserve-6dccb5cb5f-tp2gl" event={"ID":"c48e9d86-812e-4d89-a425-0ff804be530e","Type":"ContainerStarted","Data":"f1fa2e53e87d3eb141841d8aa4af5798580a8dbe9e45ffe109803102da5fdcf0"} Jul 05 13:53:23.919616 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:23.919560 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-6096ec/auth-test-model-6096ec-kserve-6dccb5cb5f-tp2gl" podStartSLOduration=0.919544973 podStartE2EDuration="919.544973ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:53:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:53:23.918208312 +0000 UTC m=+2076.813866405" watchObservedRunningTime="2026-07-05 13:53:23.919544973 +0000 UTC m=+2076.815203062" Jul 05 13:53:30.520651 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:30.520617 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:53:30.622045 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:30.622006 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:53:33.564158 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:33.564115 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-6096ec/auth-test-model-6096ec-kserve-6dccb5cb5f-tp2gl" Jul 05 13:53:33.564549 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:33.564185 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-6096ec/auth-test-model-6096ec-kserve-6dccb5cb5f-tp2gl" Jul 05 13:53:33.576732 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:33.576700 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-6096ec/auth-test-model-6096ec-kserve-6dccb5cb5f-tp2gl" Jul 05 13:53:33.962338 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:33.962310 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-6096ec/auth-test-model-6096ec-kserve-6dccb5cb5f-tp2gl" Jul 05 13:53:35.953026 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:35.952994 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:53:38.514972 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:38.514924 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:53:38.615523 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:38.615474 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:53:42.944444 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:42.944406 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-6579e1/auth-test-model-6579e1-kserve-847ff4bd7f-b7mrk"] Jul 05 13:53:42.948084 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:42.948065 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6579e1/auth-test-model-6579e1-kserve-847ff4bd7f-b7mrk" Jul 05 13:53:42.950750 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:42.950722 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-6579e1\"/\"kube-root-ca.crt\"" Jul 05 13:53:42.951563 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:42.951541 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-6579e1\"/\"openshift-service-ca.crt\"" Jul 05 13:53:42.951687 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:42.951543 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-6579e1\"/\"auth-test-model-6579e1-kserve-self-signed-certs\"" Jul 05 13:53:42.951753 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:42.951710 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-6579e1\"/\"default-dockercfg-j5j64\"" Jul 05 13:53:42.959188 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:42.959164 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-6579e1/auth-test-model-6579e1-kserve-847ff4bd7f-b7mrk"] Jul 05 13:53:42.974494 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:42.974468 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:53:43.097151 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:43.097121 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/adb8f675-b8f6-49de-9678-2769e35d71d6-home\") pod \"auth-test-model-6579e1-kserve-847ff4bd7f-b7mrk\" (UID: \"adb8f675-b8f6-49de-9678-2769e35d71d6\") " pod="ai-tenant-e2e-shared-b-6579e1/auth-test-model-6579e1-kserve-847ff4bd7f-b7mrk" Jul 05 13:53:43.097151 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:43.097152 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/adb8f675-b8f6-49de-9678-2769e35d71d6-model-cache\") pod \"auth-test-model-6579e1-kserve-847ff4bd7f-b7mrk\" (UID: \"adb8f675-b8f6-49de-9678-2769e35d71d6\") " pod="ai-tenant-e2e-shared-b-6579e1/auth-test-model-6579e1-kserve-847ff4bd7f-b7mrk" Jul 05 13:53:43.097374 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:43.097171 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8l4b\" (UniqueName: \"kubernetes.io/projected/adb8f675-b8f6-49de-9678-2769e35d71d6-kube-api-access-h8l4b\") pod \"auth-test-model-6579e1-kserve-847ff4bd7f-b7mrk\" (UID: \"adb8f675-b8f6-49de-9678-2769e35d71d6\") " pod="ai-tenant-e2e-shared-b-6579e1/auth-test-model-6579e1-kserve-847ff4bd7f-b7mrk" Jul 05 13:53:43.097374 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:43.097251 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/adb8f675-b8f6-49de-9678-2769e35d71d6-tmp-dir\") pod \"auth-test-model-6579e1-kserve-847ff4bd7f-b7mrk\" (UID: \"adb8f675-b8f6-49de-9678-2769e35d71d6\") " pod="ai-tenant-e2e-shared-b-6579e1/auth-test-model-6579e1-kserve-847ff4bd7f-b7mrk" Jul 05 13:53:43.097374 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:43.097286 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/adb8f675-b8f6-49de-9678-2769e35d71d6-tls-certs\") pod \"auth-test-model-6579e1-kserve-847ff4bd7f-b7mrk\" (UID: \"adb8f675-b8f6-49de-9678-2769e35d71d6\") " pod="ai-tenant-e2e-shared-b-6579e1/auth-test-model-6579e1-kserve-847ff4bd7f-b7mrk" Jul 05 13:53:43.097374 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:43.097367 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/adb8f675-b8f6-49de-9678-2769e35d71d6-dshm\") pod \"auth-test-model-6579e1-kserve-847ff4bd7f-b7mrk\" (UID: \"adb8f675-b8f6-49de-9678-2769e35d71d6\") " pod="ai-tenant-e2e-shared-b-6579e1/auth-test-model-6579e1-kserve-847ff4bd7f-b7mrk" Jul 05 13:53:43.198536 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:43.198443 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/adb8f675-b8f6-49de-9678-2769e35d71d6-dshm\") pod \"auth-test-model-6579e1-kserve-847ff4bd7f-b7mrk\" (UID: \"adb8f675-b8f6-49de-9678-2769e35d71d6\") " pod="ai-tenant-e2e-shared-b-6579e1/auth-test-model-6579e1-kserve-847ff4bd7f-b7mrk" Jul 05 13:53:43.198536 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:43.198501 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/adb8f675-b8f6-49de-9678-2769e35d71d6-home\") pod \"auth-test-model-6579e1-kserve-847ff4bd7f-b7mrk\" (UID: \"adb8f675-b8f6-49de-9678-2769e35d71d6\") " pod="ai-tenant-e2e-shared-b-6579e1/auth-test-model-6579e1-kserve-847ff4bd7f-b7mrk" Jul 05 13:53:43.198536 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:43.198519 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/adb8f675-b8f6-49de-9678-2769e35d71d6-model-cache\") pod \"auth-test-model-6579e1-kserve-847ff4bd7f-b7mrk\" (UID: \"adb8f675-b8f6-49de-9678-2769e35d71d6\") " pod="ai-tenant-e2e-shared-b-6579e1/auth-test-model-6579e1-kserve-847ff4bd7f-b7mrk" Jul 05 13:53:43.198536 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:43.198538 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8l4b\" (UniqueName: \"kubernetes.io/projected/adb8f675-b8f6-49de-9678-2769e35d71d6-kube-api-access-h8l4b\") pod \"auth-test-model-6579e1-kserve-847ff4bd7f-b7mrk\" (UID: \"adb8f675-b8f6-49de-9678-2769e35d71d6\") " pod="ai-tenant-e2e-shared-b-6579e1/auth-test-model-6579e1-kserve-847ff4bd7f-b7mrk" Jul 05 13:53:43.198867 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:43.198568 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/adb8f675-b8f6-49de-9678-2769e35d71d6-tmp-dir\") pod \"auth-test-model-6579e1-kserve-847ff4bd7f-b7mrk\" (UID: \"adb8f675-b8f6-49de-9678-2769e35d71d6\") " pod="ai-tenant-e2e-shared-b-6579e1/auth-test-model-6579e1-kserve-847ff4bd7f-b7mrk" Jul 05 13:53:43.198867 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:43.198585 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/adb8f675-b8f6-49de-9678-2769e35d71d6-tls-certs\") pod \"auth-test-model-6579e1-kserve-847ff4bd7f-b7mrk\" (UID: \"adb8f675-b8f6-49de-9678-2769e35d71d6\") " pod="ai-tenant-e2e-shared-b-6579e1/auth-test-model-6579e1-kserve-847ff4bd7f-b7mrk" Jul 05 13:53:43.199014 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:43.198990 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/adb8f675-b8f6-49de-9678-2769e35d71d6-home\") pod \"auth-test-model-6579e1-kserve-847ff4bd7f-b7mrk\" (UID: \"adb8f675-b8f6-49de-9678-2769e35d71d6\") " pod="ai-tenant-e2e-shared-b-6579e1/auth-test-model-6579e1-kserve-847ff4bd7f-b7mrk" Jul 05 13:53:43.199068 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:43.199008 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/adb8f675-b8f6-49de-9678-2769e35d71d6-model-cache\") pod \"auth-test-model-6579e1-kserve-847ff4bd7f-b7mrk\" (UID: \"adb8f675-b8f6-49de-9678-2769e35d71d6\") " pod="ai-tenant-e2e-shared-b-6579e1/auth-test-model-6579e1-kserve-847ff4bd7f-b7mrk" Jul 05 13:53:43.199114 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:43.199077 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/adb8f675-b8f6-49de-9678-2769e35d71d6-tmp-dir\") pod \"auth-test-model-6579e1-kserve-847ff4bd7f-b7mrk\" (UID: \"adb8f675-b8f6-49de-9678-2769e35d71d6\") " pod="ai-tenant-e2e-shared-b-6579e1/auth-test-model-6579e1-kserve-847ff4bd7f-b7mrk" Jul 05 13:53:43.200853 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:43.200823 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/adb8f675-b8f6-49de-9678-2769e35d71d6-dshm\") pod \"auth-test-model-6579e1-kserve-847ff4bd7f-b7mrk\" (UID: \"adb8f675-b8f6-49de-9678-2769e35d71d6\") " pod="ai-tenant-e2e-shared-b-6579e1/auth-test-model-6579e1-kserve-847ff4bd7f-b7mrk" Jul 05 13:53:43.201015 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:43.200937 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/adb8f675-b8f6-49de-9678-2769e35d71d6-tls-certs\") pod \"auth-test-model-6579e1-kserve-847ff4bd7f-b7mrk\" (UID: \"adb8f675-b8f6-49de-9678-2769e35d71d6\") " pod="ai-tenant-e2e-shared-b-6579e1/auth-test-model-6579e1-kserve-847ff4bd7f-b7mrk" Jul 05 13:53:43.208379 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:43.208356 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8l4b\" (UniqueName: \"kubernetes.io/projected/adb8f675-b8f6-49de-9678-2769e35d71d6-kube-api-access-h8l4b\") pod \"auth-test-model-6579e1-kserve-847ff4bd7f-b7mrk\" (UID: \"adb8f675-b8f6-49de-9678-2769e35d71d6\") " pod="ai-tenant-e2e-shared-b-6579e1/auth-test-model-6579e1-kserve-847ff4bd7f-b7mrk" Jul 05 13:53:43.258297 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:43.258250 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6579e1/auth-test-model-6579e1-kserve-847ff4bd7f-b7mrk" Jul 05 13:53:43.394065 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:43.394038 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-6579e1/auth-test-model-6579e1-kserve-847ff4bd7f-b7mrk"] Jul 05 13:53:43.394894 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:53:43.394870 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podadb8f675_b8f6_49de_9678_2769e35d71d6.slice/crio-a89f0f1c6fa7f3b28507bbe8dbc94d945d32f4f3d166086b19280bde77c22d72 WatchSource:0}: Error finding container a89f0f1c6fa7f3b28507bbe8dbc94d945d32f4f3d166086b19280bde77c22d72: Status 404 returned error can't find the container with id a89f0f1c6fa7f3b28507bbe8dbc94d945d32f4f3d166086b19280bde77c22d72 Jul 05 13:53:44.000590 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:44.000552 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6579e1/auth-test-model-6579e1-kserve-847ff4bd7f-b7mrk" event={"ID":"adb8f675-b8f6-49de-9678-2769e35d71d6","Type":"ContainerStarted","Data":"f6050f6eacd3a57a4196fd8c709528fa580ea80881d227b0e1153a32cb3e51b6"} Jul 05 13:53:44.000987 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:44.000598 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6579e1/auth-test-model-6579e1-kserve-847ff4bd7f-b7mrk" event={"ID":"adb8f675-b8f6-49de-9678-2769e35d71d6","Type":"ContainerStarted","Data":"a89f0f1c6fa7f3b28507bbe8dbc94d945d32f4f3d166086b19280bde77c22d72"} Jul 05 13:53:44.017589 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:44.017542 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-6579e1/auth-test-model-6579e1-kserve-847ff4bd7f-b7mrk" podStartSLOduration=2.017527941 podStartE2EDuration="2.017527941s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:53:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:53:44.016593174 +0000 UTC m=+2096.912251293" watchObservedRunningTime="2026-07-05 13:53:44.017527941 +0000 UTC m=+2096.913186029" Jul 05 13:53:47.648105 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:47.648068 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:53:53.258399 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:53.258369 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-6579e1/auth-test-model-6579e1-kserve-847ff4bd7f-b7mrk" Jul 05 13:53:53.258399 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:53.258411 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-6579e1/auth-test-model-6579e1-kserve-847ff4bd7f-b7mrk" Jul 05 13:53:53.271704 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:53.271680 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-6579e1/auth-test-model-6579e1-kserve-847ff4bd7f-b7mrk" Jul 05 13:53:54.060972 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:53:54.060932 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-6579e1/auth-test-model-6579e1-kserve-847ff4bd7f-b7mrk" Jul 05 13:54:02.347205 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:54:02.347169 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:54:14.744740 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:54:14.744704 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:54:32.667737 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:54:32.667698 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:54:50.443324 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:54:50.443244 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:55:06.048509 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:55:06.048476 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:55:07.772502 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:55:07.772456 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:55:07.828944 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:55:07.828902 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:55:22.055011 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:55:22.054972 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:55:38.144253 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:55:38.144218 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:55:49.792719 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:55:49.792682 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:55:50.368397 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:55:50.368360 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:55:50.654940 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:55:50.654861 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-6096ec/auth-test-model-6096ec-kserve-6dccb5cb5f-tp2gl"] Jul 05 13:55:50.676661 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:55:50.676630 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:55:51.736882 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:55:51.736848 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-6579e1/auth-test-model-6579e1-kserve-847ff4bd7f-b7mrk"] Jul 05 13:55:51.755778 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:55:51.755740 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:55:53.850779 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:55:53.850744 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:55:58.675787 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:55:58.675752 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:56:05.675120 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:05.675051 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-6096ec/auth-test-model-6096ec-kserve-6dccb5cb5f-tp2gl" podUID="c48e9d86-812e-4d89-a425-0ff804be530e" containerName="main" containerID="cri-o://b41a674aa303ea416c1059aa7b9a0e1b9db5b0791b654a20aa0bb755e8920545" gracePeriod=45 Jul 05 13:56:05.925108 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:05.925080 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6096ec/auth-test-model-6096ec-kserve-6dccb5cb5f-tp2gl" Jul 05 13:56:06.067230 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:06.067202 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c48e9d86-812e-4d89-a425-0ff804be530e-home\") pod \"c48e9d86-812e-4d89-a425-0ff804be530e\" (UID: \"c48e9d86-812e-4d89-a425-0ff804be530e\") " Jul 05 13:56:06.067410 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:06.067241 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c48e9d86-812e-4d89-a425-0ff804be530e-tls-certs\") pod \"c48e9d86-812e-4d89-a425-0ff804be530e\" (UID: \"c48e9d86-812e-4d89-a425-0ff804be530e\") " Jul 05 13:56:06.067410 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:06.067268 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c48e9d86-812e-4d89-a425-0ff804be530e-tmp-dir\") pod \"c48e9d86-812e-4d89-a425-0ff804be530e\" (UID: \"c48e9d86-812e-4d89-a425-0ff804be530e\") " Jul 05 13:56:06.067410 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:06.067311 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x9gdm\" (UniqueName: \"kubernetes.io/projected/c48e9d86-812e-4d89-a425-0ff804be530e-kube-api-access-x9gdm\") pod \"c48e9d86-812e-4d89-a425-0ff804be530e\" (UID: \"c48e9d86-812e-4d89-a425-0ff804be530e\") " Jul 05 13:56:06.067410 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:06.067385 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c48e9d86-812e-4d89-a425-0ff804be530e-model-cache\") pod \"c48e9d86-812e-4d89-a425-0ff804be530e\" (UID: \"c48e9d86-812e-4d89-a425-0ff804be530e\") " Jul 05 13:56:06.067625 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:06.067439 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c48e9d86-812e-4d89-a425-0ff804be530e-dshm\") pod \"c48e9d86-812e-4d89-a425-0ff804be530e\" (UID: \"c48e9d86-812e-4d89-a425-0ff804be530e\") " Jul 05 13:56:06.067625 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:06.067528 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c48e9d86-812e-4d89-a425-0ff804be530e-home" (OuterVolumeSpecName: "home") pod "c48e9d86-812e-4d89-a425-0ff804be530e" (UID: "c48e9d86-812e-4d89-a425-0ff804be530e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:56:06.067625 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:06.067566 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c48e9d86-812e-4d89-a425-0ff804be530e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c48e9d86-812e-4d89-a425-0ff804be530e" (UID: "c48e9d86-812e-4d89-a425-0ff804be530e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:56:06.067781 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:06.067702 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c48e9d86-812e-4d89-a425-0ff804be530e-model-cache" (OuterVolumeSpecName: "model-cache") pod "c48e9d86-812e-4d89-a425-0ff804be530e" (UID: "c48e9d86-812e-4d89-a425-0ff804be530e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:56:06.067882 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:06.067805 2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c48e9d86-812e-4d89-a425-0ff804be530e-model-cache\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:56:06.067882 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:06.067822 2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c48e9d86-812e-4d89-a425-0ff804be530e-home\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:56:06.067882 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:06.067837 2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c48e9d86-812e-4d89-a425-0ff804be530e-tmp-dir\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:56:06.069508 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:06.069475 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c48e9d86-812e-4d89-a425-0ff804be530e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c48e9d86-812e-4d89-a425-0ff804be530e" (UID: "c48e9d86-812e-4d89-a425-0ff804be530e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:56:06.069628 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:06.069531 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c48e9d86-812e-4d89-a425-0ff804be530e-dshm" (OuterVolumeSpecName: "dshm") pod "c48e9d86-812e-4d89-a425-0ff804be530e" (UID: "c48e9d86-812e-4d89-a425-0ff804be530e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:56:06.069628 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:06.069541 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c48e9d86-812e-4d89-a425-0ff804be530e-kube-api-access-x9gdm" (OuterVolumeSpecName: "kube-api-access-x9gdm") pod "c48e9d86-812e-4d89-a425-0ff804be530e" (UID: "c48e9d86-812e-4d89-a425-0ff804be530e"). InnerVolumeSpecName "kube-api-access-x9gdm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:56:06.168347 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:06.168311 2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c48e9d86-812e-4d89-a425-0ff804be530e-dshm\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:56:06.168347 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:06.168341 2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c48e9d86-812e-4d89-a425-0ff804be530e-tls-certs\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:56:06.168347 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:06.168353 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x9gdm\" (UniqueName: \"kubernetes.io/projected/c48e9d86-812e-4d89-a425-0ff804be530e-kube-api-access-x9gdm\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:56:06.743941 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:06.743903 2573 generic.go:356] "Generic (PLEG): container finished" podID="c48e9d86-812e-4d89-a425-0ff804be530e" containerID="b41a674aa303ea416c1059aa7b9a0e1b9db5b0791b654a20aa0bb755e8920545" exitCode=0 Jul 05 13:56:06.744386 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:06.743948 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6096ec/auth-test-model-6096ec-kserve-6dccb5cb5f-tp2gl" Jul 05 13:56:06.744386 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:06.743974 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6096ec/auth-test-model-6096ec-kserve-6dccb5cb5f-tp2gl" event={"ID":"c48e9d86-812e-4d89-a425-0ff804be530e","Type":"ContainerDied","Data":"b41a674aa303ea416c1059aa7b9a0e1b9db5b0791b654a20aa0bb755e8920545"} Jul 05 13:56:06.744386 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:06.744006 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6096ec/auth-test-model-6096ec-kserve-6dccb5cb5f-tp2gl" event={"ID":"c48e9d86-812e-4d89-a425-0ff804be530e","Type":"ContainerDied","Data":"f1fa2e53e87d3eb141841d8aa4af5798580a8dbe9e45ffe109803102da5fdcf0"} Jul 05 13:56:06.744386 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:06.744027 2573 scope.go:117] "RemoveContainer" containerID="b41a674aa303ea416c1059aa7b9a0e1b9db5b0791b654a20aa0bb755e8920545" Jul 05 13:56:06.753284 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:06.753268 2573 scope.go:117] "RemoveContainer" containerID="b41a674aa303ea416c1059aa7b9a0e1b9db5b0791b654a20aa0bb755e8920545" Jul 05 13:56:06.753551 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:56:06.753533 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b41a674aa303ea416c1059aa7b9a0e1b9db5b0791b654a20aa0bb755e8920545\": container with ID starting with b41a674aa303ea416c1059aa7b9a0e1b9db5b0791b654a20aa0bb755e8920545 not found: ID does not exist" containerID="b41a674aa303ea416c1059aa7b9a0e1b9db5b0791b654a20aa0bb755e8920545" Jul 05 13:56:06.753602 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:06.753559 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b41a674aa303ea416c1059aa7b9a0e1b9db5b0791b654a20aa0bb755e8920545"} err="failed to get container status \"b41a674aa303ea416c1059aa7b9a0e1b9db5b0791b654a20aa0bb755e8920545\": rpc error: code = NotFound desc = could not find container \"b41a674aa303ea416c1059aa7b9a0e1b9db5b0791b654a20aa0bb755e8920545\": container with ID starting with b41a674aa303ea416c1059aa7b9a0e1b9db5b0791b654a20aa0bb755e8920545 not found: ID does not exist" Jul 05 13:56:06.757304 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:06.757280 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-6579e1/auth-test-model-6579e1-kserve-847ff4bd7f-b7mrk" podUID="adb8f675-b8f6-49de-9678-2769e35d71d6" containerName="main" containerID="cri-o://f6050f6eacd3a57a4196fd8c709528fa580ea80881d227b0e1153a32cb3e51b6" gracePeriod=45 Jul 05 13:56:06.764853 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:06.764828 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-6096ec/auth-test-model-6096ec-kserve-6dccb5cb5f-tp2gl"] Jul 05 13:56:06.768505 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:06.768484 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-6096ec/auth-test-model-6096ec-kserve-6dccb5cb5f-tp2gl"] Jul 05 13:56:07.012030 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:07.012008 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6579e1/auth-test-model-6579e1-kserve-847ff4bd7f-b7mrk" Jul 05 13:56:07.176326 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:07.176293 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/adb8f675-b8f6-49de-9678-2769e35d71d6-home\") pod \"adb8f675-b8f6-49de-9678-2769e35d71d6\" (UID: \"adb8f675-b8f6-49de-9678-2769e35d71d6\") " Jul 05 13:56:07.176497 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:07.176376 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/adb8f675-b8f6-49de-9678-2769e35d71d6-model-cache\") pod \"adb8f675-b8f6-49de-9678-2769e35d71d6\" (UID: \"adb8f675-b8f6-49de-9678-2769e35d71d6\") " Jul 05 13:56:07.176497 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:07.176405 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/adb8f675-b8f6-49de-9678-2769e35d71d6-tmp-dir\") pod \"adb8f675-b8f6-49de-9678-2769e35d71d6\" (UID: \"adb8f675-b8f6-49de-9678-2769e35d71d6\") " Jul 05 13:56:07.176497 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:07.176448 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/adb8f675-b8f6-49de-9678-2769e35d71d6-dshm\") pod \"adb8f675-b8f6-49de-9678-2769e35d71d6\" (UID: \"adb8f675-b8f6-49de-9678-2769e35d71d6\") " Jul 05 13:56:07.176497 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:07.176485 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h8l4b\" (UniqueName: \"kubernetes.io/projected/adb8f675-b8f6-49de-9678-2769e35d71d6-kube-api-access-h8l4b\") pod \"adb8f675-b8f6-49de-9678-2769e35d71d6\" (UID: \"adb8f675-b8f6-49de-9678-2769e35d71d6\") " Jul 05 13:56:07.176736 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:07.176518 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/adb8f675-b8f6-49de-9678-2769e35d71d6-tls-certs\") pod \"adb8f675-b8f6-49de-9678-2769e35d71d6\" (UID: \"adb8f675-b8f6-49de-9678-2769e35d71d6\") " Jul 05 13:56:07.176736 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:07.176592 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/adb8f675-b8f6-49de-9678-2769e35d71d6-home" (OuterVolumeSpecName: "home") pod "adb8f675-b8f6-49de-9678-2769e35d71d6" (UID: "adb8f675-b8f6-49de-9678-2769e35d71d6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:56:07.176736 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:07.176685 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/adb8f675-b8f6-49de-9678-2769e35d71d6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "adb8f675-b8f6-49de-9678-2769e35d71d6" (UID: "adb8f675-b8f6-49de-9678-2769e35d71d6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:56:07.176736 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:07.176697 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/adb8f675-b8f6-49de-9678-2769e35d71d6-model-cache" (OuterVolumeSpecName: "model-cache") pod "adb8f675-b8f6-49de-9678-2769e35d71d6" (UID: "adb8f675-b8f6-49de-9678-2769e35d71d6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:56:07.176926 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:07.176864 2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/adb8f675-b8f6-49de-9678-2769e35d71d6-model-cache\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:56:07.176926 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:07.176882 2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/adb8f675-b8f6-49de-9678-2769e35d71d6-tmp-dir\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:56:07.176926 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:07.176897 2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/adb8f675-b8f6-49de-9678-2769e35d71d6-home\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:56:07.178726 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:07.178702 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/adb8f675-b8f6-49de-9678-2769e35d71d6-dshm" (OuterVolumeSpecName: "dshm") pod "adb8f675-b8f6-49de-9678-2769e35d71d6" (UID: "adb8f675-b8f6-49de-9678-2769e35d71d6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:56:07.178726 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:07.178714 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/adb8f675-b8f6-49de-9678-2769e35d71d6-kube-api-access-h8l4b" (OuterVolumeSpecName: "kube-api-access-h8l4b") pod "adb8f675-b8f6-49de-9678-2769e35d71d6" (UID: "adb8f675-b8f6-49de-9678-2769e35d71d6"). InnerVolumeSpecName "kube-api-access-h8l4b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:56:07.178884 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:07.178752 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/adb8f675-b8f6-49de-9678-2769e35d71d6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "adb8f675-b8f6-49de-9678-2769e35d71d6" (UID: "adb8f675-b8f6-49de-9678-2769e35d71d6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:56:07.278444 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:07.278364 2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/adb8f675-b8f6-49de-9678-2769e35d71d6-dshm\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:56:07.278444 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:07.278392 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h8l4b\" (UniqueName: \"kubernetes.io/projected/adb8f675-b8f6-49de-9678-2769e35d71d6-kube-api-access-h8l4b\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:56:07.278444 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:07.278403 2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/adb8f675-b8f6-49de-9678-2769e35d71d6-tls-certs\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:56:07.736193 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:07.736162 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c48e9d86-812e-4d89-a425-0ff804be530e" path="/var/lib/kubelet/pods/c48e9d86-812e-4d89-a425-0ff804be530e/volumes" Jul 05 13:56:07.749933 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:07.749910 2573 generic.go:356] "Generic (PLEG): container finished" podID="adb8f675-b8f6-49de-9678-2769e35d71d6" containerID="f6050f6eacd3a57a4196fd8c709528fa580ea80881d227b0e1153a32cb3e51b6" exitCode=0 Jul 05 13:56:07.750336 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:07.749966 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6579e1/auth-test-model-6579e1-kserve-847ff4bd7f-b7mrk" event={"ID":"adb8f675-b8f6-49de-9678-2769e35d71d6","Type":"ContainerDied","Data":"f6050f6eacd3a57a4196fd8c709528fa580ea80881d227b0e1153a32cb3e51b6"} Jul 05 13:56:07.750336 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:07.749996 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6579e1/auth-test-model-6579e1-kserve-847ff4bd7f-b7mrk" event={"ID":"adb8f675-b8f6-49de-9678-2769e35d71d6","Type":"ContainerDied","Data":"a89f0f1c6fa7f3b28507bbe8dbc94d945d32f4f3d166086b19280bde77c22d72"} Jul 05 13:56:07.750336 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:07.750012 2573 scope.go:117] "RemoveContainer" containerID="f6050f6eacd3a57a4196fd8c709528fa580ea80881d227b0e1153a32cb3e51b6" Jul 05 13:56:07.750336 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:07.749968 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6579e1/auth-test-model-6579e1-kserve-847ff4bd7f-b7mrk" Jul 05 13:56:07.758596 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:07.758578 2573 scope.go:117] "RemoveContainer" containerID="f6050f6eacd3a57a4196fd8c709528fa580ea80881d227b0e1153a32cb3e51b6" Jul 05 13:56:07.758852 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:56:07.758828 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6050f6eacd3a57a4196fd8c709528fa580ea80881d227b0e1153a32cb3e51b6\": container with ID starting with f6050f6eacd3a57a4196fd8c709528fa580ea80881d227b0e1153a32cb3e51b6 not found: ID does not exist" containerID="f6050f6eacd3a57a4196fd8c709528fa580ea80881d227b0e1153a32cb3e51b6" Jul 05 13:56:07.758938 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:07.758856 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6050f6eacd3a57a4196fd8c709528fa580ea80881d227b0e1153a32cb3e51b6"} err="failed to get container status \"f6050f6eacd3a57a4196fd8c709528fa580ea80881d227b0e1153a32cb3e51b6\": rpc error: code = NotFound desc = could not find container \"f6050f6eacd3a57a4196fd8c709528fa580ea80881d227b0e1153a32cb3e51b6\": container with ID starting with f6050f6eacd3a57a4196fd8c709528fa580ea80881d227b0e1153a32cb3e51b6 not found: ID does not exist" Jul 05 13:56:07.766310 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:07.766289 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-6579e1/auth-test-model-6579e1-kserve-847ff4bd7f-b7mrk"] Jul 05 13:56:07.770348 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:07.770328 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-6579e1/auth-test-model-6579e1-kserve-847ff4bd7f-b7mrk"] Jul 05 13:56:09.735662 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:09.735586 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="adb8f675-b8f6-49de-9678-2769e35d71d6" path="/var/lib/kubelet/pods/adb8f675-b8f6-49de-9678-2769e35d71d6/volumes" Jul 05 13:56:10.471413 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:10.471381 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:56:32.846435 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:32.846403 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-6579e1/sub-test-model-6579e1-kserve-5c47796c97-j7lf9"] Jul 05 13:56:32.846870 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:32.846754 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c48e9d86-812e-4d89-a425-0ff804be530e" containerName="main" Jul 05 13:56:32.846870 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:32.846766 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="c48e9d86-812e-4d89-a425-0ff804be530e" containerName="main" Jul 05 13:56:32.846870 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:32.846775 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="adb8f675-b8f6-49de-9678-2769e35d71d6" containerName="main" Jul 05 13:56:32.846870 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:32.846780 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="adb8f675-b8f6-49de-9678-2769e35d71d6" containerName="main" Jul 05 13:56:32.846870 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:32.846851 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="adb8f675-b8f6-49de-9678-2769e35d71d6" containerName="main" Jul 05 13:56:32.846870 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:32.846862 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="c48e9d86-812e-4d89-a425-0ff804be530e" containerName="main" Jul 05 13:56:32.849855 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:32.849834 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6579e1/sub-test-model-6579e1-kserve-5c47796c97-j7lf9" Jul 05 13:56:32.853128 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:32.853098 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-6579e1\"/\"sub-test-model-6579e1-kserve-self-signed-certs\"" Jul 05 13:56:32.853253 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:32.853135 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-6579e1\"/\"default-dockercfg-j5j64\"" Jul 05 13:56:32.853253 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:32.853144 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-6579e1\"/\"openshift-service-ca.crt\"" Jul 05 13:56:32.853253 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:32.853218 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-6579e1\"/\"kube-root-ca.crt\"" Jul 05 13:56:32.860019 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:32.859992 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-6579e1/sub-test-model-6579e1-kserve-5c47796c97-j7lf9"] Jul 05 13:56:32.880862 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:32.880839 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:56:32.897891 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:32.897870 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8a00eb5-1e87-491c-964a-30494725730b-tmp-dir\") pod \"sub-test-model-6579e1-kserve-5c47796c97-j7lf9\" (UID: \"f8a00eb5-1e87-491c-964a-30494725730b\") " pod="ai-tenant-e2e-shared-b-6579e1/sub-test-model-6579e1-kserve-5c47796c97-j7lf9" Jul 05 13:56:32.897985 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:32.897901 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8a00eb5-1e87-491c-964a-30494725730b-home\") pod \"sub-test-model-6579e1-kserve-5c47796c97-j7lf9\" (UID: \"f8a00eb5-1e87-491c-964a-30494725730b\") " pod="ai-tenant-e2e-shared-b-6579e1/sub-test-model-6579e1-kserve-5c47796c97-j7lf9" Jul 05 13:56:32.898029 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:32.897991 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8a00eb5-1e87-491c-964a-30494725730b-model-cache\") pod \"sub-test-model-6579e1-kserve-5c47796c97-j7lf9\" (UID: \"f8a00eb5-1e87-491c-964a-30494725730b\") " pod="ai-tenant-e2e-shared-b-6579e1/sub-test-model-6579e1-kserve-5c47796c97-j7lf9" Jul 05 13:56:32.898065 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:32.898046 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8a00eb5-1e87-491c-964a-30494725730b-dshm\") pod \"sub-test-model-6579e1-kserve-5c47796c97-j7lf9\" (UID: \"f8a00eb5-1e87-491c-964a-30494725730b\") " pod="ai-tenant-e2e-shared-b-6579e1/sub-test-model-6579e1-kserve-5c47796c97-j7lf9" Jul 05 13:56:32.898125 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:32.898111 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6mcf\" (UniqueName: \"kubernetes.io/projected/f8a00eb5-1e87-491c-964a-30494725730b-kube-api-access-x6mcf\") pod \"sub-test-model-6579e1-kserve-5c47796c97-j7lf9\" (UID: \"f8a00eb5-1e87-491c-964a-30494725730b\") " pod="ai-tenant-e2e-shared-b-6579e1/sub-test-model-6579e1-kserve-5c47796c97-j7lf9" Jul 05 13:56:32.898181 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:32.898168 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8a00eb5-1e87-491c-964a-30494725730b-tls-certs\") pod \"sub-test-model-6579e1-kserve-5c47796c97-j7lf9\" (UID: \"f8a00eb5-1e87-491c-964a-30494725730b\") " pod="ai-tenant-e2e-shared-b-6579e1/sub-test-model-6579e1-kserve-5c47796c97-j7lf9" Jul 05 13:56:32.999282 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:32.999248 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8a00eb5-1e87-491c-964a-30494725730b-model-cache\") pod \"sub-test-model-6579e1-kserve-5c47796c97-j7lf9\" (UID: \"f8a00eb5-1e87-491c-964a-30494725730b\") " pod="ai-tenant-e2e-shared-b-6579e1/sub-test-model-6579e1-kserve-5c47796c97-j7lf9" Jul 05 13:56:32.999282 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:32.999283 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8a00eb5-1e87-491c-964a-30494725730b-dshm\") pod \"sub-test-model-6579e1-kserve-5c47796c97-j7lf9\" (UID: \"f8a00eb5-1e87-491c-964a-30494725730b\") " pod="ai-tenant-e2e-shared-b-6579e1/sub-test-model-6579e1-kserve-5c47796c97-j7lf9" Jul 05 13:56:32.999497 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:32.999316 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6mcf\" (UniqueName: \"kubernetes.io/projected/f8a00eb5-1e87-491c-964a-30494725730b-kube-api-access-x6mcf\") pod \"sub-test-model-6579e1-kserve-5c47796c97-j7lf9\" (UID: \"f8a00eb5-1e87-491c-964a-30494725730b\") " pod="ai-tenant-e2e-shared-b-6579e1/sub-test-model-6579e1-kserve-5c47796c97-j7lf9" Jul 05 13:56:32.999497 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:32.999354 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8a00eb5-1e87-491c-964a-30494725730b-tls-certs\") pod \"sub-test-model-6579e1-kserve-5c47796c97-j7lf9\" (UID: \"f8a00eb5-1e87-491c-964a-30494725730b\") " pod="ai-tenant-e2e-shared-b-6579e1/sub-test-model-6579e1-kserve-5c47796c97-j7lf9" Jul 05 13:56:32.999497 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:32.999381 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8a00eb5-1e87-491c-964a-30494725730b-tmp-dir\") pod \"sub-test-model-6579e1-kserve-5c47796c97-j7lf9\" (UID: \"f8a00eb5-1e87-491c-964a-30494725730b\") " pod="ai-tenant-e2e-shared-b-6579e1/sub-test-model-6579e1-kserve-5c47796c97-j7lf9" Jul 05 13:56:32.999497 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:32.999413 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8a00eb5-1e87-491c-964a-30494725730b-home\") pod \"sub-test-model-6579e1-kserve-5c47796c97-j7lf9\" (UID: \"f8a00eb5-1e87-491c-964a-30494725730b\") " pod="ai-tenant-e2e-shared-b-6579e1/sub-test-model-6579e1-kserve-5c47796c97-j7lf9" Jul 05 13:56:32.999726 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:32.999698 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8a00eb5-1e87-491c-964a-30494725730b-model-cache\") pod \"sub-test-model-6579e1-kserve-5c47796c97-j7lf9\" (UID: \"f8a00eb5-1e87-491c-964a-30494725730b\") " pod="ai-tenant-e2e-shared-b-6579e1/sub-test-model-6579e1-kserve-5c47796c97-j7lf9" Jul 05 13:56:32.999791 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:32.999740 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8a00eb5-1e87-491c-964a-30494725730b-tmp-dir\") pod \"sub-test-model-6579e1-kserve-5c47796c97-j7lf9\" (UID: \"f8a00eb5-1e87-491c-964a-30494725730b\") " pod="ai-tenant-e2e-shared-b-6579e1/sub-test-model-6579e1-kserve-5c47796c97-j7lf9" Jul 05 13:56:32.999791 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:32.999785 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8a00eb5-1e87-491c-964a-30494725730b-home\") pod \"sub-test-model-6579e1-kserve-5c47796c97-j7lf9\" (UID: \"f8a00eb5-1e87-491c-964a-30494725730b\") " pod="ai-tenant-e2e-shared-b-6579e1/sub-test-model-6579e1-kserve-5c47796c97-j7lf9" Jul 05 13:56:33.001579 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:33.001557 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8a00eb5-1e87-491c-964a-30494725730b-dshm\") pod \"sub-test-model-6579e1-kserve-5c47796c97-j7lf9\" (UID: \"f8a00eb5-1e87-491c-964a-30494725730b\") " pod="ai-tenant-e2e-shared-b-6579e1/sub-test-model-6579e1-kserve-5c47796c97-j7lf9" Jul 05 13:56:33.001876 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:33.001858 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8a00eb5-1e87-491c-964a-30494725730b-tls-certs\") pod \"sub-test-model-6579e1-kserve-5c47796c97-j7lf9\" (UID: \"f8a00eb5-1e87-491c-964a-30494725730b\") " pod="ai-tenant-e2e-shared-b-6579e1/sub-test-model-6579e1-kserve-5c47796c97-j7lf9" Jul 05 13:56:33.007529 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:33.007505 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6mcf\" (UniqueName: \"kubernetes.io/projected/f8a00eb5-1e87-491c-964a-30494725730b-kube-api-access-x6mcf\") pod \"sub-test-model-6579e1-kserve-5c47796c97-j7lf9\" (UID: \"f8a00eb5-1e87-491c-964a-30494725730b\") " pod="ai-tenant-e2e-shared-b-6579e1/sub-test-model-6579e1-kserve-5c47796c97-j7lf9" Jul 05 13:56:33.160798 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:33.160721 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6579e1/sub-test-model-6579e1-kserve-5c47796c97-j7lf9" Jul 05 13:56:33.294242 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:33.294219 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-6579e1/sub-test-model-6579e1-kserve-5c47796c97-j7lf9"] Jul 05 13:56:33.295824 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:56:33.295801 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf8a00eb5_1e87_491c_964a_30494725730b.slice/crio-abec00cf37701d4bc1b931be767775e9c4265babdadc46573ae9a8b02ccd5715 WatchSource:0}: Error finding container abec00cf37701d4bc1b931be767775e9c4265babdadc46573ae9a8b02ccd5715: Status 404 returned error can't find the container with id abec00cf37701d4bc1b931be767775e9c4265babdadc46573ae9a8b02ccd5715 Jul 05 13:56:33.884605 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:33.884566 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6579e1/sub-test-model-6579e1-kserve-5c47796c97-j7lf9" event={"ID":"f8a00eb5-1e87-491c-964a-30494725730b","Type":"ContainerStarted","Data":"1f4aeb865eebbe50fa67815627bba979c2ceb17f48bff268a110f82dbe3348e2"} Jul 05 13:56:33.884605 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:33.884608 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6579e1/sub-test-model-6579e1-kserve-5c47796c97-j7lf9" event={"ID":"f8a00eb5-1e87-491c-964a-30494725730b","Type":"ContainerStarted","Data":"abec00cf37701d4bc1b931be767775e9c4265babdadc46573ae9a8b02ccd5715"} Jul 05 13:56:33.907727 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:33.907675 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-6579e1/sub-test-model-6579e1-kserve-5c47796c97-j7lf9" podStartSLOduration=1.907657071 podStartE2EDuration="1.907657071s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:56:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:56:33.90640993 +0000 UTC m=+2266.802068020" watchObservedRunningTime="2026-07-05 13:56:33.907657071 +0000 UTC m=+2266.803315159" Jul 05 13:56:37.149850 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:37.149813 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:56:43.161787 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:43.161753 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-6579e1/sub-test-model-6579e1-kserve-5c47796c97-j7lf9" Jul 05 13:56:43.162325 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:43.161805 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-6579e1/sub-test-model-6579e1-kserve-5c47796c97-j7lf9" Jul 05 13:56:43.174197 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:43.174170 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-6579e1/sub-test-model-6579e1-kserve-5c47796c97-j7lf9" Jul 05 13:56:43.944454 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:43.944425 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-6579e1/sub-test-model-6579e1-kserve-5c47796c97-j7lf9" Jul 05 13:56:47.926357 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:47.926323 2573 scope.go:117] "RemoveContainer" containerID="b9952be236dd20f4a0899ed318f8b8c49236528d41c8e3c30263575614d28cd0" Jul 05 13:56:51.345740 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:56:51.345703 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:57:07.405391 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:07.405359 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:57:08.361629 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:08.361596 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-6579e1/sub-test-model-6579e1-kserve-5c47796c97-j7lf9"] Jul 05 13:57:08.384602 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:08.384571 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:57:11.547972 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:11.547928 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:57:13.548728 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:13.548693 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-6096ec/rate-test-model-6096ec-kserve-95d6bd967-cdkwh"] Jul 05 13:57:13.552219 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:13.552202 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6096ec/rate-test-model-6096ec-kserve-95d6bd967-cdkwh" Jul 05 13:57:13.554825 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:13.554790 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-6096ec\"/\"openshift-service-ca.crt\"" Jul 05 13:57:13.555007 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:13.554941 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-6096ec\"/\"kube-root-ca.crt\"" Jul 05 13:57:13.555752 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:13.555733 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-6096ec\"/\"rate-test-model-6096ec-kserve-self-signed-certs\"" Jul 05 13:57:13.555841 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:13.555770 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-6096ec\"/\"default-dockercfg-djr8f\"" Jul 05 13:57:13.563914 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:13.563891 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-6096ec/rate-test-model-6096ec-kserve-95d6bd967-cdkwh"] Jul 05 13:57:13.597367 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:13.597335 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:57:13.646653 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:13.646619 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5qbc\" (UniqueName: \"kubernetes.io/projected/8255612e-0a8c-40b0-88ef-c94f5251d4c1-kube-api-access-m5qbc\") pod \"rate-test-model-6096ec-kserve-95d6bd967-cdkwh\" (UID: \"8255612e-0a8c-40b0-88ef-c94f5251d4c1\") " pod="ai-tenant-e2e-shared-a-6096ec/rate-test-model-6096ec-kserve-95d6bd967-cdkwh" Jul 05 13:57:13.646813 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:13.646672 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8255612e-0a8c-40b0-88ef-c94f5251d4c1-home\") pod \"rate-test-model-6096ec-kserve-95d6bd967-cdkwh\" (UID: \"8255612e-0a8c-40b0-88ef-c94f5251d4c1\") " pod="ai-tenant-e2e-shared-a-6096ec/rate-test-model-6096ec-kserve-95d6bd967-cdkwh" Jul 05 13:57:13.646813 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:13.646689 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8255612e-0a8c-40b0-88ef-c94f5251d4c1-dshm\") pod \"rate-test-model-6096ec-kserve-95d6bd967-cdkwh\" (UID: \"8255612e-0a8c-40b0-88ef-c94f5251d4c1\") " pod="ai-tenant-e2e-shared-a-6096ec/rate-test-model-6096ec-kserve-95d6bd967-cdkwh" Jul 05 13:57:13.646813 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:13.646743 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8255612e-0a8c-40b0-88ef-c94f5251d4c1-tls-certs\") pod \"rate-test-model-6096ec-kserve-95d6bd967-cdkwh\" (UID: \"8255612e-0a8c-40b0-88ef-c94f5251d4c1\") " pod="ai-tenant-e2e-shared-a-6096ec/rate-test-model-6096ec-kserve-95d6bd967-cdkwh" Jul 05 13:57:13.646918 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:13.646810 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8255612e-0a8c-40b0-88ef-c94f5251d4c1-tmp-dir\") pod \"rate-test-model-6096ec-kserve-95d6bd967-cdkwh\" (UID: \"8255612e-0a8c-40b0-88ef-c94f5251d4c1\") " pod="ai-tenant-e2e-shared-a-6096ec/rate-test-model-6096ec-kserve-95d6bd967-cdkwh" Jul 05 13:57:13.646918 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:13.646865 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8255612e-0a8c-40b0-88ef-c94f5251d4c1-model-cache\") pod \"rate-test-model-6096ec-kserve-95d6bd967-cdkwh\" (UID: \"8255612e-0a8c-40b0-88ef-c94f5251d4c1\") " pod="ai-tenant-e2e-shared-a-6096ec/rate-test-model-6096ec-kserve-95d6bd967-cdkwh" Jul 05 13:57:13.747242 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:13.747210 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8255612e-0a8c-40b0-88ef-c94f5251d4c1-tmp-dir\") pod \"rate-test-model-6096ec-kserve-95d6bd967-cdkwh\" (UID: \"8255612e-0a8c-40b0-88ef-c94f5251d4c1\") " pod="ai-tenant-e2e-shared-a-6096ec/rate-test-model-6096ec-kserve-95d6bd967-cdkwh" Jul 05 13:57:13.747437 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:13.747298 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8255612e-0a8c-40b0-88ef-c94f5251d4c1-model-cache\") pod \"rate-test-model-6096ec-kserve-95d6bd967-cdkwh\" (UID: \"8255612e-0a8c-40b0-88ef-c94f5251d4c1\") " pod="ai-tenant-e2e-shared-a-6096ec/rate-test-model-6096ec-kserve-95d6bd967-cdkwh" Jul 05 13:57:13.747437 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:13.747346 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5qbc\" (UniqueName: \"kubernetes.io/projected/8255612e-0a8c-40b0-88ef-c94f5251d4c1-kube-api-access-m5qbc\") pod \"rate-test-model-6096ec-kserve-95d6bd967-cdkwh\" (UID: \"8255612e-0a8c-40b0-88ef-c94f5251d4c1\") " pod="ai-tenant-e2e-shared-a-6096ec/rate-test-model-6096ec-kserve-95d6bd967-cdkwh" Jul 05 13:57:13.747437 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:13.747402 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8255612e-0a8c-40b0-88ef-c94f5251d4c1-home\") pod \"rate-test-model-6096ec-kserve-95d6bd967-cdkwh\" (UID: \"8255612e-0a8c-40b0-88ef-c94f5251d4c1\") " pod="ai-tenant-e2e-shared-a-6096ec/rate-test-model-6096ec-kserve-95d6bd967-cdkwh" Jul 05 13:57:13.747437 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:13.747425 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8255612e-0a8c-40b0-88ef-c94f5251d4c1-dshm\") pod \"rate-test-model-6096ec-kserve-95d6bd967-cdkwh\" (UID: \"8255612e-0a8c-40b0-88ef-c94f5251d4c1\") " pod="ai-tenant-e2e-shared-a-6096ec/rate-test-model-6096ec-kserve-95d6bd967-cdkwh" Jul 05 13:57:13.747688 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:13.747450 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8255612e-0a8c-40b0-88ef-c94f5251d4c1-tls-certs\") pod \"rate-test-model-6096ec-kserve-95d6bd967-cdkwh\" (UID: \"8255612e-0a8c-40b0-88ef-c94f5251d4c1\") " pod="ai-tenant-e2e-shared-a-6096ec/rate-test-model-6096ec-kserve-95d6bd967-cdkwh" Jul 05 13:57:13.747688 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:13.747654 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8255612e-0a8c-40b0-88ef-c94f5251d4c1-tmp-dir\") pod \"rate-test-model-6096ec-kserve-95d6bd967-cdkwh\" (UID: \"8255612e-0a8c-40b0-88ef-c94f5251d4c1\") " pod="ai-tenant-e2e-shared-a-6096ec/rate-test-model-6096ec-kserve-95d6bd967-cdkwh" Jul 05 13:57:13.747779 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:13.747722 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8255612e-0a8c-40b0-88ef-c94f5251d4c1-model-cache\") pod \"rate-test-model-6096ec-kserve-95d6bd967-cdkwh\" (UID: \"8255612e-0a8c-40b0-88ef-c94f5251d4c1\") " pod="ai-tenant-e2e-shared-a-6096ec/rate-test-model-6096ec-kserve-95d6bd967-cdkwh" Jul 05 13:57:13.747920 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:13.747893 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8255612e-0a8c-40b0-88ef-c94f5251d4c1-home\") pod \"rate-test-model-6096ec-kserve-95d6bd967-cdkwh\" (UID: \"8255612e-0a8c-40b0-88ef-c94f5251d4c1\") " pod="ai-tenant-e2e-shared-a-6096ec/rate-test-model-6096ec-kserve-95d6bd967-cdkwh" Jul 05 13:57:13.749886 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:13.749859 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8255612e-0a8c-40b0-88ef-c94f5251d4c1-dshm\") pod \"rate-test-model-6096ec-kserve-95d6bd967-cdkwh\" (UID: \"8255612e-0a8c-40b0-88ef-c94f5251d4c1\") " pod="ai-tenant-e2e-shared-a-6096ec/rate-test-model-6096ec-kserve-95d6bd967-cdkwh" Jul 05 13:57:13.750050 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:13.750031 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8255612e-0a8c-40b0-88ef-c94f5251d4c1-tls-certs\") pod \"rate-test-model-6096ec-kserve-95d6bd967-cdkwh\" (UID: \"8255612e-0a8c-40b0-88ef-c94f5251d4c1\") " pod="ai-tenant-e2e-shared-a-6096ec/rate-test-model-6096ec-kserve-95d6bd967-cdkwh" Jul 05 13:57:13.756004 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:13.755940 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5qbc\" (UniqueName: \"kubernetes.io/projected/8255612e-0a8c-40b0-88ef-c94f5251d4c1-kube-api-access-m5qbc\") pod \"rate-test-model-6096ec-kserve-95d6bd967-cdkwh\" (UID: \"8255612e-0a8c-40b0-88ef-c94f5251d4c1\") " pod="ai-tenant-e2e-shared-a-6096ec/rate-test-model-6096ec-kserve-95d6bd967-cdkwh" Jul 05 13:57:13.861908 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:13.861804 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6096ec/rate-test-model-6096ec-kserve-95d6bd967-cdkwh" Jul 05 13:57:14.005141 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:14.005110 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-6096ec/rate-test-model-6096ec-kserve-95d6bd967-cdkwh"] Jul 05 13:57:14.006333 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:57:14.006300 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8255612e_0a8c_40b0_88ef_c94f5251d4c1.slice/crio-abb2288d90aba4c45073aaa161055f98cda037737ec2d3df0232d3179ac1e23e WatchSource:0}: Error finding container abb2288d90aba4c45073aaa161055f98cda037737ec2d3df0232d3179ac1e23e: Status 404 returned error can't find the container with id abb2288d90aba4c45073aaa161055f98cda037737ec2d3df0232d3179ac1e23e Jul 05 13:57:14.086256 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:14.086222 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6096ec/rate-test-model-6096ec-kserve-95d6bd967-cdkwh" event={"ID":"8255612e-0a8c-40b0-88ef-c94f5251d4c1","Type":"ContainerStarted","Data":"abb2288d90aba4c45073aaa161055f98cda037737ec2d3df0232d3179ac1e23e"} Jul 05 13:57:15.092359 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:15.092324 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6096ec/rate-test-model-6096ec-kserve-95d6bd967-cdkwh" event={"ID":"8255612e-0a8c-40b0-88ef-c94f5251d4c1","Type":"ContainerStarted","Data":"a1e93602186364d9502f7f9d798750a01e9188e348ee799249c12dae055b112b"} Jul 05 13:57:15.112222 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:15.112173 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-6096ec/rate-test-model-6096ec-kserve-95d6bd967-cdkwh" podStartSLOduration=2.112155603 podStartE2EDuration="2.112155603s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:57:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:57:15.110861451 +0000 UTC m=+2308.006519541" watchObservedRunningTime="2026-07-05 13:57:15.112155603 +0000 UTC m=+2308.007813691" Jul 05 13:57:23.381445 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:23.381403 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-6579e1/sub-test-model-6579e1-kserve-5c47796c97-j7lf9" podUID="f8a00eb5-1e87-491c-964a-30494725730b" containerName="main" containerID="cri-o://1f4aeb865eebbe50fa67815627bba979c2ceb17f48bff268a110f82dbe3348e2" gracePeriod=45 Jul 05 13:57:23.626155 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:23.626134 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6579e1/sub-test-model-6579e1-kserve-5c47796c97-j7lf9" Jul 05 13:57:23.742915 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:23.742889 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8a00eb5-1e87-491c-964a-30494725730b-tmp-dir\") pod \"f8a00eb5-1e87-491c-964a-30494725730b\" (UID: \"f8a00eb5-1e87-491c-964a-30494725730b\") " Jul 05 13:57:23.743105 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:23.742933 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8a00eb5-1e87-491c-964a-30494725730b-dshm\") pod \"f8a00eb5-1e87-491c-964a-30494725730b\" (UID: \"f8a00eb5-1e87-491c-964a-30494725730b\") " Jul 05 13:57:23.743105 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:23.742971 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8a00eb5-1e87-491c-964a-30494725730b-tls-certs\") pod \"f8a00eb5-1e87-491c-964a-30494725730b\" (UID: \"f8a00eb5-1e87-491c-964a-30494725730b\") " Jul 05 13:57:23.743105 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:23.743009 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8a00eb5-1e87-491c-964a-30494725730b-model-cache\") pod \"f8a00eb5-1e87-491c-964a-30494725730b\" (UID: \"f8a00eb5-1e87-491c-964a-30494725730b\") " Jul 05 13:57:23.743291 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:23.743161 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x6mcf\" (UniqueName: \"kubernetes.io/projected/f8a00eb5-1e87-491c-964a-30494725730b-kube-api-access-x6mcf\") pod \"f8a00eb5-1e87-491c-964a-30494725730b\" (UID: \"f8a00eb5-1e87-491c-964a-30494725730b\") " Jul 05 13:57:23.743291 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:23.743197 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8a00eb5-1e87-491c-964a-30494725730b-home\") pod \"f8a00eb5-1e87-491c-964a-30494725730b\" (UID: \"f8a00eb5-1e87-491c-964a-30494725730b\") " Jul 05 13:57:23.743291 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:23.743210 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8a00eb5-1e87-491c-964a-30494725730b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f8a00eb5-1e87-491c-964a-30494725730b" (UID: "f8a00eb5-1e87-491c-964a-30494725730b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:57:23.743291 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:23.743239 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8a00eb5-1e87-491c-964a-30494725730b-model-cache" (OuterVolumeSpecName: "model-cache") pod "f8a00eb5-1e87-491c-964a-30494725730b" (UID: "f8a00eb5-1e87-491c-964a-30494725730b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:57:23.743505 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:23.743396 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8a00eb5-1e87-491c-964a-30494725730b-home" (OuterVolumeSpecName: "home") pod "f8a00eb5-1e87-491c-964a-30494725730b" (UID: "f8a00eb5-1e87-491c-964a-30494725730b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:57:23.743560 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:23.743506 2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8a00eb5-1e87-491c-964a-30494725730b-tmp-dir\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:57:23.743560 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:23.743530 2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8a00eb5-1e87-491c-964a-30494725730b-model-cache\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:57:23.743560 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:23.743544 2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8a00eb5-1e87-491c-964a-30494725730b-home\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:57:23.745147 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:23.745117 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8a00eb5-1e87-491c-964a-30494725730b-dshm" (OuterVolumeSpecName: "dshm") pod "f8a00eb5-1e87-491c-964a-30494725730b" (UID: "f8a00eb5-1e87-491c-964a-30494725730b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:57:23.745251 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:23.745144 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8a00eb5-1e87-491c-964a-30494725730b-kube-api-access-x6mcf" (OuterVolumeSpecName: "kube-api-access-x6mcf") pod "f8a00eb5-1e87-491c-964a-30494725730b" (UID: "f8a00eb5-1e87-491c-964a-30494725730b"). InnerVolumeSpecName "kube-api-access-x6mcf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:57:23.745251 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:23.745164 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8a00eb5-1e87-491c-964a-30494725730b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f8a00eb5-1e87-491c-964a-30494725730b" (UID: "f8a00eb5-1e87-491c-964a-30494725730b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:57:23.844274 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:23.844236 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6mcf\" (UniqueName: \"kubernetes.io/projected/f8a00eb5-1e87-491c-964a-30494725730b-kube-api-access-x6mcf\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:57:23.844274 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:23.844271 2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8a00eb5-1e87-491c-964a-30494725730b-dshm\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:57:23.844274 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:23.844280 2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8a00eb5-1e87-491c-964a-30494725730b-tls-certs\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:57:23.862148 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:23.862112 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-6096ec/rate-test-model-6096ec-kserve-95d6bd967-cdkwh" Jul 05 13:57:23.862293 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:23.862156 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-6096ec/rate-test-model-6096ec-kserve-95d6bd967-cdkwh" Jul 05 13:57:23.874945 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:23.874919 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-6096ec/rate-test-model-6096ec-kserve-95d6bd967-cdkwh" Jul 05 13:57:24.142754 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:24.142673 2573 generic.go:356] "Generic (PLEG): container finished" podID="f8a00eb5-1e87-491c-964a-30494725730b" containerID="1f4aeb865eebbe50fa67815627bba979c2ceb17f48bff268a110f82dbe3348e2" exitCode=0 Jul 05 13:57:24.142754 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:24.142714 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6579e1/sub-test-model-6579e1-kserve-5c47796c97-j7lf9" Jul 05 13:57:24.142935 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:24.142762 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6579e1/sub-test-model-6579e1-kserve-5c47796c97-j7lf9" event={"ID":"f8a00eb5-1e87-491c-964a-30494725730b","Type":"ContainerDied","Data":"1f4aeb865eebbe50fa67815627bba979c2ceb17f48bff268a110f82dbe3348e2"} Jul 05 13:57:24.142935 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:24.142798 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6579e1/sub-test-model-6579e1-kserve-5c47796c97-j7lf9" event={"ID":"f8a00eb5-1e87-491c-964a-30494725730b","Type":"ContainerDied","Data":"abec00cf37701d4bc1b931be767775e9c4265babdadc46573ae9a8b02ccd5715"} Jul 05 13:57:24.142935 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:24.142819 2573 scope.go:117] "RemoveContainer" containerID="1f4aeb865eebbe50fa67815627bba979c2ceb17f48bff268a110f82dbe3348e2" Jul 05 13:57:24.152031 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:24.152003 2573 scope.go:117] "RemoveContainer" containerID="1f4aeb865eebbe50fa67815627bba979c2ceb17f48bff268a110f82dbe3348e2" Jul 05 13:57:24.152299 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:57:24.152280 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f4aeb865eebbe50fa67815627bba979c2ceb17f48bff268a110f82dbe3348e2\": container with ID starting with 1f4aeb865eebbe50fa67815627bba979c2ceb17f48bff268a110f82dbe3348e2 not found: ID does not exist" containerID="1f4aeb865eebbe50fa67815627bba979c2ceb17f48bff268a110f82dbe3348e2" Jul 05 13:57:24.152364 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:24.152305 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f4aeb865eebbe50fa67815627bba979c2ceb17f48bff268a110f82dbe3348e2"} err="failed to get container status \"1f4aeb865eebbe50fa67815627bba979c2ceb17f48bff268a110f82dbe3348e2\": rpc error: code = NotFound desc = could not find container \"1f4aeb865eebbe50fa67815627bba979c2ceb17f48bff268a110f82dbe3348e2\": container with ID starting with 1f4aeb865eebbe50fa67815627bba979c2ceb17f48bff268a110f82dbe3348e2 not found: ID does not exist" Jul 05 13:57:24.154963 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:24.154933 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-6096ec/rate-test-model-6096ec-kserve-95d6bd967-cdkwh" Jul 05 13:57:24.169649 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:24.169618 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-6579e1/sub-test-model-6579e1-kserve-5c47796c97-j7lf9"] Jul 05 13:57:24.172174 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:24.172153 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-6579e1/sub-test-model-6579e1-kserve-5c47796c97-j7lf9"] Jul 05 13:57:25.455082 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:25.455041 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:57:25.735742 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:25.735664 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8a00eb5-1e87-491c-964a-30494725730b" path="/var/lib/kubelet/pods/f8a00eb5-1e87-491c-964a-30494725730b/volumes" Jul 05 13:57:50.852220 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:50.852178 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-6579e1/rate-test-model-6579e1-kserve-568cd49f-4zs4f"] Jul 05 13:57:50.852722 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:50.852661 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8a00eb5-1e87-491c-964a-30494725730b" containerName="main" Jul 05 13:57:50.852722 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:50.852681 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8a00eb5-1e87-491c-964a-30494725730b" containerName="main" Jul 05 13:57:50.852840 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:50.852793 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8a00eb5-1e87-491c-964a-30494725730b" containerName="main" Jul 05 13:57:50.857391 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:50.857368 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6579e1/rate-test-model-6579e1-kserve-568cd49f-4zs4f" Jul 05 13:57:50.859763 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:50.859740 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-6579e1\"/\"default-dockercfg-j5j64\"" Jul 05 13:57:50.859865 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:50.859801 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-6579e1\"/\"kube-root-ca.crt\"" Jul 05 13:57:50.860605 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:50.860590 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-6579e1\"/\"openshift-service-ca.crt\"" Jul 05 13:57:50.860687 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:50.860648 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-6579e1\"/\"rate-test-model-6579e1-kserve-self-signed-certs\"" Jul 05 13:57:50.870529 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:50.869571 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-6579e1/rate-test-model-6579e1-kserve-568cd49f-4zs4f"] Jul 05 13:57:50.885016 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:50.884989 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:57:50.983797 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:50.983773 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f687ac3a-8966-475c-9258-b3ad6d726e66-home\") pod \"rate-test-model-6579e1-kserve-568cd49f-4zs4f\" (UID: \"f687ac3a-8966-475c-9258-b3ad6d726e66\") " pod="ai-tenant-e2e-shared-b-6579e1/rate-test-model-6579e1-kserve-568cd49f-4zs4f" Jul 05 13:57:50.983896 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:50.983824 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2m29\" (UniqueName: \"kubernetes.io/projected/f687ac3a-8966-475c-9258-b3ad6d726e66-kube-api-access-p2m29\") pod \"rate-test-model-6579e1-kserve-568cd49f-4zs4f\" (UID: \"f687ac3a-8966-475c-9258-b3ad6d726e66\") " pod="ai-tenant-e2e-shared-b-6579e1/rate-test-model-6579e1-kserve-568cd49f-4zs4f" Jul 05 13:57:50.983896 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:50.983865 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f687ac3a-8966-475c-9258-b3ad6d726e66-model-cache\") pod \"rate-test-model-6579e1-kserve-568cd49f-4zs4f\" (UID: \"f687ac3a-8966-475c-9258-b3ad6d726e66\") " pod="ai-tenant-e2e-shared-b-6579e1/rate-test-model-6579e1-kserve-568cd49f-4zs4f" Jul 05 13:57:50.983896 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:50.983883 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f687ac3a-8966-475c-9258-b3ad6d726e66-tmp-dir\") pod \"rate-test-model-6579e1-kserve-568cd49f-4zs4f\" (UID: \"f687ac3a-8966-475c-9258-b3ad6d726e66\") " pod="ai-tenant-e2e-shared-b-6579e1/rate-test-model-6579e1-kserve-568cd49f-4zs4f" Jul 05 13:57:50.984043 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:50.983910 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f687ac3a-8966-475c-9258-b3ad6d726e66-tls-certs\") pod \"rate-test-model-6579e1-kserve-568cd49f-4zs4f\" (UID: \"f687ac3a-8966-475c-9258-b3ad6d726e66\") " pod="ai-tenant-e2e-shared-b-6579e1/rate-test-model-6579e1-kserve-568cd49f-4zs4f" Jul 05 13:57:50.984043 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:50.983934 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f687ac3a-8966-475c-9258-b3ad6d726e66-dshm\") pod \"rate-test-model-6579e1-kserve-568cd49f-4zs4f\" (UID: \"f687ac3a-8966-475c-9258-b3ad6d726e66\") " pod="ai-tenant-e2e-shared-b-6579e1/rate-test-model-6579e1-kserve-568cd49f-4zs4f" Jul 05 13:57:51.084743 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:51.084698 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f687ac3a-8966-475c-9258-b3ad6d726e66-home\") pod \"rate-test-model-6579e1-kserve-568cd49f-4zs4f\" (UID: \"f687ac3a-8966-475c-9258-b3ad6d726e66\") " pod="ai-tenant-e2e-shared-b-6579e1/rate-test-model-6579e1-kserve-568cd49f-4zs4f" Jul 05 13:57:51.084927 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:51.084760 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2m29\" (UniqueName: \"kubernetes.io/projected/f687ac3a-8966-475c-9258-b3ad6d726e66-kube-api-access-p2m29\") pod \"rate-test-model-6579e1-kserve-568cd49f-4zs4f\" (UID: \"f687ac3a-8966-475c-9258-b3ad6d726e66\") " pod="ai-tenant-e2e-shared-b-6579e1/rate-test-model-6579e1-kserve-568cd49f-4zs4f" Jul 05 13:57:51.084927 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:51.084791 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f687ac3a-8966-475c-9258-b3ad6d726e66-model-cache\") pod \"rate-test-model-6579e1-kserve-568cd49f-4zs4f\" (UID: \"f687ac3a-8966-475c-9258-b3ad6d726e66\") " pod="ai-tenant-e2e-shared-b-6579e1/rate-test-model-6579e1-kserve-568cd49f-4zs4f" Jul 05 13:57:51.084927 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:51.084808 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f687ac3a-8966-475c-9258-b3ad6d726e66-tmp-dir\") pod \"rate-test-model-6579e1-kserve-568cd49f-4zs4f\" (UID: \"f687ac3a-8966-475c-9258-b3ad6d726e66\") " pod="ai-tenant-e2e-shared-b-6579e1/rate-test-model-6579e1-kserve-568cd49f-4zs4f" Jul 05 13:57:51.084927 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:51.084835 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f687ac3a-8966-475c-9258-b3ad6d726e66-tls-certs\") pod \"rate-test-model-6579e1-kserve-568cd49f-4zs4f\" (UID: \"f687ac3a-8966-475c-9258-b3ad6d726e66\") " pod="ai-tenant-e2e-shared-b-6579e1/rate-test-model-6579e1-kserve-568cd49f-4zs4f" Jul 05 13:57:51.084927 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:51.084853 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f687ac3a-8966-475c-9258-b3ad6d726e66-dshm\") pod \"rate-test-model-6579e1-kserve-568cd49f-4zs4f\" (UID: \"f687ac3a-8966-475c-9258-b3ad6d726e66\") " pod="ai-tenant-e2e-shared-b-6579e1/rate-test-model-6579e1-kserve-568cd49f-4zs4f" Jul 05 13:57:51.085248 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:51.085146 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f687ac3a-8966-475c-9258-b3ad6d726e66-home\") pod \"rate-test-model-6579e1-kserve-568cd49f-4zs4f\" (UID: \"f687ac3a-8966-475c-9258-b3ad6d726e66\") " pod="ai-tenant-e2e-shared-b-6579e1/rate-test-model-6579e1-kserve-568cd49f-4zs4f" Jul 05 13:57:51.085248 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:51.085208 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f687ac3a-8966-475c-9258-b3ad6d726e66-model-cache\") pod \"rate-test-model-6579e1-kserve-568cd49f-4zs4f\" (UID: \"f687ac3a-8966-475c-9258-b3ad6d726e66\") " pod="ai-tenant-e2e-shared-b-6579e1/rate-test-model-6579e1-kserve-568cd49f-4zs4f" Jul 05 13:57:51.085348 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:51.085262 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f687ac3a-8966-475c-9258-b3ad6d726e66-tmp-dir\") pod \"rate-test-model-6579e1-kserve-568cd49f-4zs4f\" (UID: \"f687ac3a-8966-475c-9258-b3ad6d726e66\") " pod="ai-tenant-e2e-shared-b-6579e1/rate-test-model-6579e1-kserve-568cd49f-4zs4f" Jul 05 13:57:51.087223 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:51.087201 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f687ac3a-8966-475c-9258-b3ad6d726e66-dshm\") pod \"rate-test-model-6579e1-kserve-568cd49f-4zs4f\" (UID: \"f687ac3a-8966-475c-9258-b3ad6d726e66\") " pod="ai-tenant-e2e-shared-b-6579e1/rate-test-model-6579e1-kserve-568cd49f-4zs4f" Jul 05 13:57:51.087417 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:51.087397 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f687ac3a-8966-475c-9258-b3ad6d726e66-tls-certs\") pod \"rate-test-model-6579e1-kserve-568cd49f-4zs4f\" (UID: \"f687ac3a-8966-475c-9258-b3ad6d726e66\") " pod="ai-tenant-e2e-shared-b-6579e1/rate-test-model-6579e1-kserve-568cd49f-4zs4f" Jul 05 13:57:51.092543 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:51.092519 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2m29\" (UniqueName: \"kubernetes.io/projected/f687ac3a-8966-475c-9258-b3ad6d726e66-kube-api-access-p2m29\") pod \"rate-test-model-6579e1-kserve-568cd49f-4zs4f\" (UID: \"f687ac3a-8966-475c-9258-b3ad6d726e66\") " pod="ai-tenant-e2e-shared-b-6579e1/rate-test-model-6579e1-kserve-568cd49f-4zs4f" Jul 05 13:57:51.169253 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:51.169198 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6579e1/rate-test-model-6579e1-kserve-568cd49f-4zs4f" Jul 05 13:57:51.304152 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:51.304122 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-6579e1/rate-test-model-6579e1-kserve-568cd49f-4zs4f"] Jul 05 13:57:51.307250 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:57:51.307230 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf687ac3a_8966_475c_9258_b3ad6d726e66.slice/crio-e6a4fab1462e52533b582e469aeea50cf8a87cb12e24e7a26619f1687bff01dc WatchSource:0}: Error finding container e6a4fab1462e52533b582e469aeea50cf8a87cb12e24e7a26619f1687bff01dc: Status 404 returned error can't find the container with id e6a4fab1462e52533b582e469aeea50cf8a87cb12e24e7a26619f1687bff01dc Jul 05 13:57:52.289395 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:52.289362 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6579e1/rate-test-model-6579e1-kserve-568cd49f-4zs4f" event={"ID":"f687ac3a-8966-475c-9258-b3ad6d726e66","Type":"ContainerStarted","Data":"f2821c4a172d1427f24c4ae8044dd08a0710d360cdf4f7306c6fdbdaeacb2734"} Jul 05 13:57:52.289809 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:52.289404 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6579e1/rate-test-model-6579e1-kserve-568cd49f-4zs4f" event={"ID":"f687ac3a-8966-475c-9258-b3ad6d726e66","Type":"ContainerStarted","Data":"e6a4fab1462e52533b582e469aeea50cf8a87cb12e24e7a26619f1687bff01dc"} Jul 05 13:57:52.305758 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:52.305714 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-6579e1/rate-test-model-6579e1-kserve-568cd49f-4zs4f" podStartSLOduration=2.305699685 podStartE2EDuration="2.305699685s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:57:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:57:52.30521798 +0000 UTC m=+2345.200876070" watchObservedRunningTime="2026-07-05 13:57:52.305699685 +0000 UTC m=+2345.201357773" Jul 05 13:57:55.754437 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:57:55.754404 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:58:00.719530 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:00.719487 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:58:01.169945 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:01.169909 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-6579e1/rate-test-model-6579e1-kserve-568cd49f-4zs4f" Jul 05 13:58:01.170154 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:01.169996 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-6579e1/rate-test-model-6579e1-kserve-568cd49f-4zs4f" Jul 05 13:58:01.182745 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:01.182716 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-6579e1/rate-test-model-6579e1-kserve-568cd49f-4zs4f" Jul 05 13:58:01.345250 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:01.345223 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-6579e1/rate-test-model-6579e1-kserve-568cd49f-4zs4f" Jul 05 13:58:11.277043 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:11.277011 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:58:11.377042 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:11.376996 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:58:14.551450 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:14.551414 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:58:29.251285 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:29.251251 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:58:31.391639 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:31.391604 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:58:31.491188 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:31.491130 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:58:34.735901 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:34.735871 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-6096ec/rate-test-model-6096ec-kserve-95d6bd967-cdkwh"] Jul 05 13:58:34.756203 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:34.756173 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:58:35.763946 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:35.763906 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-6579e1/rate-test-model-6579e1-kserve-568cd49f-4zs4f"] Jul 05 13:58:35.775384 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:35.775354 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:58:36.784561 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:36.784528 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:58:37.165054 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:37.164931 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:58:37.264717 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:37.264672 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:58:39.475684 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:39.475649 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:58:39.574810 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:39.574775 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:58:42.238071 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:42.238031 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:58:46.753494 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:46.753459 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:58:49.755659 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:49.755604 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-6096ec/rate-test-model-6096ec-kserve-95d6bd967-cdkwh" podUID="8255612e-0a8c-40b0-88ef-c94f5251d4c1" containerName="main" containerID="cri-o://a1e93602186364d9502f7f9d798750a01e9188e348ee799249c12dae055b112b" gracePeriod=45 Jul 05 13:58:50.004366 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:50.004341 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6096ec/rate-test-model-6096ec-kserve-95d6bd967-cdkwh" Jul 05 13:58:50.093353 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:50.093275 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8255612e-0a8c-40b0-88ef-c94f5251d4c1-home\") pod \"8255612e-0a8c-40b0-88ef-c94f5251d4c1\" (UID: \"8255612e-0a8c-40b0-88ef-c94f5251d4c1\") " Jul 05 13:58:50.093353 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:50.093312 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8255612e-0a8c-40b0-88ef-c94f5251d4c1-dshm\") pod \"8255612e-0a8c-40b0-88ef-c94f5251d4c1\" (UID: \"8255612e-0a8c-40b0-88ef-c94f5251d4c1\") " Jul 05 13:58:50.093353 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:50.093347 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8255612e-0a8c-40b0-88ef-c94f5251d4c1-tmp-dir\") pod \"8255612e-0a8c-40b0-88ef-c94f5251d4c1\" (UID: \"8255612e-0a8c-40b0-88ef-c94f5251d4c1\") " Jul 05 13:58:50.093621 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:50.093377 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m5qbc\" (UniqueName: \"kubernetes.io/projected/8255612e-0a8c-40b0-88ef-c94f5251d4c1-kube-api-access-m5qbc\") pod \"8255612e-0a8c-40b0-88ef-c94f5251d4c1\" (UID: \"8255612e-0a8c-40b0-88ef-c94f5251d4c1\") " Jul 05 13:58:50.093621 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:50.093410 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8255612e-0a8c-40b0-88ef-c94f5251d4c1-model-cache\") pod \"8255612e-0a8c-40b0-88ef-c94f5251d4c1\" (UID: \"8255612e-0a8c-40b0-88ef-c94f5251d4c1\") " Jul 05 13:58:50.093621 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:50.093458 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8255612e-0a8c-40b0-88ef-c94f5251d4c1-tls-certs\") pod \"8255612e-0a8c-40b0-88ef-c94f5251d4c1\" (UID: \"8255612e-0a8c-40b0-88ef-c94f5251d4c1\") " Jul 05 13:58:50.093621 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:50.093581 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8255612e-0a8c-40b0-88ef-c94f5251d4c1-home" (OuterVolumeSpecName: "home") pod "8255612e-0a8c-40b0-88ef-c94f5251d4c1" (UID: "8255612e-0a8c-40b0-88ef-c94f5251d4c1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:58:50.093810 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:50.093635 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8255612e-0a8c-40b0-88ef-c94f5251d4c1-model-cache" (OuterVolumeSpecName: "model-cache") pod "8255612e-0a8c-40b0-88ef-c94f5251d4c1" (UID: "8255612e-0a8c-40b0-88ef-c94f5251d4c1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:58:50.093810 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:50.093665 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8255612e-0a8c-40b0-88ef-c94f5251d4c1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8255612e-0a8c-40b0-88ef-c94f5251d4c1" (UID: "8255612e-0a8c-40b0-88ef-c94f5251d4c1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:58:50.093810 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:50.093787 2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8255612e-0a8c-40b0-88ef-c94f5251d4c1-home\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:58:50.093810 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:50.093798 2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8255612e-0a8c-40b0-88ef-c94f5251d4c1-tmp-dir\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:58:50.093810 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:50.093809 2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8255612e-0a8c-40b0-88ef-c94f5251d4c1-model-cache\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:58:50.095360 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:50.095337 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8255612e-0a8c-40b0-88ef-c94f5251d4c1-dshm" (OuterVolumeSpecName: "dshm") pod "8255612e-0a8c-40b0-88ef-c94f5251d4c1" (UID: "8255612e-0a8c-40b0-88ef-c94f5251d4c1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:58:50.095521 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:50.095491 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8255612e-0a8c-40b0-88ef-c94f5251d4c1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8255612e-0a8c-40b0-88ef-c94f5251d4c1" (UID: "8255612e-0a8c-40b0-88ef-c94f5251d4c1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:58:50.095584 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:50.095525 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8255612e-0a8c-40b0-88ef-c94f5251d4c1-kube-api-access-m5qbc" (OuterVolumeSpecName: "kube-api-access-m5qbc") pod "8255612e-0a8c-40b0-88ef-c94f5251d4c1" (UID: "8255612e-0a8c-40b0-88ef-c94f5251d4c1"). InnerVolumeSpecName "kube-api-access-m5qbc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:58:50.195054 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:50.195022 2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8255612e-0a8c-40b0-88ef-c94f5251d4c1-dshm\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:58:50.195054 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:50.195050 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m5qbc\" (UniqueName: \"kubernetes.io/projected/8255612e-0a8c-40b0-88ef-c94f5251d4c1-kube-api-access-m5qbc\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:58:50.195054 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:50.195061 2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8255612e-0a8c-40b0-88ef-c94f5251d4c1-tls-certs\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:58:50.590504 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:50.590472 2573 generic.go:356] "Generic (PLEG): container finished" podID="8255612e-0a8c-40b0-88ef-c94f5251d4c1" containerID="a1e93602186364d9502f7f9d798750a01e9188e348ee799249c12dae055b112b" exitCode=0 Jul 05 13:58:50.590689 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:50.590513 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6096ec/rate-test-model-6096ec-kserve-95d6bd967-cdkwh" Jul 05 13:58:50.590689 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:50.590561 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6096ec/rate-test-model-6096ec-kserve-95d6bd967-cdkwh" event={"ID":"8255612e-0a8c-40b0-88ef-c94f5251d4c1","Type":"ContainerDied","Data":"a1e93602186364d9502f7f9d798750a01e9188e348ee799249c12dae055b112b"} Jul 05 13:58:50.590689 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:50.590603 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6096ec/rate-test-model-6096ec-kserve-95d6bd967-cdkwh" event={"ID":"8255612e-0a8c-40b0-88ef-c94f5251d4c1","Type":"ContainerDied","Data":"abb2288d90aba4c45073aaa161055f98cda037737ec2d3df0232d3179ac1e23e"} Jul 05 13:58:50.590689 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:50.590620 2573 scope.go:117] "RemoveContainer" containerID="a1e93602186364d9502f7f9d798750a01e9188e348ee799249c12dae055b112b" Jul 05 13:58:50.599840 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:50.599825 2573 scope.go:117] "RemoveContainer" containerID="a1e93602186364d9502f7f9d798750a01e9188e348ee799249c12dae055b112b" Jul 05 13:58:50.600087 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:58:50.600072 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1e93602186364d9502f7f9d798750a01e9188e348ee799249c12dae055b112b\": container with ID starting with a1e93602186364d9502f7f9d798750a01e9188e348ee799249c12dae055b112b not found: ID does not exist" containerID="a1e93602186364d9502f7f9d798750a01e9188e348ee799249c12dae055b112b" Jul 05 13:58:50.600146 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:50.600094 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1e93602186364d9502f7f9d798750a01e9188e348ee799249c12dae055b112b"} err="failed to get container status \"a1e93602186364d9502f7f9d798750a01e9188e348ee799249c12dae055b112b\": rpc error: code = NotFound desc = could not find container \"a1e93602186364d9502f7f9d798750a01e9188e348ee799249c12dae055b112b\": container with ID starting with a1e93602186364d9502f7f9d798750a01e9188e348ee799249c12dae055b112b not found: ID does not exist" Jul 05 13:58:50.616032 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:50.616009 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-6096ec/rate-test-model-6096ec-kserve-95d6bd967-cdkwh"] Jul 05 13:58:50.618887 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:50.618866 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-6096ec/rate-test-model-6096ec-kserve-95d6bd967-cdkwh"] Jul 05 13:58:50.783878 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:50.783822 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-6579e1/rate-test-model-6579e1-kserve-568cd49f-4zs4f" podUID="f687ac3a-8966-475c-9258-b3ad6d726e66" containerName="main" containerID="cri-o://f2821c4a172d1427f24c4ae8044dd08a0710d360cdf4f7306c6fdbdaeacb2734" gracePeriod=45 Jul 05 13:58:51.036280 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:51.036253 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6579e1/rate-test-model-6579e1-kserve-568cd49f-4zs4f" Jul 05 13:58:51.102050 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:51.102016 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p2m29\" (UniqueName: \"kubernetes.io/projected/f687ac3a-8966-475c-9258-b3ad6d726e66-kube-api-access-p2m29\") pod \"f687ac3a-8966-475c-9258-b3ad6d726e66\" (UID: \"f687ac3a-8966-475c-9258-b3ad6d726e66\") " Jul 05 13:58:51.102050 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:51.102061 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f687ac3a-8966-475c-9258-b3ad6d726e66-model-cache\") pod \"f687ac3a-8966-475c-9258-b3ad6d726e66\" (UID: \"f687ac3a-8966-475c-9258-b3ad6d726e66\") " Jul 05 13:58:51.102330 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:51.102087 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f687ac3a-8966-475c-9258-b3ad6d726e66-tls-certs\") pod \"f687ac3a-8966-475c-9258-b3ad6d726e66\" (UID: \"f687ac3a-8966-475c-9258-b3ad6d726e66\") " Jul 05 13:58:51.102330 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:51.102120 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f687ac3a-8966-475c-9258-b3ad6d726e66-tmp-dir\") pod \"f687ac3a-8966-475c-9258-b3ad6d726e66\" (UID: \"f687ac3a-8966-475c-9258-b3ad6d726e66\") " Jul 05 13:58:51.102330 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:51.102233 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f687ac3a-8966-475c-9258-b3ad6d726e66-dshm\") pod \"f687ac3a-8966-475c-9258-b3ad6d726e66\" (UID: \"f687ac3a-8966-475c-9258-b3ad6d726e66\") " Jul 05 13:58:51.102330 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:51.102280 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f687ac3a-8966-475c-9258-b3ad6d726e66-home\") pod \"f687ac3a-8966-475c-9258-b3ad6d726e66\" (UID: \"f687ac3a-8966-475c-9258-b3ad6d726e66\") " Jul 05 13:58:51.102535 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:51.102378 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f687ac3a-8966-475c-9258-b3ad6d726e66-model-cache" (OuterVolumeSpecName: "model-cache") pod "f687ac3a-8966-475c-9258-b3ad6d726e66" (UID: "f687ac3a-8966-475c-9258-b3ad6d726e66"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:58:51.102535 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:51.102386 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f687ac3a-8966-475c-9258-b3ad6d726e66-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f687ac3a-8966-475c-9258-b3ad6d726e66" (UID: "f687ac3a-8966-475c-9258-b3ad6d726e66"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:58:51.102655 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:51.102541 2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f687ac3a-8966-475c-9258-b3ad6d726e66-model-cache\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:58:51.102655 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:51.102559 2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f687ac3a-8966-475c-9258-b3ad6d726e66-tmp-dir\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:58:51.102655 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:51.102589 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f687ac3a-8966-475c-9258-b3ad6d726e66-home" (OuterVolumeSpecName: "home") pod "f687ac3a-8966-475c-9258-b3ad6d726e66" (UID: "f687ac3a-8966-475c-9258-b3ad6d726e66"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:58:51.104370 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:51.104341 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f687ac3a-8966-475c-9258-b3ad6d726e66-dshm" (OuterVolumeSpecName: "dshm") pod "f687ac3a-8966-475c-9258-b3ad6d726e66" (UID: "f687ac3a-8966-475c-9258-b3ad6d726e66"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:58:51.104476 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:51.104375 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f687ac3a-8966-475c-9258-b3ad6d726e66-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f687ac3a-8966-475c-9258-b3ad6d726e66" (UID: "f687ac3a-8966-475c-9258-b3ad6d726e66"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:58:51.104476 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:51.104372 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f687ac3a-8966-475c-9258-b3ad6d726e66-kube-api-access-p2m29" (OuterVolumeSpecName: "kube-api-access-p2m29") pod "f687ac3a-8966-475c-9258-b3ad6d726e66" (UID: "f687ac3a-8966-475c-9258-b3ad6d726e66"). InnerVolumeSpecName "kube-api-access-p2m29". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:58:51.203735 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:51.203708 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p2m29\" (UniqueName: \"kubernetes.io/projected/f687ac3a-8966-475c-9258-b3ad6d726e66-kube-api-access-p2m29\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:58:51.203735 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:51.203734 2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f687ac3a-8966-475c-9258-b3ad6d726e66-tls-certs\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:58:51.203920 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:51.203743 2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f687ac3a-8966-475c-9258-b3ad6d726e66-dshm\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:58:51.203920 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:51.203751 2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f687ac3a-8966-475c-9258-b3ad6d726e66-home\") on node \"ip-10-0-139-155.ec2.internal\" DevicePath \"\"" Jul 05 13:58:51.597226 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:51.597142 2573 generic.go:356] "Generic (PLEG): container finished" podID="f687ac3a-8966-475c-9258-b3ad6d726e66" containerID="f2821c4a172d1427f24c4ae8044dd08a0710d360cdf4f7306c6fdbdaeacb2734" exitCode=0 Jul 05 13:58:51.597226 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:51.597183 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6579e1/rate-test-model-6579e1-kserve-568cd49f-4zs4f" Jul 05 13:58:51.597449 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:51.597224 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6579e1/rate-test-model-6579e1-kserve-568cd49f-4zs4f" event={"ID":"f687ac3a-8966-475c-9258-b3ad6d726e66","Type":"ContainerDied","Data":"f2821c4a172d1427f24c4ae8044dd08a0710d360cdf4f7306c6fdbdaeacb2734"} Jul 05 13:58:51.597449 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:51.597262 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6579e1/rate-test-model-6579e1-kserve-568cd49f-4zs4f" event={"ID":"f687ac3a-8966-475c-9258-b3ad6d726e66","Type":"ContainerDied","Data":"e6a4fab1462e52533b582e469aeea50cf8a87cb12e24e7a26619f1687bff01dc"} Jul 05 13:58:51.597449 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:51.597283 2573 scope.go:117] "RemoveContainer" containerID="f2821c4a172d1427f24c4ae8044dd08a0710d360cdf4f7306c6fdbdaeacb2734" Jul 05 13:58:51.606390 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:51.606373 2573 scope.go:117] "RemoveContainer" containerID="f2821c4a172d1427f24c4ae8044dd08a0710d360cdf4f7306c6fdbdaeacb2734" Jul 05 13:58:51.606634 ip-10-0-139-155 kubenswrapper[2573]: E0705 13:58:51.606614 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2821c4a172d1427f24c4ae8044dd08a0710d360cdf4f7306c6fdbdaeacb2734\": container with ID starting with f2821c4a172d1427f24c4ae8044dd08a0710d360cdf4f7306c6fdbdaeacb2734 not found: ID does not exist" containerID="f2821c4a172d1427f24c4ae8044dd08a0710d360cdf4f7306c6fdbdaeacb2734" Jul 05 13:58:51.606684 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:51.606645 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2821c4a172d1427f24c4ae8044dd08a0710d360cdf4f7306c6fdbdaeacb2734"} err="failed to get container status \"f2821c4a172d1427f24c4ae8044dd08a0710d360cdf4f7306c6fdbdaeacb2734\": rpc error: code = NotFound desc = could not find container \"f2821c4a172d1427f24c4ae8044dd08a0710d360cdf4f7306c6fdbdaeacb2734\": container with ID starting with f2821c4a172d1427f24c4ae8044dd08a0710d360cdf4f7306c6fdbdaeacb2734 not found: ID does not exist" Jul 05 13:58:51.618341 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:51.618316 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-6579e1/rate-test-model-6579e1-kserve-568cd49f-4zs4f"] Jul 05 13:58:51.624434 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:51.624411 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-6579e1/rate-test-model-6579e1-kserve-568cd49f-4zs4f"] Jul 05 13:58:51.736304 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:51.736271 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8255612e-0a8c-40b0-88ef-c94f5251d4c1" path="/var/lib/kubelet/pods/8255612e-0a8c-40b0-88ef-c94f5251d4c1/volumes" Jul 05 13:58:51.736671 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:58:51.736658 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f687ac3a-8966-475c-9258-b3ad6d726e66" path="/var/lib/kubelet/pods/f687ac3a-8966-475c-9258-b3ad6d726e66/volumes" Jul 05 13:59:07.565619 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:07.565581 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:59:19.785211 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:19.785131 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:59:20.065942 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:20.065854 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:59:20.290829 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:20.290797 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:59:20.447720 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:20.447683 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:59:20.846028 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:20.845927 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:59:20.945766 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:20.945727 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:59:23.148506 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:23.148475 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wzt5q"] Jul 05 13:59:27.865632 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:27.865595 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:59:27.934367 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:27.934326 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:59:28.136546 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:28.136449 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:59:28.355972 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:28.355922 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:59:28.752110 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:28.752075 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6fc7fd77f-4bqc2_19474780-521c-42be-904f-5d3d59bd53bf/authorino/0.log" Jul 05 13:59:29.503910 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:29.503861 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:59:29.665592 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:29.665555 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:59:33.703537 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:33.703508 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29720970-dmn4z_7551d605-e483-4238-81f3-b0fd5ee6a226/cleanup/0.log" Jul 05 13:59:34.498516 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:34.498487 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b4f468d94-l5zdz_6c0a4bc3-5917-4fc7-88e6-38c057669a80/copy-manifests/0.log" Jul 05 13:59:34.519255 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:34.519226 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b4f468d94-l5zdz_6c0a4bc3-5917-4fc7-88e6-38c057669a80/manager/0.log" Jul 05 13:59:35.379169 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:35.379137 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sj8gs_25a4701d-6fa6-4415-9125-b6c8ecea61ef/util/0.log" Jul 05 13:59:35.385384 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:35.385363 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sj8gs_25a4701d-6fa6-4415-9125-b6c8ecea61ef/pull/0.log" Jul 05 13:59:35.391764 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:35.391742 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sj8gs_25a4701d-6fa6-4415-9125-b6c8ecea61ef/extract/0.log" Jul 05 13:59:35.496992 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:35.496947 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rdm7q_b8c4ff8d-80dc-40c8-b995-234066e407ac/util/0.log" Jul 05 13:59:35.503550 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:35.503526 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rdm7q_b8c4ff8d-80dc-40c8-b995-234066e407ac/pull/0.log" Jul 05 13:59:35.509890 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:35.509865 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rdm7q_b8c4ff8d-80dc-40c8-b995-234066e407ac/extract/0.log" Jul 05 13:59:35.615267 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:35.615226 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zkc22_7b7cee0a-df79-414c-a597-f004df20451a/util/0.log" Jul 05 13:59:35.621343 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:35.621319 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zkc22_7b7cee0a-df79-414c-a597-f004df20451a/pull/0.log" Jul 05 13:59:35.627014 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:35.626991 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zkc22_7b7cee0a-df79-414c-a597-f004df20451a/extract/0.log" Jul 05 13:59:35.735244 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:35.735213 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ll22v_c35c9430-6adb-45bc-ac83-668b51f8e611/util/0.log" Jul 05 13:59:35.741485 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:35.741462 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ll22v_c35c9430-6adb-45bc-ac83-668b51f8e611/pull/0.log" Jul 05 13:59:35.747638 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:35.747621 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ll22v_c35c9430-6adb-45bc-ac83-668b51f8e611/extract/0.log" Jul 05 13:59:35.863966 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:35.863920 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6fc7fd77f-4bqc2_19474780-521c-42be-904f-5d3d59bd53bf/authorino/0.log" Jul 05 13:59:36.203180 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:36.203154 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-j5lkx_c574c726-dea7-45f6-86bb-84a22216d065/kuadrant-console-plugin/0.log" Jul 05 13:59:36.536735 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:36.536660 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-wzt5q_2273f7be-a201-45c8-8e69-232ea0156608/limitador/0.log" Jul 05 13:59:37.325491 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:37.325459 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-67cd4f9bbf-nw66v_2d3096e6-d86f-4547-864e-dc4633eeaf9f/kube-auth-proxy/0.log" Jul 05 13:59:38.553718 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:38.553687 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-sl6tv_3de7428c-db34-439c-92b2-a6cd6e2336b4/main/0.log" Jul 05 13:59:38.662842 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:38.662811 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-69fnx_930ebf4c-3588-40da-9e0a-eb8e6902317f/main/0.log" Jul 05 13:59:42.495170 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:42.495134 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-tjmkf/must-gather-nmc6r"] Jul 05 13:59:42.495579 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:42.495457 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f687ac3a-8966-475c-9258-b3ad6d726e66" containerName="main" Jul 05 13:59:42.495579 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:42.495468 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="f687ac3a-8966-475c-9258-b3ad6d726e66" containerName="main" Jul 05 13:59:42.495579 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:42.495485 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8255612e-0a8c-40b0-88ef-c94f5251d4c1" containerName="main" Jul 05 13:59:42.495579 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:42.495490 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="8255612e-0a8c-40b0-88ef-c94f5251d4c1" containerName="main" Jul 05 13:59:42.495579 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:42.495547 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="f687ac3a-8966-475c-9258-b3ad6d726e66" containerName="main" Jul 05 13:59:42.495579 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:42.495560 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="8255612e-0a8c-40b0-88ef-c94f5251d4c1" containerName="main" Jul 05 13:59:42.498675 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:42.498651 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tjmkf/must-gather-nmc6r" Jul 05 13:59:42.500968 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:42.500932 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tjmkf\"/\"openshift-service-ca.crt\"" Jul 05 13:59:42.501072 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:42.500994 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-tjmkf\"/\"default-dockercfg-6mf8z\"" Jul 05 13:59:42.501865 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:42.501849 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tjmkf\"/\"kube-root-ca.crt\"" Jul 05 13:59:42.507303 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:42.507258 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:59:42.507875 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:42.507853 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tjmkf/must-gather-nmc6r"] Jul 05 13:59:42.554258 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:42.554220 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/dfb6519d-19da-45d5-9b35-588b36f3933f-must-gather-output\") pod \"must-gather-nmc6r\" (UID: \"dfb6519d-19da-45d5-9b35-588b36f3933f\") " pod="openshift-must-gather-tjmkf/must-gather-nmc6r" Jul 05 13:59:42.554435 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:42.554276 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gx5db\" (UniqueName: \"kubernetes.io/projected/dfb6519d-19da-45d5-9b35-588b36f3933f-kube-api-access-gx5db\") pod \"must-gather-nmc6r\" (UID: \"dfb6519d-19da-45d5-9b35-588b36f3933f\") " pod="openshift-must-gather-tjmkf/must-gather-nmc6r" Jul 05 13:59:42.570904 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:42.570852 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" limit=1024 Jul 05 13:59:42.654740 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:42.654708 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/dfb6519d-19da-45d5-9b35-588b36f3933f-must-gather-output\") pod \"must-gather-nmc6r\" (UID: \"dfb6519d-19da-45d5-9b35-588b36f3933f\") " pod="openshift-must-gather-tjmkf/must-gather-nmc6r" Jul 05 13:59:42.654884 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:42.654756 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gx5db\" (UniqueName: \"kubernetes.io/projected/dfb6519d-19da-45d5-9b35-588b36f3933f-kube-api-access-gx5db\") pod \"must-gather-nmc6r\" (UID: \"dfb6519d-19da-45d5-9b35-588b36f3933f\") " pod="openshift-must-gather-tjmkf/must-gather-nmc6r" Jul 05 13:59:42.655075 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:42.655055 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/dfb6519d-19da-45d5-9b35-588b36f3933f-must-gather-output\") pod \"must-gather-nmc6r\" (UID: \"dfb6519d-19da-45d5-9b35-588b36f3933f\") " pod="openshift-must-gather-tjmkf/must-gather-nmc6r" Jul 05 13:59:42.663066 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:42.663046 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gx5db\" (UniqueName: \"kubernetes.io/projected/dfb6519d-19da-45d5-9b35-588b36f3933f-kube-api-access-gx5db\") pod \"must-gather-nmc6r\" (UID: \"dfb6519d-19da-45d5-9b35-588b36f3933f\") " pod="openshift-must-gather-tjmkf/must-gather-nmc6r" Jul 05 13:59:42.808279 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:42.808202 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tjmkf/must-gather-nmc6r" Jul 05 13:59:42.939588 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:42.939551 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tjmkf/must-gather-nmc6r"] Jul 05 13:59:42.940712 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:59:42.940689 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddfb6519d_19da_45d5_9b35_588b36f3933f.slice/crio-93832b980b503bf739175fd5706b1c4b66b566e96805806872ce7444ac3dbf03 WatchSource:0}: Error finding container 93832b980b503bf739175fd5706b1c4b66b566e96805806872ce7444ac3dbf03: Status 404 returned error can't find the container with id 93832b980b503bf739175fd5706b1c4b66b566e96805806872ce7444ac3dbf03 Jul 05 13:59:42.942530 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:42.942512 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 05 13:59:43.864253 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:43.864158 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tjmkf/must-gather-nmc6r" event={"ID":"dfb6519d-19da-45d5-9b35-588b36f3933f","Type":"ContainerStarted","Data":"9e8719a03c27a679308155e98ceccc2fc01b059fb6877dd473c196c24db6a38d"} Jul 05 13:59:43.864253 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:43.864210 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tjmkf/must-gather-nmc6r" event={"ID":"dfb6519d-19da-45d5-9b35-588b36f3933f","Type":"ContainerStarted","Data":"a5396c0c128f1f0cae113ac227e69a1e3d58e826734b90544926b70f37830598"} Jul 05 13:59:43.864253 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:43.864227 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tjmkf/must-gather-nmc6r" event={"ID":"dfb6519d-19da-45d5-9b35-588b36f3933f","Type":"ContainerStarted","Data":"93832b980b503bf739175fd5706b1c4b66b566e96805806872ce7444ac3dbf03"} Jul 05 13:59:43.883232 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:43.883167 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-tjmkf/must-gather-nmc6r" podStartSLOduration=1.317147169 podStartE2EDuration="1.883146292s" totalImagesPullingTime="565.999123ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:59:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:59:43.880569136 +0000 UTC m=+2456.776227224" watchObservedRunningTime="2026-07-05 13:59:43.883146292 +0000 UTC m=+2456.778804382" Jul 05 13:59:45.189966 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:45.189911 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-l6cc6_33610912-09f3-406f-be0f-a6ffbff4c9ba/global-pull-secret-syncer/0.log" Jul 05 13:59:45.253779 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:45.253750 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-dr5b4_514779be-55a5-41cb-9bce-42dd1c7398e5/konnectivity-agent/0.log" Jul 05 13:59:45.363113 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:45.363080 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-155.ec2.internal_f7be285f4bc4ee8d9639260737851004/haproxy/0.log" Jul 05 13:59:49.887837 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:49.887736 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sj8gs_25a4701d-6fa6-4415-9125-b6c8ecea61ef/extract/0.log" Jul 05 13:59:49.913454 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:49.913348 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sj8gs_25a4701d-6fa6-4415-9125-b6c8ecea61ef/util/0.log" Jul 05 13:59:49.939311 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:49.939282 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sj8gs_25a4701d-6fa6-4415-9125-b6c8ecea61ef/pull/0.log" Jul 05 13:59:49.968836 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:49.968809 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rdm7q_b8c4ff8d-80dc-40c8-b995-234066e407ac/extract/0.log" Jul 05 13:59:50.012253 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:50.012176 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rdm7q_b8c4ff8d-80dc-40c8-b995-234066e407ac/util/0.log" Jul 05 13:59:50.033940 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:50.033899 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rdm7q_b8c4ff8d-80dc-40c8-b995-234066e407ac/pull/0.log" Jul 05 13:59:50.060454 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:50.060420 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zkc22_7b7cee0a-df79-414c-a597-f004df20451a/extract/0.log" Jul 05 13:59:50.082649 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:50.082611 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zkc22_7b7cee0a-df79-414c-a597-f004df20451a/util/0.log" Jul 05 13:59:50.104548 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:50.104518 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zkc22_7b7cee0a-df79-414c-a597-f004df20451a/pull/0.log" Jul 05 13:59:50.133909 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:50.133868 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ll22v_c35c9430-6adb-45bc-ac83-668b51f8e611/extract/0.log" Jul 05 13:59:50.156521 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:50.156446 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ll22v_c35c9430-6adb-45bc-ac83-668b51f8e611/util/0.log" Jul 05 13:59:50.181913 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:50.181882 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ll22v_c35c9430-6adb-45bc-ac83-668b51f8e611/pull/0.log" Jul 05 13:59:50.651094 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:50.650688 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6fc7fd77f-4bqc2_19474780-521c-42be-904f-5d3d59bd53bf/authorino/0.log" Jul 05 13:59:50.740204 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:50.740166 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-j5lkx_c574c726-dea7-45f6-86bb-84a22216d065/kuadrant-console-plugin/0.log" Jul 05 13:59:50.921407 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:50.921333 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-wzt5q_2273f7be-a201-45c8-8e69-232ea0156608/limitador/0.log" Jul 05 13:59:53.131776 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:53.131743 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-847bdf55cb-xdd6s_161555b1-1894-4a2c-9770-faac3772cb12/metrics-server/0.log" Jul 05 13:59:53.158564 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:53.158515 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-b5df8_46d56b08-161c-465d-9adc-dbd9273e69c7/monitoring-plugin/0.log" Jul 05 13:59:53.409329 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:53.409239 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6zff6_ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1/node-exporter/0.log" Jul 05 13:59:53.448738 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:53.448695 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6zff6_ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1/kube-rbac-proxy/0.log" Jul 05 13:59:53.474307 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:53.474273 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6zff6_ee3dad58-0d5b-42c7-9047-fa5b9e2e1ad1/init-textfile/0.log" Jul 05 13:59:53.601693 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:53.601660 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-hvhgg_5711bc78-2a69-45f9-921f-0e00b7d51e39/kube-rbac-proxy-main/0.log" Jul 05 13:59:53.626016 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:53.625945 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-hvhgg_5711bc78-2a69-45f9-921f-0e00b7d51e39/kube-rbac-proxy-self/0.log" Jul 05 13:59:53.646732 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:53.646690 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-hvhgg_5711bc78-2a69-45f9-921f-0e00b7d51e39/openshift-state-metrics/0.log" Jul 05 13:59:53.919857 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:53.919825 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-mxdnm_6936197e-9cd2-4582-a142-0ee0fff64166/prometheus-operator-admission-webhook/0.log" Jul 05 13:59:53.950902 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:53.950869 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6d456696c8-75wb6_55118c56-ac0a-4696-aefb-dd8bcb891cc7/telemeter-client/0.log" Jul 05 13:59:53.971507 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:53.971473 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6d456696c8-75wb6_55118c56-ac0a-4696-aefb-dd8bcb891cc7/reload/0.log" Jul 05 13:59:53.994997 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:53.994926 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6d456696c8-75wb6_55118c56-ac0a-4696-aefb-dd8bcb891cc7/kube-rbac-proxy/0.log" Jul 05 13:59:55.060479 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:55.060443 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-tjmkf/perf-node-gather-daemonset-k4qqz"] Jul 05 13:59:55.064826 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:55.064793 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tjmkf/perf-node-gather-daemonset-k4qqz" Jul 05 13:59:55.075479 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:55.075191 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0750be83-08be-440b-b532-6fe940851317-sys\") pod \"perf-node-gather-daemonset-k4qqz\" (UID: \"0750be83-08be-440b-b532-6fe940851317\") " pod="openshift-must-gather-tjmkf/perf-node-gather-daemonset-k4qqz" Jul 05 13:59:55.075479 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:55.075272 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0750be83-08be-440b-b532-6fe940851317-proc\") pod \"perf-node-gather-daemonset-k4qqz\" (UID: \"0750be83-08be-440b-b532-6fe940851317\") " pod="openshift-must-gather-tjmkf/perf-node-gather-daemonset-k4qqz" Jul 05 13:59:55.075479 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:55.075309 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0750be83-08be-440b-b532-6fe940851317-lib-modules\") pod \"perf-node-gather-daemonset-k4qqz\" (UID: \"0750be83-08be-440b-b532-6fe940851317\") " pod="openshift-must-gather-tjmkf/perf-node-gather-daemonset-k4qqz" Jul 05 13:59:55.075479 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:55.075365 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0750be83-08be-440b-b532-6fe940851317-podres\") pod \"perf-node-gather-daemonset-k4qqz\" (UID: \"0750be83-08be-440b-b532-6fe940851317\") " pod="openshift-must-gather-tjmkf/perf-node-gather-daemonset-k4qqz" Jul 05 13:59:55.075479 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:55.075404 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2w6nq\" (UniqueName: \"kubernetes.io/projected/0750be83-08be-440b-b532-6fe940851317-kube-api-access-2w6nq\") pod \"perf-node-gather-daemonset-k4qqz\" (UID: \"0750be83-08be-440b-b532-6fe940851317\") " pod="openshift-must-gather-tjmkf/perf-node-gather-daemonset-k4qqz" Jul 05 13:59:55.075939 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:55.075908 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tjmkf/perf-node-gather-daemonset-k4qqz"] Jul 05 13:59:55.176509 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:55.176465 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0750be83-08be-440b-b532-6fe940851317-sys\") pod \"perf-node-gather-daemonset-k4qqz\" (UID: \"0750be83-08be-440b-b532-6fe940851317\") " pod="openshift-must-gather-tjmkf/perf-node-gather-daemonset-k4qqz" Jul 05 13:59:55.176768 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:55.176684 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0750be83-08be-440b-b532-6fe940851317-sys\") pod \"perf-node-gather-daemonset-k4qqz\" (UID: \"0750be83-08be-440b-b532-6fe940851317\") " pod="openshift-must-gather-tjmkf/perf-node-gather-daemonset-k4qqz" Jul 05 13:59:55.176913 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:55.176839 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0750be83-08be-440b-b532-6fe940851317-proc\") pod \"perf-node-gather-daemonset-k4qqz\" (UID: \"0750be83-08be-440b-b532-6fe940851317\") " pod="openshift-must-gather-tjmkf/perf-node-gather-daemonset-k4qqz" Jul 05 13:59:55.177054 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:55.176918 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0750be83-08be-440b-b532-6fe940851317-proc\") pod \"perf-node-gather-daemonset-k4qqz\" (UID: \"0750be83-08be-440b-b532-6fe940851317\") " pod="openshift-must-gather-tjmkf/perf-node-gather-daemonset-k4qqz" Jul 05 13:59:55.177206 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:55.177120 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0750be83-08be-440b-b532-6fe940851317-lib-modules\") pod \"perf-node-gather-daemonset-k4qqz\" (UID: \"0750be83-08be-440b-b532-6fe940851317\") " pod="openshift-must-gather-tjmkf/perf-node-gather-daemonset-k4qqz" Jul 05 13:59:55.177298 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:55.177233 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0750be83-08be-440b-b532-6fe940851317-lib-modules\") pod \"perf-node-gather-daemonset-k4qqz\" (UID: \"0750be83-08be-440b-b532-6fe940851317\") " pod="openshift-must-gather-tjmkf/perf-node-gather-daemonset-k4qqz" Jul 05 13:59:55.177432 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:55.177407 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0750be83-08be-440b-b532-6fe940851317-podres\") pod \"perf-node-gather-daemonset-k4qqz\" (UID: \"0750be83-08be-440b-b532-6fe940851317\") " pod="openshift-must-gather-tjmkf/perf-node-gather-daemonset-k4qqz" Jul 05 13:59:55.177518 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:55.177477 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2w6nq\" (UniqueName: \"kubernetes.io/projected/0750be83-08be-440b-b532-6fe940851317-kube-api-access-2w6nq\") pod \"perf-node-gather-daemonset-k4qqz\" (UID: \"0750be83-08be-440b-b532-6fe940851317\") " pod="openshift-must-gather-tjmkf/perf-node-gather-daemonset-k4qqz" Jul 05 13:59:55.177582 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:55.177565 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0750be83-08be-440b-b532-6fe940851317-podres\") pod \"perf-node-gather-daemonset-k4qqz\" (UID: \"0750be83-08be-440b-b532-6fe940851317\") " pod="openshift-must-gather-tjmkf/perf-node-gather-daemonset-k4qqz" Jul 05 13:59:55.186366 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:55.186347 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2w6nq\" (UniqueName: \"kubernetes.io/projected/0750be83-08be-440b-b532-6fe940851317-kube-api-access-2w6nq\") pod \"perf-node-gather-daemonset-k4qqz\" (UID: \"0750be83-08be-440b-b532-6fe940851317\") " pod="openshift-must-gather-tjmkf/perf-node-gather-daemonset-k4qqz" Jul 05 13:59:55.380443 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:55.380357 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tjmkf/perf-node-gather-daemonset-k4qqz" Jul 05 13:59:55.557767 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:55.557738 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tjmkf/perf-node-gather-daemonset-k4qqz"] Jul 05 13:59:55.559564 ip-10-0-139-155 kubenswrapper[2573]: W0705 13:59:55.559532 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod0750be83_08be_440b_b532_6fe940851317.slice/crio-76ad19e18ff1f4a7dac47bf8cc2d2a865ba4a2d2d193da82712c75acd681b4bf WatchSource:0}: Error finding container 76ad19e18ff1f4a7dac47bf8cc2d2a865ba4a2d2d193da82712c75acd681b4bf: Status 404 returned error can't find the container with id 76ad19e18ff1f4a7dac47bf8cc2d2a865ba4a2d2d193da82712c75acd681b4bf Jul 05 13:59:55.958509 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:55.958466 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tjmkf/perf-node-gather-daemonset-k4qqz" event={"ID":"0750be83-08be-440b-b532-6fe940851317","Type":"ContainerStarted","Data":"838972ed57235596c2a9b375b43e0477eaf04e5f65d285568c221efc43cb87d2"} Jul 05 13:59:55.958719 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:55.958540 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tjmkf/perf-node-gather-daemonset-k4qqz" event={"ID":"0750be83-08be-440b-b532-6fe940851317","Type":"ContainerStarted","Data":"76ad19e18ff1f4a7dac47bf8cc2d2a865ba4a2d2d193da82712c75acd681b4bf"} Jul 05 13:59:55.958967 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:55.958928 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-tjmkf/perf-node-gather-daemonset-k4qqz" Jul 05 13:59:55.976936 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:55.976868 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-tjmkf/perf-node-gather-daemonset-k4qqz" podStartSLOduration=0.976846725 podStartE2EDuration="976.846725ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:59:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:59:55.974705533 +0000 UTC m=+2468.870363623" watchObservedRunningTime="2026-07-05 13:59:55.976846725 +0000 UTC m=+2468.872504815" Jul 05 13:59:56.363591 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:56.363502 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-857f5b4496-mhrrz_56772071-55ed-4173-8976-cab465bd6353/console/0.log" Jul 05 13:59:56.401664 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:56.401610 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-khbr5_9f72532b-b2f9-42a8-90ae-f0d74cfe7a04/download-server/0.log" Jul 05 13:59:58.038396 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:58.038353 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gvrpq_d52daf5f-c799-45a9-b187-a9613bfe066c/dns/0.log" Jul 05 13:59:58.059390 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:58.059361 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gvrpq_d52daf5f-c799-45a9-b187-a9613bfe066c/kube-rbac-proxy/0.log" Jul 05 13:59:58.153964 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:58.153916 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-j5pwd_7d069d57-4ff7-4edd-94b6-10a12329f310/dns-node-resolver/0.log" Jul 05 13:59:58.737328 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:58.737293 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-db95784f7-lfv57_1be77740-54dc-45dc-a42f-afb9356c8023/registry/0.log" Jul 05 13:59:58.782805 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:58.782774 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-hlvjf_2a975403-6fed-4035-bf2d-c161d33baf8e/node-ca/0.log" Jul 05 13:59:59.900706 ip-10-0-139-155 kubenswrapper[2573]: I0705 13:59:59.900678 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-67cd4f9bbf-nw66v_2d3096e6-d86f-4547-864e-dc4633eeaf9f/kube-auth-proxy/0.log" Jul 05 14:00:00.846321 ip-10-0-139-155 kubenswrapper[2573]: I0705 14:00:00.846289 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gw7sw_b385ab14-87cd-4c01-81fe-b2e4e37528f0/serve-healthcheck-canary/0.log" Jul 05 14:00:01.609753 ip-10-0-139-155 kubenswrapper[2573]: I0705 14:00:01.609720 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-t6rnf_0b88064d-dcff-40b1-af6a-195f267576c6/kube-rbac-proxy/0.log" Jul 05 14:00:01.633454 ip-10-0-139-155 kubenswrapper[2573]: I0705 14:00:01.633420 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-t6rnf_0b88064d-dcff-40b1-af6a-195f267576c6/exporter/0.log" Jul 05 14:00:01.655904 ip-10-0-139-155 kubenswrapper[2573]: I0705 14:00:01.655875 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-t6rnf_0b88064d-dcff-40b1-af6a-195f267576c6/extractor/0.log" Jul 05 14:00:01.981755 ip-10-0-139-155 kubenswrapper[2573]: I0705 14:00:01.981723 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-tjmkf/perf-node-gather-daemonset-k4qqz" Jul 05 14:00:04.184941 ip-10-0-139-155 kubenswrapper[2573]: I0705 14:00:04.184916 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29720970-dmn4z_7551d605-e483-4238-81f3-b0fd5ee6a226/cleanup/0.log" Jul 05 14:00:04.713828 ip-10-0-139-155 kubenswrapper[2573]: I0705 14:00:04.713403 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b4f468d94-l5zdz_6c0a4bc3-5917-4fc7-88e6-38c057669a80/manager/0.log" Jul 05 14:00:04.732326 ip-10-0-139-155 kubenswrapper[2573]: I0705 14:00:04.732297 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b4f468d94-l5zdz_6c0a4bc3-5917-4fc7-88e6-38c057669a80/copy-manifests/0.log" Jul 05 14:00:06.116611 ip-10-0-139-155 kubenswrapper[2573]: I0705 14:00:06.116573 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6f6c4984d4-g7qxl_ef58a174-c066-45aa-876f-9178f8ca42b7/manager/0.log" Jul 05 14:00:06.168530 ip-10-0-139-155 kubenswrapper[2573]: I0705 14:00:06.168503 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-67cd4f9bbf-nw66v_2d3096e6-d86f-4547-864e-dc4633eeaf9f/kube-auth-proxy/0.log" Jul 05 14:00:07.567206 ip-10-0-139-155 kubenswrapper[2573]: I0705 14:00:07.567174 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-67cd4f9bbf-nw66v_2d3096e6-d86f-4547-864e-dc4633eeaf9f/kube-auth-proxy/0.log" Jul 05 14:00:11.169909 ip-10-0-139-155 kubenswrapper[2573]: I0705 14:00:11.169875 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-qv6qs_320513ee-d812-4b15-87d2-f0bc0a35eaa6/migrator/0.log" Jul 05 14:00:11.189596 ip-10-0-139-155 kubenswrapper[2573]: I0705 14:00:11.189565 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-qv6qs_320513ee-d812-4b15-87d2-f0bc0a35eaa6/graceful-termination/0.log" Jul 05 14:00:12.615819 ip-10-0-139-155 kubenswrapper[2573]: I0705 14:00:12.615782 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-4p2vl_8ec840e9-44f1-4daa-963a-96632c87bb8a/kube-multus/0.log"