Jul 06 01:26:44.447753 ip-10-0-139-44 systemd[1]: Starting Kubernetes Kubelet... Jul 06 01:26:44.784837 ip-10-0-139-44 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 01:26:44.784837 ip-10-0-139-44 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 06 01:26:44.784837 ip-10-0-139-44 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 01:26:44.784837 ip-10-0-139-44 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 06 01:26:44.784837 ip-10-0-139-44 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 01:26:44.785670 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.785582 2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 06 01:26:44.789758 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789741 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 01:26:44.789758 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789758 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 01:26:44.789830 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789761 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 01:26:44.789830 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789765 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 01:26:44.789830 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789768 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 01:26:44.789830 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789771 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 01:26:44.789830 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789774 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 01:26:44.789830 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789777 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 01:26:44.789830 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789780 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 01:26:44.789830 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789783 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 01:26:44.789830 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789786 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 01:26:44.789830 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789788 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 01:26:44.789830 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789791 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 01:26:44.789830 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789799 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 01:26:44.789830 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789802 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 01:26:44.789830 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789805 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 01:26:44.789830 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789807 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 01:26:44.789830 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789810 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 01:26:44.789830 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789812 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 01:26:44.789830 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789815 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 01:26:44.789830 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789817 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 01:26:44.789830 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789820 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 01:26:44.790298 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789822 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 01:26:44.790298 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789825 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 01:26:44.790298 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789827 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 01:26:44.790298 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789830 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 01:26:44.790298 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789833 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 01:26:44.790298 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789836 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 01:26:44.790298 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789839 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 01:26:44.790298 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789842 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 01:26:44.790298 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789845 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 01:26:44.790298 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789847 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 01:26:44.790298 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789850 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 01:26:44.790298 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789852 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 01:26:44.790298 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789854 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 01:26:44.790298 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789857 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 01:26:44.790298 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789859 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 01:26:44.790298 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789861 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 01:26:44.790298 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789864 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 01:26:44.790298 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789868 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 01:26:44.790298 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789872 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 01:26:44.790816 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789875 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 01:26:44.790816 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789878 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 01:26:44.790816 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789880 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 01:26:44.790816 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789883 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 01:26:44.790816 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789886 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 01:26:44.790816 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789888 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 01:26:44.790816 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789891 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 01:26:44.790816 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789893 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 01:26:44.790816 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789896 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 01:26:44.790816 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789898 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 01:26:44.790816 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789901 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 01:26:44.790816 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789903 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 01:26:44.790816 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789906 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 01:26:44.790816 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789908 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 01:26:44.790816 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789912 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 01:26:44.790816 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789915 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 01:26:44.790816 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789917 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 01:26:44.790816 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789919 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 01:26:44.790816 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789922 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 01:26:44.790816 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789924 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 01:26:44.791290 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789932 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 01:26:44.791290 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789934 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 01:26:44.791290 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789937 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 01:26:44.791290 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789939 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 01:26:44.791290 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789942 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 01:26:44.791290 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789944 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 01:26:44.791290 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789947 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 06 01:26:44.791290 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789949 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 01:26:44.791290 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789952 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 01:26:44.791290 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789954 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 01:26:44.791290 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789957 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 01:26:44.791290 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789960 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 01:26:44.791290 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789963 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 01:26:44.791290 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789965 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 01:26:44.791290 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789968 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 01:26:44.791290 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789971 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 01:26:44.791290 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789973 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 01:26:44.791290 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789976 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 01:26:44.791290 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789978 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 01:26:44.791290 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789980 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 01:26:44.791888 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789983 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 01:26:44.791888 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789985 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 01:26:44.791888 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789988 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 01:26:44.791888 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789990 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 01:26:44.791888 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789993 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 01:26:44.791888 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.789998 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 01:26:44.791888 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790002 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 01:26:44.791888 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790005 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 01:26:44.791888 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790008 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 01:26:44.791888 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790011 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 01:26:44.791888 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790014 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 01:26:44.791888 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790018 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 01:26:44.791888 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790021 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 01:26:44.791888 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790024 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 01:26:44.791888 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790027 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 01:26:44.791888 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790030 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 01:26:44.791888 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790032 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 01:26:44.791888 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790035 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 01:26:44.791888 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790760 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 01:26:44.792345 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790767 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 01:26:44.792345 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790770 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 01:26:44.792345 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790774 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 01:26:44.792345 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790777 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 01:26:44.792345 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790780 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 01:26:44.792345 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790782 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 01:26:44.792345 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790785 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 01:26:44.792345 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790787 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 01:26:44.792345 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790790 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 01:26:44.792345 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790793 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 01:26:44.792345 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790796 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 01:26:44.792345 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790798 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 01:26:44.792345 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790801 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 01:26:44.792345 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790803 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 06 01:26:44.792345 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790806 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 01:26:44.792345 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790808 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 01:26:44.792345 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790810 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 01:26:44.792345 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790813 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 01:26:44.792345 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790815 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 01:26:44.792345 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790818 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 01:26:44.792856 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790821 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 01:26:44.792856 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790824 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 01:26:44.792856 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790826 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 01:26:44.792856 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790829 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 01:26:44.792856 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790832 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 01:26:44.792856 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790834 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 01:26:44.792856 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790837 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 01:26:44.792856 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790839 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 01:26:44.792856 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790842 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 01:26:44.792856 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790844 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 01:26:44.792856 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790847 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 01:26:44.792856 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790849 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 01:26:44.792856 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790852 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 01:26:44.792856 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790854 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 01:26:44.792856 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790856 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 01:26:44.792856 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790859 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 01:26:44.792856 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790861 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 01:26:44.792856 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790864 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 01:26:44.792856 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790867 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 01:26:44.792856 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790869 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 01:26:44.793346 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790871 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 01:26:44.793346 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790874 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 01:26:44.793346 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790876 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 01:26:44.793346 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790879 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 01:26:44.793346 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790881 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 01:26:44.793346 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790883 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 01:26:44.793346 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790886 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 01:26:44.793346 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790888 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 01:26:44.793346 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790891 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 01:26:44.793346 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790895 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 01:26:44.793346 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790899 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 01:26:44.793346 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790904 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 01:26:44.793346 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790907 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 01:26:44.793346 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790910 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 01:26:44.793346 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790913 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 01:26:44.793346 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790917 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 01:26:44.793346 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790919 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 01:26:44.793346 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790922 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 01:26:44.793346 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790924 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 01:26:44.793845 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790927 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 01:26:44.793845 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790929 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 01:26:44.793845 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790932 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 01:26:44.793845 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790934 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 01:26:44.793845 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790936 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 01:26:44.793845 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790939 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 01:26:44.793845 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790941 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 01:26:44.793845 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790943 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 01:26:44.793845 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790946 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 01:26:44.793845 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790948 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 01:26:44.793845 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790951 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 01:26:44.793845 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790954 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 01:26:44.793845 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790957 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 01:26:44.793845 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790959 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 01:26:44.793845 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790962 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 01:26:44.793845 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790964 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 01:26:44.793845 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790967 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 01:26:44.793845 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790969 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 01:26:44.793845 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790972 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 01:26:44.793845 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790975 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 01:26:44.794326 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790978 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 01:26:44.794326 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790980 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 01:26:44.794326 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790983 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 01:26:44.794326 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790985 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 01:26:44.794326 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790988 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 01:26:44.794326 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790991 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 01:26:44.794326 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790993 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 01:26:44.794326 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790995 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 01:26:44.794326 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.790998 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 01:26:44.794326 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791000 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 01:26:44.794326 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791002 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 01:26:44.794326 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791005 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 01:26:44.794326 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791007 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 01:26:44.794326 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791010 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 01:26:44.794326 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791013 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 01:26:44.794326 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791017 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 01:26:44.794326 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791020 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 01:26:44.794326 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791023 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 01:26:44.794326 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791026 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 01:26:44.794326 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791395 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 01:26:44.794827 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791399 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 01:26:44.794827 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791403 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 01:26:44.794827 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791408 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 01:26:44.794827 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791414 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 01:26:44.794827 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791417 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 01:26:44.794827 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791420 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 01:26:44.794827 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791422 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 01:26:44.794827 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791425 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 01:26:44.794827 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791427 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 01:26:44.794827 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791430 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 01:26:44.794827 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791449 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 01:26:44.794827 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791452 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 01:26:44.794827 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791455 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 01:26:44.794827 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791457 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 01:26:44.794827 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791460 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 01:26:44.794827 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791463 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 01:26:44.794827 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791466 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 01:26:44.794827 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791468 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 01:26:44.794827 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791471 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 01:26:44.795282 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791475 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 01:26:44.795282 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791478 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 01:26:44.795282 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791481 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 01:26:44.795282 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791484 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 01:26:44.795282 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791488 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 01:26:44.795282 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791491 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 01:26:44.795282 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791494 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 01:26:44.795282 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791496 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 01:26:44.795282 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791500 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 01:26:44.795282 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791504 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 01:26:44.795282 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791506 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 01:26:44.795282 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791509 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 06 01:26:44.795282 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791513 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 01:26:44.795282 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791515 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 01:26:44.795282 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791518 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 01:26:44.795282 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791520 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 01:26:44.795282 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791524 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 01:26:44.795282 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791527 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 01:26:44.795282 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791530 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 01:26:44.795798 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791533 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 01:26:44.795798 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791535 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 01:26:44.795798 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791538 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 01:26:44.795798 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791541 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 01:26:44.795798 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791543 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 01:26:44.795798 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791546 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 01:26:44.795798 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791548 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 01:26:44.795798 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791551 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 01:26:44.795798 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791553 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 01:26:44.795798 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791556 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 01:26:44.795798 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791559 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 01:26:44.795798 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791562 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 01:26:44.795798 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791564 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 01:26:44.795798 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791566 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 01:26:44.795798 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791569 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 01:26:44.795798 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791571 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 01:26:44.795798 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791574 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 01:26:44.795798 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791576 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 01:26:44.795798 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791579 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 01:26:44.795798 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791581 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 01:26:44.796284 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791583 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 01:26:44.796284 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791586 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 01:26:44.796284 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791588 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 01:26:44.796284 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791590 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 01:26:44.796284 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791593 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 01:26:44.796284 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791595 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 01:26:44.796284 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791598 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 01:26:44.796284 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791600 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 01:26:44.796284 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791603 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 01:26:44.796284 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791607 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 01:26:44.796284 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791610 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 01:26:44.796284 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791619 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 01:26:44.796284 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791621 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 01:26:44.796284 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791624 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 01:26:44.796284 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791626 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 01:26:44.796284 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791629 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 01:26:44.796284 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791631 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 01:26:44.796284 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791634 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 01:26:44.796284 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791636 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 01:26:44.796284 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791639 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 01:26:44.796773 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791641 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 01:26:44.796773 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791644 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 01:26:44.796773 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791646 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 01:26:44.796773 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791649 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 01:26:44.796773 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791651 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 01:26:44.796773 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791654 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 01:26:44.796773 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791656 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 01:26:44.796773 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791658 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 01:26:44.796773 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791661 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 01:26:44.796773 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791664 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 01:26:44.796773 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791666 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 01:26:44.796773 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791668 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 01:26:44.796773 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791671 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 01:26:44.796773 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791673 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 01:26:44.796773 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791676 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 01:26:44.796773 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791678 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 01:26:44.796773 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791681 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 01:26:44.796773 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791683 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 01:26:44.796773 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791685 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 01:26:44.797252 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.791688 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 01:26:44.797252 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791772 2580 flags.go:64] FLAG: --address="0.0.0.0" Jul 06 01:26:44.797252 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791780 2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 06 01:26:44.797252 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791786 2580 flags.go:64] FLAG: --anonymous-auth="true" Jul 06 01:26:44.797252 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791790 2580 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 06 01:26:44.797252 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791795 2580 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 06 01:26:44.797252 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791798 2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 06 01:26:44.797252 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791803 2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 06 01:26:44.797252 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791807 2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 06 01:26:44.797252 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791810 2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 06 01:26:44.797252 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791812 2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 06 01:26:44.797252 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791816 2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 06 01:26:44.797252 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791819 2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 06 01:26:44.797252 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791821 2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 06 01:26:44.797252 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791825 2580 flags.go:64] FLAG: --cgroup-root="" Jul 06 01:26:44.797252 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791827 2580 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 06 01:26:44.797252 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791830 2580 flags.go:64] FLAG: --client-ca-file="" Jul 06 01:26:44.797252 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791833 2580 flags.go:64] FLAG: --cloud-provider="external" Jul 06 01:26:44.797252 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791836 2580 flags.go:64] FLAG: --cluster-dns="[]" Jul 06 01:26:44.797252 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791840 2580 flags.go:64] FLAG: --cluster-domain="" Jul 06 01:26:44.797252 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791843 2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 06 01:26:44.797252 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791846 2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 06 01:26:44.797252 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791849 2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 06 01:26:44.797252 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791852 2580 flags.go:64] FLAG: --container-log-max-files="5" Jul 06 01:26:44.797885 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791855 2580 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 06 01:26:44.797885 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791858 2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 06 01:26:44.797885 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791861 2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 06 01:26:44.797885 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791864 2580 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 06 01:26:44.797885 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791867 2580 flags.go:64] FLAG: --contention-profiling="false" Jul 06 01:26:44.797885 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791870 2580 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 06 01:26:44.797885 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791873 2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 06 01:26:44.797885 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791876 2580 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 06 01:26:44.797885 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791879 2580 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 06 01:26:44.797885 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791882 2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 06 01:26:44.797885 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791887 2580 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 06 01:26:44.797885 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791890 2580 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 06 01:26:44.797885 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791893 2580 flags.go:64] FLAG: --enable-load-reader="false" Jul 06 01:26:44.797885 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791896 2580 flags.go:64] FLAG: --enable-server="true" Jul 06 01:26:44.797885 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791900 2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 06 01:26:44.797885 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791904 2580 flags.go:64] FLAG: --event-burst="100" Jul 06 01:26:44.797885 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791907 2580 flags.go:64] FLAG: --event-qps="50" Jul 06 01:26:44.797885 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791910 2580 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 06 01:26:44.797885 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791913 2580 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 06 01:26:44.797885 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791915 2580 flags.go:64] FLAG: --eviction-hard="" Jul 06 01:26:44.797885 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791919 2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 06 01:26:44.797885 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791922 2580 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 06 01:26:44.797885 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791925 2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 06 01:26:44.797885 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791928 2580 flags.go:64] FLAG: --eviction-soft="" Jul 06 01:26:44.797885 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791930 2580 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 06 01:26:44.798511 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791933 2580 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 06 01:26:44.798511 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791936 2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 06 01:26:44.798511 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791939 2580 flags.go:64] FLAG: --experimental-mounter-path="" Jul 06 01:26:44.798511 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791942 2580 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 06 01:26:44.798511 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791944 2580 flags.go:64] FLAG: --fail-swap-on="true" Jul 06 01:26:44.798511 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791947 2580 flags.go:64] FLAG: --feature-gates="" Jul 06 01:26:44.798511 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791951 2580 flags.go:64] FLAG: --file-check-frequency="20s" Jul 06 01:26:44.798511 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791954 2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 06 01:26:44.798511 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791957 2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 06 01:26:44.798511 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791960 2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 06 01:26:44.798511 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791963 2580 flags.go:64] FLAG: --healthz-port="10248" Jul 06 01:26:44.798511 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791965 2580 flags.go:64] FLAG: --help="false" Jul 06 01:26:44.798511 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791968 2580 flags.go:64] FLAG: --hostname-override="ip-10-0-139-44.ec2.internal" Jul 06 01:26:44.798511 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791971 2580 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 06 01:26:44.798511 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791976 2580 flags.go:64] FLAG: --http-check-frequency="20s" Jul 06 01:26:44.798511 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791978 2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 06 01:26:44.798511 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791982 2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 06 01:26:44.798511 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791987 2580 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 06 01:26:44.798511 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791990 2580 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 06 01:26:44.798511 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791992 2580 flags.go:64] FLAG: --image-service-endpoint="" Jul 06 01:26:44.798511 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791995 2580 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 06 01:26:44.798511 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.791998 2580 flags.go:64] FLAG: --kube-api-burst="100" Jul 06 01:26:44.798511 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792001 2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 06 01:26:44.799092 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792004 2580 flags.go:64] FLAG: --kube-api-qps="50" Jul 06 01:26:44.799092 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792007 2580 flags.go:64] FLAG: --kube-reserved="" Jul 06 01:26:44.799092 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792010 2580 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 06 01:26:44.799092 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792012 2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 06 01:26:44.799092 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792015 2580 flags.go:64] FLAG: --kubelet-cgroups="" Jul 06 01:26:44.799092 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792018 2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 06 01:26:44.799092 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792020 2580 flags.go:64] FLAG: --lock-file="" Jul 06 01:26:44.799092 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792023 2580 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 06 01:26:44.799092 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792026 2580 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 06 01:26:44.799092 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792029 2580 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 06 01:26:44.799092 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792034 2580 flags.go:64] FLAG: --log-json-split-stream="false" Jul 06 01:26:44.799092 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792036 2580 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 06 01:26:44.799092 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792039 2580 flags.go:64] FLAG: --log-text-split-stream="false" Jul 06 01:26:44.799092 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792042 2580 flags.go:64] FLAG: --logging-format="text" Jul 06 01:26:44.799092 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792044 2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 06 01:26:44.799092 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792047 2580 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 06 01:26:44.799092 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792050 2580 flags.go:64] FLAG: --manifest-url="" Jul 06 01:26:44.799092 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792052 2580 flags.go:64] FLAG: --manifest-url-header="" Jul 06 01:26:44.799092 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792057 2580 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 06 01:26:44.799092 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792059 2580 flags.go:64] FLAG: --max-open-files="1000000" Jul 06 01:26:44.799092 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792063 2580 flags.go:64] FLAG: --max-pods="110" Jul 06 01:26:44.799092 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792066 2580 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 06 01:26:44.799092 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792068 2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 06 01:26:44.799092 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792072 2580 flags.go:64] FLAG: --memory-manager-policy="None" Jul 06 01:26:44.799092 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792075 2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 06 01:26:44.799819 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792078 2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 06 01:26:44.799819 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792082 2580 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 06 01:26:44.799819 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792085 2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 06 01:26:44.799819 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792091 2580 flags.go:64] FLAG: --node-status-max-images="50" Jul 06 01:26:44.799819 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792094 2580 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 06 01:26:44.799819 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792097 2580 flags.go:64] FLAG: --oom-score-adj="-999" Jul 06 01:26:44.799819 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792100 2580 flags.go:64] FLAG: --pod-cidr="" Jul 06 01:26:44.799819 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792103 2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 06 01:26:44.799819 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792108 2580 flags.go:64] FLAG: --pod-manifest-path="" Jul 06 01:26:44.799819 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792111 2580 flags.go:64] FLAG: --pod-max-pids="-1" Jul 06 01:26:44.799819 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792114 2580 flags.go:64] FLAG: --pods-per-core="0" Jul 06 01:26:44.799819 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792117 2580 flags.go:64] FLAG: --port="10250" Jul 06 01:26:44.799819 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792120 2580 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 06 01:26:44.799819 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792123 2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f91d7096f7a7fca1" Jul 06 01:26:44.799819 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792126 2580 flags.go:64] FLAG: --qos-reserved="" Jul 06 01:26:44.799819 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792129 2580 flags.go:64] FLAG: --read-only-port="10255" Jul 06 01:26:44.799819 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792132 2580 flags.go:64] FLAG: --register-node="true" Jul 06 01:26:44.799819 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792135 2580 flags.go:64] FLAG: --register-with-taints="" Jul 06 01:26:44.799819 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792138 2580 flags.go:64] FLAG: --registry-burst="10" Jul 06 01:26:44.799819 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792141 2580 flags.go:64] FLAG: --registry-qps="5" Jul 06 01:26:44.799819 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792144 2580 flags.go:64] FLAG: --reserved-cpus="" Jul 06 01:26:44.799819 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792146 2580 flags.go:64] FLAG: --reserved-memory="" Jul 06 01:26:44.799819 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792150 2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 06 01:26:44.799819 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792153 2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 06 01:26:44.799819 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792156 2580 flags.go:64] FLAG: --rotate-certificates="false" Jul 06 01:26:44.800416 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792159 2580 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 06 01:26:44.800416 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792161 2580 flags.go:64] FLAG: --runonce="false" Jul 06 01:26:44.800416 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792164 2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 06 01:26:44.800416 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792167 2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 06 01:26:44.800416 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792170 2580 flags.go:64] FLAG: --seccomp-default="false" Jul 06 01:26:44.800416 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792177 2580 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 06 01:26:44.800416 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792180 2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 06 01:26:44.800416 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792183 2580 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 06 01:26:44.800416 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792187 2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 06 01:26:44.800416 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792191 2580 flags.go:64] FLAG: --storage-driver-password="root" Jul 06 01:26:44.800416 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792193 2580 flags.go:64] FLAG: --storage-driver-secure="false" Jul 06 01:26:44.800416 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792196 2580 flags.go:64] FLAG: --storage-driver-table="stats" Jul 06 01:26:44.800416 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792199 2580 flags.go:64] FLAG: --storage-driver-user="root" Jul 06 01:26:44.800416 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792202 2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 06 01:26:44.800416 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792205 2580 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 06 01:26:44.800416 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792208 2580 flags.go:64] FLAG: --system-cgroups="" Jul 06 01:26:44.800416 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792211 2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 06 01:26:44.800416 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792216 2580 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 06 01:26:44.800416 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792218 2580 flags.go:64] FLAG: --tls-cert-file="" Jul 06 01:26:44.800416 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792221 2580 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 06 01:26:44.800416 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792225 2580 flags.go:64] FLAG: --tls-min-version="" Jul 06 01:26:44.800416 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792228 2580 flags.go:64] FLAG: --tls-private-key-file="" Jul 06 01:26:44.800416 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792231 2580 flags.go:64] FLAG: --topology-manager-policy="none" Jul 06 01:26:44.800416 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792234 2580 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 06 01:26:44.800416 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792237 2580 flags.go:64] FLAG: --topology-manager-scope="container" Jul 06 01:26:44.801024 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792239 2580 flags.go:64] FLAG: --v="2" Jul 06 01:26:44.801024 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792243 2580 flags.go:64] FLAG: --version="false" Jul 06 01:26:44.801024 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792247 2580 flags.go:64] FLAG: --vmodule="" Jul 06 01:26:44.801024 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792250 2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 06 01:26:44.801024 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792253 2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 06 01:26:44.801024 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792386 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 01:26:44.801024 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792389 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 01:26:44.801024 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792393 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 01:26:44.801024 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792396 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 01:26:44.801024 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792398 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 01:26:44.801024 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792401 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 01:26:44.801024 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792403 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 01:26:44.801024 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792408 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 01:26:44.801024 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792410 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 01:26:44.801024 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792413 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 01:26:44.801024 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792415 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 01:26:44.801024 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792418 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 01:26:44.801024 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792420 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 01:26:44.801024 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792423 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 01:26:44.801024 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792426 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 01:26:44.801024 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792428 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 01:26:44.801564 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792445 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 01:26:44.801564 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792448 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 06 01:26:44.801564 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792451 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 01:26:44.801564 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792454 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 01:26:44.801564 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792456 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 01:26:44.801564 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792458 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 01:26:44.801564 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792461 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 01:26:44.801564 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792463 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 01:26:44.801564 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792466 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 01:26:44.801564 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792468 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 01:26:44.801564 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792471 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 01:26:44.801564 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792473 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 01:26:44.801564 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792476 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 01:26:44.801564 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792478 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 01:26:44.801564 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792481 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 01:26:44.801564 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792483 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 01:26:44.801564 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792485 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 01:26:44.801564 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792488 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 01:26:44.801564 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792490 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 01:26:44.801564 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792493 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 01:26:44.802043 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792495 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 01:26:44.802043 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792497 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 01:26:44.802043 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792500 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 01:26:44.802043 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792504 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 01:26:44.802043 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792506 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 01:26:44.802043 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792509 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 01:26:44.802043 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792511 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 01:26:44.802043 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792514 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 01:26:44.802043 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792517 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 01:26:44.802043 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792519 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 01:26:44.802043 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792522 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 01:26:44.802043 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792524 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 01:26:44.802043 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792527 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 01:26:44.802043 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792529 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 01:26:44.802043 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792532 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 01:26:44.802043 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792537 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 01:26:44.802043 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792540 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 01:26:44.802043 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792542 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 01:26:44.802043 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792545 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 01:26:44.802043 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792548 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 01:26:44.802725 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792550 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 01:26:44.802725 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792552 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 01:26:44.802725 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792555 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 01:26:44.802725 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792557 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 01:26:44.802725 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792559 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 01:26:44.802725 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792564 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 01:26:44.802725 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792567 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 01:26:44.802725 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792570 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 01:26:44.802725 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792573 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 01:26:44.802725 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792575 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 01:26:44.802725 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792578 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 01:26:44.802725 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792580 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 01:26:44.802725 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792582 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 01:26:44.802725 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792584 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 01:26:44.802725 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792587 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 01:26:44.802725 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792591 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 01:26:44.802725 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792594 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 01:26:44.802725 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792596 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 01:26:44.802725 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792599 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 01:26:44.803463 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792601 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 01:26:44.803463 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792604 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 01:26:44.803463 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792606 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 01:26:44.803463 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792609 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 01:26:44.803463 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792611 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 01:26:44.803463 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792614 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 01:26:44.803463 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792616 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 01:26:44.803463 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792618 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 01:26:44.803463 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792621 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 01:26:44.803463 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792624 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 01:26:44.803463 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792626 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 01:26:44.803463 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792629 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 01:26:44.803463 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792631 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 01:26:44.803463 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792633 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 01:26:44.803463 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792636 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 01:26:44.803463 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792638 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 01:26:44.803463 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792642 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 01:26:44.803463 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792645 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 01:26:44.803463 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792647 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 01:26:44.804068 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792650 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 01:26:44.804068 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792652 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 01:26:44.804068 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792655 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 01:26:44.804068 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792659 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 01:26:44.804068 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.792662 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 01:26:44.804068 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.792669 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 01:26:44.804068 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.799895 2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 06 01:26:44.804068 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.799912 2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 06 01:26:44.804068 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.799928 2580 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 06 01:26:44.804068 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.799933 2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 06 01:26:44.804068 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800004 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 01:26:44.804068 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800009 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 01:26:44.804068 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800012 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 01:26:44.804068 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800015 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 01:26:44.804423 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800018 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 01:26:44.804423 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800021 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 01:26:44.804423 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800024 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 01:26:44.804423 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800026 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 01:26:44.804423 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800029 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 01:26:44.804423 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800031 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 01:26:44.804423 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800034 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 01:26:44.804423 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800036 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 01:26:44.804423 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800038 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 01:26:44.804423 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800041 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 01:26:44.804423 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800043 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 01:26:44.804423 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800046 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 01:26:44.804423 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800049 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 01:26:44.804423 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800051 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 01:26:44.804423 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800053 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 01:26:44.804423 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800056 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 01:26:44.804423 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800058 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 01:26:44.804423 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800061 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 01:26:44.804423 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800065 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 01:26:44.804964 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800068 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 01:26:44.804964 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800072 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 01:26:44.804964 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800075 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 01:26:44.804964 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800078 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 01:26:44.804964 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800080 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 01:26:44.804964 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800083 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 01:26:44.804964 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800092 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 01:26:44.804964 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800094 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 01:26:44.804964 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800097 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 01:26:44.804964 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800099 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 01:26:44.804964 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800102 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 01:26:44.804964 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800105 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 01:26:44.804964 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800108 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 01:26:44.804964 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800110 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 01:26:44.804964 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800113 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 01:26:44.804964 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800115 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 01:26:44.804964 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800118 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 01:26:44.804964 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800120 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 01:26:44.804964 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800123 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 01:26:44.804964 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800125 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 01:26:44.805501 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800128 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 01:26:44.805501 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800130 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 01:26:44.805501 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800133 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 01:26:44.805501 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800135 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 01:26:44.805501 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800138 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 01:26:44.805501 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800140 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 01:26:44.805501 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800143 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 01:26:44.805501 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800145 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 01:26:44.805501 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800148 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 01:26:44.805501 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800150 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 01:26:44.805501 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800152 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 01:26:44.805501 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800155 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 01:26:44.805501 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800157 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 01:26:44.805501 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800160 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 01:26:44.805501 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800162 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 01:26:44.805501 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800165 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 01:26:44.805501 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800167 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 01:26:44.805501 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800169 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 01:26:44.805501 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800172 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 01:26:44.805501 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800180 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 01:26:44.806002 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800182 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 01:26:44.806002 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800185 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 01:26:44.806002 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800187 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 01:26:44.806002 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800190 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 01:26:44.806002 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800193 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 01:26:44.806002 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800195 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 01:26:44.806002 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800197 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 01:26:44.806002 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800200 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 01:26:44.806002 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800202 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 01:26:44.806002 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800205 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 01:26:44.806002 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800207 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 01:26:44.806002 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800210 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 01:26:44.806002 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800212 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 01:26:44.806002 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800214 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 01:26:44.806002 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800217 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 01:26:44.806002 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800219 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 01:26:44.806002 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800222 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 01:26:44.806002 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800224 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 01:26:44.806002 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800226 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 01:26:44.806002 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800229 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 01:26:44.806574 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800231 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 01:26:44.806574 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800233 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 01:26:44.806574 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800236 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 01:26:44.806574 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800238 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 01:26:44.806574 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800240 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 01:26:44.806574 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800243 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 01:26:44.806574 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800247 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 01:26:44.806574 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800249 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 01:26:44.806574 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800252 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 06 01:26:44.806574 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800255 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 01:26:44.806574 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800258 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 01:26:44.806574 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800261 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 01:26:44.806574 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800269 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 01:26:44.806574 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800272 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 01:26:44.806574 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800274 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 01:26:44.806574 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800277 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 01:26:44.806955 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.800282 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 01:26:44.806955 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800459 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 01:26:44.806955 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800465 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 01:26:44.806955 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800468 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 01:26:44.806955 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800471 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 01:26:44.806955 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800474 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 01:26:44.806955 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800477 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 01:26:44.806955 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800479 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 01:26:44.806955 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800483 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 01:26:44.806955 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800487 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 01:26:44.806955 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800489 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 01:26:44.806955 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800492 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 06 01:26:44.806955 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800494 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 01:26:44.806955 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800497 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 01:26:44.806955 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800499 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 01:26:44.807324 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800502 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 01:26:44.807324 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800504 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 01:26:44.807324 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800507 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 01:26:44.807324 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800509 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 01:26:44.807324 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800512 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 01:26:44.807324 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800514 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 01:26:44.807324 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800517 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 01:26:44.807324 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800519 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 01:26:44.807324 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800522 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 01:26:44.807324 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800524 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 01:26:44.807324 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800527 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 01:26:44.807324 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800529 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 01:26:44.807324 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800532 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 01:26:44.807324 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800534 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 01:26:44.807324 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800544 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 01:26:44.807324 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800547 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 01:26:44.807324 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800549 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 01:26:44.807324 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800552 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 01:26:44.807324 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800555 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 01:26:44.807324 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800557 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 01:26:44.807828 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800560 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 01:26:44.807828 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800562 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 01:26:44.807828 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800566 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 01:26:44.807828 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800569 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 01:26:44.807828 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800572 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 01:26:44.807828 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800575 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 01:26:44.807828 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800578 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 01:26:44.807828 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800581 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 01:26:44.807828 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800583 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 01:26:44.807828 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800585 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 01:26:44.807828 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800588 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 01:26:44.807828 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800591 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 01:26:44.807828 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800593 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 01:26:44.807828 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800596 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 01:26:44.807828 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800598 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 01:26:44.807828 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800600 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 01:26:44.807828 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800603 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 01:26:44.807828 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800605 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 01:26:44.807828 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800607 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 01:26:44.808295 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800610 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 01:26:44.808295 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800612 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 01:26:44.808295 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800615 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 01:26:44.808295 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800617 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 01:26:44.808295 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800620 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 01:26:44.808295 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800622 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 01:26:44.808295 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800624 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 01:26:44.808295 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800627 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 01:26:44.808295 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800634 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 01:26:44.808295 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800637 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 01:26:44.808295 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800639 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 01:26:44.808295 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800643 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 01:26:44.808295 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800646 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 01:26:44.808295 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800649 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 01:26:44.808295 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800652 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 01:26:44.808295 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800655 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 01:26:44.808295 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800657 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 01:26:44.808295 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800660 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 01:26:44.808295 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800662 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 01:26:44.808295 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800665 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 01:26:44.808792 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800667 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 01:26:44.808792 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800669 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 01:26:44.808792 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800672 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 01:26:44.808792 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800674 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 01:26:44.808792 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800677 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 01:26:44.808792 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800679 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 01:26:44.808792 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800681 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 01:26:44.808792 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800684 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 01:26:44.808792 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800686 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 01:26:44.808792 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800688 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 01:26:44.808792 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800691 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 01:26:44.808792 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800693 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 01:26:44.808792 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800695 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 01:26:44.808792 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800698 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 01:26:44.808792 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800700 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 01:26:44.808792 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800703 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 01:26:44.808792 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800705 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 01:26:44.808792 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800707 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 01:26:44.808792 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800709 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 01:26:44.808792 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800712 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 01:26:44.809323 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800714 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 01:26:44.809323 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800722 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 01:26:44.809323 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800724 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 01:26:44.809323 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800727 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 01:26:44.809323 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800729 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 01:26:44.809323 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:44.800732 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 01:26:44.809323 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.800737 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 01:26:44.809323 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.800858 2580 server.go:962] "Client rotation is on, will bootstrap in background" Jul 06 01:26:44.809323 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.804428 2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 06 01:26:44.809323 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.805657 2580 server.go:1019] "Starting client certificate rotation" Jul 06 01:26:44.809323 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.805757 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 01:26:44.809323 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.806387 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 01:26:44.809323 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.807879 2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 01:26:44.810962 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.810947 2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 01:26:44.817814 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.817798 2580 log.go:25] "Validated CRI v1 runtime API" Jul 06 01:26:44.821215 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.821202 2580 log.go:25] "Validated CRI v1 image API" Jul 06 01:26:44.822053 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.822034 2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 06 01:26:44.824906 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.824888 2580 fs.go:135] Filesystem UUIDs: map[00071291-f44e-42a5-880b-923f7b41f4c7:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 d8b7d142-6d23-48e1-8b87-05f5f27dfca4:/dev/nvme0n1p3] Jul 06 01:26:44.824960 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.824906 2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 06 01:26:44.830517 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.830376 2580 manager.go:216] Machine: {Timestamp:2026-07-06 01:26:44.829151565 +0000 UTC m=+0.279404660 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3104048 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec287c80dec692648b7817450b40be9b SystemUUID:ec287c80-dec6-9264-8b78-17450b40be9b BootID:403d1feb-81fb-47ac-afb7-b804e5d408ca Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:aa:bd:e9:53:41 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:aa:bd:e9:53:41 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:3a:74:6d:14:80:85 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 06 01:26:44.830517 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.830511 2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 06 01:26:44.830633 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.830621 2580 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 06 01:26:44.833649 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.833619 2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 06 01:26:44.833807 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.833652 2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-44.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 06 01:26:44.834808 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.834797 2580 topology_manager.go:138] "Creating topology manager with none policy" Jul 06 01:26:44.834838 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.834811 2580 container_manager_linux.go:309] "Creating device plugin manager" Jul 06 01:26:44.834838 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.834818 2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 01:26:44.835304 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.835289 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 01:26:44.835374 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.835361 2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 01:26:44.836813 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.836802 2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 06 01:26:44.840549 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.840538 2580 state_mem.go:36] "Initialized new in-memory state store" Jul 06 01:26:44.840682 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.840673 2580 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 06 01:26:44.840711 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.840702 2580 kubelet.go:486] "Attempting to sync node with API server" Jul 06 01:26:44.840748 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.840716 2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 06 01:26:44.840748 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.840728 2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 06 01:26:44.840748 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.840737 2580 kubelet.go:398] "Adding apiserver pod source" Jul 06 01:26:44.840748 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.840744 2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 06 01:26:44.842029 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.842007 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 01:26:44.842121 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.842056 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 01:26:44.845055 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.845027 2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 06 01:26:44.846760 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.846745 2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 06 01:26:44.846818 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.846776 2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 06 01:26:44.847797 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.847785 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 06 01:26:44.847839 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.847803 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 06 01:26:44.847839 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.847810 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 06 01:26:44.847839 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.847815 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 06 01:26:44.847839 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.847824 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 06 01:26:44.847839 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.847830 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 06 01:26:44.847839 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.847837 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 06 01:26:44.847997 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.847843 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 06 01:26:44.847997 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.847850 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 06 01:26:44.847997 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.847855 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 06 01:26:44.847997 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.847863 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 06 01:26:44.847997 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.847869 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 06 01:26:44.848495 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.848485 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 06 01:26:44.848528 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.848497 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 06 01:26:44.850863 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.850842 2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-44.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 06 01:26:44.850934 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:44.850870 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-44.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 06 01:26:44.850934 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:44.850871 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 06 01:26:44.851885 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.851872 2580 server.go:1268] "Started kubelet" Jul 06 01:26:44.851978 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.851945 2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 06 01:26:44.852035 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.851985 2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 06 01:26:44.852096 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.852084 2580 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 06 01:26:44.852748 ip-10-0-139-44 systemd[1]: Started Kubernetes Kubelet. Jul 06 01:26:44.853452 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.853422 2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 06 01:26:44.854127 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.854102 2580 server.go:310] "Adding debug handlers to kubelet server" Jul 06 01:26:44.855268 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.855248 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4877s" Jul 06 01:26:44.858456 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:44.858411 2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 06 01:26:44.859355 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.859337 2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 06 01:26:44.859459 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.859343 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 06 01:26:44.859823 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:44.859569 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-44.ec2.internal\" not found" Jul 06 01:26:44.860091 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.859972 2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 06 01:26:44.860091 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.859973 2580 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 06 01:26:44.860091 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.859998 2580 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 06 01:26:44.860245 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.860136 2580 reconstruct.go:97] "Volume reconstruction finished" Jul 06 01:26:44.860245 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.860146 2580 reconciler.go:29] "Reconciler: start to sync state" Jul 06 01:26:44.860245 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.860146 2580 factory.go:55] Registering systemd factory Jul 06 01:26:44.860245 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.860188 2580 factory.go:223] Registration of the systemd container factory successfully Jul 06 01:26:44.860679 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.860659 2580 factory.go:153] Registering CRI-O factory Jul 06 01:26:44.860679 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.860680 2580 factory.go:223] Registration of the crio container factory successfully Jul 06 01:26:44.860820 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.860745 2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 06 01:26:44.860820 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.860767 2580 factory.go:103] Registering Raw factory Jul 06 01:26:44.860820 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.860778 2580 manager.go:1196] Started watching for new ooms in manager Jul 06 01:26:44.861107 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.861074 2580 manager.go:349] Starting recovery of all containers Jul 06 01:26:44.862415 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.862397 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4877s" Jul 06 01:26:44.865010 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:44.864979 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 06 01:26:44.865450 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:44.865382 2580 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-139-44.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 06 01:26:44.866044 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:44.865028 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-44.ec2.internal.18bf8eff5ab85a0d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-44.ec2.internal,UID:ip-10-0-139-44.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-44.ec2.internal,},FirstTimestamp:2026-07-06 01:26:44.851841549 +0000 UTC m=+0.302094710,LastTimestamp:2026-07-06 01:26:44.851841549 +0000 UTC m=+0.302094710,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-44.ec2.internal,}" Jul 06 01:26:44.872718 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.872699 2580 manager.go:354] Recovery completed Jul 06 01:26:44.877464 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.877449 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 01:26:44.879227 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.879208 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-44.ec2.internal" event="NodeHasSufficientMemory" Jul 06 01:26:44.879320 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.879244 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-44.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 01:26:44.879320 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.879257 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-44.ec2.internal" event="NodeHasSufficientPID" Jul 06 01:26:44.879848 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.879833 2580 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 06 01:26:44.879848 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.879848 2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 06 01:26:44.879968 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.879865 2580 state_mem.go:36] "Initialized new in-memory state store" Jul 06 01:26:44.879968 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.879841 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 01:26:44.880520 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:44.880459 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-44.ec2.internal.18bf8eff5c5a338f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-44.ec2.internal,UID:ip-10-0-139-44.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-139-44.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-139-44.ec2.internal,},FirstTimestamp:2026-07-06 01:26:44.879225743 +0000 UTC m=+0.329478838,LastTimestamp:2026-07-06 01:26:44.879225743 +0000 UTC m=+0.329478838,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-44.ec2.internal,}" Jul 06 01:26:44.880704 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.880687 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-44.ec2.internal" event="NodeHasSufficientMemory" Jul 06 01:26:44.880754 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.880720 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-44.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 01:26:44.880754 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.880729 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-44.ec2.internal" event="NodeHasSufficientPID" Jul 06 01:26:44.881949 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.881937 2580 policy_none.go:49] "None policy: Start" Jul 06 01:26:44.881985 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.881955 2580 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 06 01:26:44.881985 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.881965 2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 06 01:26:44.883921 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.883910 2580 policy_none.go:47] "Start" Jul 06 01:26:44.923083 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.923068 2580 manager.go:337] "Starting Device Plugin manager" Jul 06 01:26:44.923153 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:44.923102 2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 06 01:26:44.923153 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.923113 2580 server.go:87] "Starting device plugin registration server" Jul 06 01:26:44.923390 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.923378 2580 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 06 01:26:44.923505 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.923390 2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 06 01:26:44.923555 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.923515 2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 06 01:26:44.923623 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.923605 2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 06 01:26:44.923623 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.923623 2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 06 01:26:44.924243 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:44.924219 2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 06 01:26:44.924343 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:44.924279 2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-44.ec2.internal\" not found" Jul 06 01:26:44.972845 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.972801 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 06 01:26:44.978727 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.974179 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 06 01:26:44.978727 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.974216 2580 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 06 01:26:44.978727 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.974243 2580 kubelet.go:2448] "Starting kubelet main sync loop" Jul 06 01:26:44.978727 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:44.974277 2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 06 01:26:44.978727 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:44.977302 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 01:26:45.024222 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:45.024191 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 01:26:45.025382 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:45.025360 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-44.ec2.internal" event="NodeHasSufficientMemory" Jul 06 01:26:45.025481 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:45.025397 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-44.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 01:26:45.025481 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:45.025408 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-44.ec2.internal" event="NodeHasSufficientPID" Jul 06 01:26:45.025481 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:45.025447 2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-44.ec2.internal" Jul 06 01:26:45.033115 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:45.033088 2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-44.ec2.internal" Jul 06 01:26:45.033163 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:45.033125 2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-44.ec2.internal\": node \"ip-10-0-139-44.ec2.internal\" not found" Jul 06 01:26:45.047852 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:45.047790 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-44.ec2.internal\" not found" Jul 06 01:26:45.075135 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:45.075094 2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-44.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-44.ec2.internal"] Jul 06 01:26:45.075233 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:45.075175 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 01:26:45.076706 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:45.076686 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-44.ec2.internal" event="NodeHasSufficientMemory" Jul 06 01:26:45.076770 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:45.076726 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-44.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 01:26:45.076770 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:45.076737 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-44.ec2.internal" event="NodeHasSufficientPID" Jul 06 01:26:45.079111 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:45.079097 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 01:26:45.079233 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:45.079218 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-44.ec2.internal" Jul 06 01:26:45.079268 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:45.079248 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 01:26:45.079955 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:45.079936 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-44.ec2.internal" event="NodeHasSufficientMemory" Jul 06 01:26:45.080083 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:45.079965 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-44.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 01:26:45.080083 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:45.079938 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-44.ec2.internal" event="NodeHasSufficientMemory" Jul 06 01:26:45.080083 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:45.080004 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-44.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 01:26:45.080083 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:45.079975 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-44.ec2.internal" event="NodeHasSufficientPID" Jul 06 01:26:45.080083 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:45.080020 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-44.ec2.internal" event="NodeHasSufficientPID" Jul 06 01:26:45.082341 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:45.082325 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-44.ec2.internal" Jul 06 01:26:45.082455 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:45.082354 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 01:26:45.083121 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:45.083103 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-44.ec2.internal" event="NodeHasSufficientMemory" Jul 06 01:26:45.083206 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:45.083133 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-44.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 01:26:45.083206 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:45.083143 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-44.ec2.internal" event="NodeHasSufficientPID" Jul 06 01:26:45.098082 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:45.098057 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-44.ec2.internal\" not found" node="ip-10-0-139-44.ec2.internal" Jul 06 01:26:45.102637 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:45.102621 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-44.ec2.internal\" not found" node="ip-10-0-139-44.ec2.internal" Jul 06 01:26:45.147927 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:45.147897 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-44.ec2.internal\" not found" Jul 06 01:26:45.161351 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:45.161327 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5c47210a4f94f2c956e2eb14609b8c29-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-44.ec2.internal\" (UID: \"5c47210a4f94f2c956e2eb14609b8c29\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-44.ec2.internal" Jul 06 01:26:45.161470 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:45.161355 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5c47210a4f94f2c956e2eb14609b8c29-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-44.ec2.internal\" (UID: \"5c47210a4f94f2c956e2eb14609b8c29\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-44.ec2.internal" Jul 06 01:26:45.161470 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:45.161373 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fa04038e4e7372411052bdcfb35944fb-config\") pod \"kube-apiserver-proxy-ip-10-0-139-44.ec2.internal\" (UID: \"fa04038e4e7372411052bdcfb35944fb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-44.ec2.internal" Jul 06 01:26:45.248666 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:45.248630 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-44.ec2.internal\" not found" Jul 06 01:26:45.262070 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:45.262048 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5c47210a4f94f2c956e2eb14609b8c29-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-44.ec2.internal\" (UID: \"5c47210a4f94f2c956e2eb14609b8c29\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-44.ec2.internal" Jul 06 01:26:45.262164 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:45.262078 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fa04038e4e7372411052bdcfb35944fb-config\") pod \"kube-apiserver-proxy-ip-10-0-139-44.ec2.internal\" (UID: \"fa04038e4e7372411052bdcfb35944fb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-44.ec2.internal" Jul 06 01:26:45.262164 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:45.262111 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5c47210a4f94f2c956e2eb14609b8c29-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-44.ec2.internal\" (UID: \"5c47210a4f94f2c956e2eb14609b8c29\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-44.ec2.internal" Jul 06 01:26:45.262164 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:45.262140 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5c47210a4f94f2c956e2eb14609b8c29-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-44.ec2.internal\" (UID: \"5c47210a4f94f2c956e2eb14609b8c29\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-44.ec2.internal" Jul 06 01:26:45.262164 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:45.262153 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fa04038e4e7372411052bdcfb35944fb-config\") pod \"kube-apiserver-proxy-ip-10-0-139-44.ec2.internal\" (UID: \"fa04038e4e7372411052bdcfb35944fb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-44.ec2.internal" Jul 06 01:26:45.262282 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:45.262159 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5c47210a4f94f2c956e2eb14609b8c29-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-44.ec2.internal\" (UID: \"5c47210a4f94f2c956e2eb14609b8c29\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-44.ec2.internal" Jul 06 01:26:45.349576 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:45.349484 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-44.ec2.internal\" not found" Jul 06 01:26:45.401031 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:45.400991 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-44.ec2.internal" Jul 06 01:26:45.406229 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:45.406004 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-44.ec2.internal" Jul 06 01:26:45.450208 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:45.450176 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-44.ec2.internal\" not found" Jul 06 01:26:45.550738 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:45.550701 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-44.ec2.internal\" not found" Jul 06 01:26:45.651349 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:45.651267 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-44.ec2.internal\" not found" Jul 06 01:26:45.751913 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:45.751876 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-44.ec2.internal\" not found" Jul 06 01:26:45.805357 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:45.805319 2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 06 01:26:45.806035 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:45.805527 2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 01:26:45.834711 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:45.833339 2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 01:26:45.852054 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:45.852020 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-44.ec2.internal\" not found" Jul 06 01:26:47.351423 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:47.351389 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 01:21:46 +0000 UTC" deadline="2028-04-19 00:50:20.18411982 +0000 UTC" Jul 06 01:26:47.351558 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:47.351424 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15671h23m32.832699217s" Jul 06 01:26:47.431747 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:47.431704 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-44.ec2.internal\" not found" Jul 06 01:26:47.475910 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:47.475876 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa04038e4e7372411052bdcfb35944fb.slice/crio-ca883c476f17f1ad91fd982241137542547312a01f36100199858cbdd449912d WatchSource:0}: Error finding container ca883c476f17f1ad91fd982241137542547312a01f36100199858cbdd449912d: Status 404 returned error can't find the container with id ca883c476f17f1ad91fd982241137542547312a01f36100199858cbdd449912d Jul 06 01:26:47.476278 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:47.476263 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c47210a4f94f2c956e2eb14609b8c29.slice/crio-e61a2915438fb91525d89eaf9bd3f665b31a846cd2ccc6890ffc4bc40928d761 WatchSource:0}: Error finding container e61a2915438fb91525d89eaf9bd3f665b31a846cd2ccc6890ffc4bc40928d761: Status 404 returned error can't find the container with id e61a2915438fb91525d89eaf9bd3f665b31a846cd2ccc6890ffc4bc40928d761 Jul 06 01:26:47.482250 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:47.482226 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 01:26:47.515132 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:47.515096 2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 01:26:47.532396 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:47.532364 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-44.ec2.internal\" not found" Jul 06 01:26:47.633020 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:47.632983 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-44.ec2.internal\" not found" Jul 06 01:26:47.733566 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:47.733479 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-44.ec2.internal\" not found" Jul 06 01:26:47.816117 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:47.816082 2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 01:26:47.839098 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:47.839065 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-44.ec2.internal" Jul 06 01:26:47.855041 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:47.855010 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 01:26:47.856373 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:47.856349 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-44.ec2.internal" Jul 06 01:26:47.866813 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:47.866788 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 01:26:48.320868 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.320836 2580 apiserver.go:52] "Watching apiserver" Jul 06 01:26:48.328679 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.328647 2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 06 01:26:48.329074 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.329048 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9gm9","openshift-cluster-node-tuning-operator/tuned-2pmf7","openshift-dns/node-resolver-82mwf","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-44.ec2.internal","openshift-multus/multus-xszvt","openshift-multus/network-metrics-daemon-qnp65","kube-system/konnectivity-agent-cksd7","openshift-image-registry/node-ca-b46kx","openshift-multus/multus-additional-cni-plugins-bdpq8","openshift-network-diagnostics/network-check-target-57drk","openshift-network-operator/iptables-alerter-qmxp5","openshift-ovn-kubernetes/ovnkube-node-2sd2m","kube-system/global-pull-secret-syncer-xf5mp","kube-system/kube-apiserver-proxy-ip-10-0-139-44.ec2.internal"] Jul 06 01:26:48.331799 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.331773 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57drk" Jul 06 01:26:48.331937 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:48.331869 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-57drk" podUID="ab026023-6959-46fd-aaf6-0b145d5a956e" Jul 06 01:26:48.333958 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.333932 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xf5mp" Jul 06 01:26:48.334084 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:48.334011 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xf5mp" podUID="e690e9f2-b314-4e44-a211-433d7e8100a1" Jul 06 01:26:48.336190 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.336159 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-82mwf" Jul 06 01:26:48.338701 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.338675 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 06 01:26:48.339030 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.339014 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 06 01:26:48.339239 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.339225 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-6mg99\"" Jul 06 01:26:48.339620 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.339605 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 06 01:26:48.340216 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.340198 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-b46kx" Jul 06 01:26:48.341977 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.341958 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-xszvt" Jul 06 01:26:48.343051 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.343032 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 06 01:26:48.343221 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.343204 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 06 01:26:48.343546 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.343498 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 06 01:26:48.343546 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.343511 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-mdm9w\"" Jul 06 01:26:48.343687 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.343551 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qnp65" Jul 06 01:26:48.343687 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:48.343624 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qnp65" podUID="15c8ffd6-1e00-4975-b807-63e00ad36b44" Jul 06 01:26:48.344661 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.344645 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-h6b4t\"" Jul 06 01:26:48.344904 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.344886 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 06 01:26:48.345098 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.345081 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 06 01:26:48.345347 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.345329 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 06 01:26:48.345596 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.345579 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 06 01:26:48.346821 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.346324 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bdpq8" Jul 06 01:26:48.349659 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.349575 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-cksd7" Jul 06 01:26:48.350179 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.349695 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdkgm\" (UniqueName: \"kubernetes.io/projected/ab026023-6959-46fd-aaf6-0b145d5a956e-kube-api-access-tdkgm\") pod \"network-check-target-57drk\" (UID: \"ab026023-6959-46fd-aaf6-0b145d5a956e\") " pod="openshift-network-diagnostics/network-check-target-57drk" Jul 06 01:26:48.350273 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.350217 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e690e9f2-b314-4e44-a211-433d7e8100a1-dbus\") pod \"global-pull-secret-syncer-xf5mp\" (UID: \"e690e9f2-b314-4e44-a211-433d7e8100a1\") " pod="kube-system/global-pull-secret-syncer-xf5mp" Jul 06 01:26:48.350273 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.350253 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ee8fc4f9-9bc1-4b56-a435-f158b76784fb-multus-daemon-config\") pod \"multus-xszvt\" (UID: \"ee8fc4f9-9bc1-4b56-a435-f158b76784fb\") " pod="openshift-multus/multus-xszvt" Jul 06 01:26:48.350372 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.350291 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ee8fc4f9-9bc1-4b56-a435-f158b76784fb-multus-conf-dir\") pod \"multus-xszvt\" (UID: \"ee8fc4f9-9bc1-4b56-a435-f158b76784fb\") " pod="openshift-multus/multus-xszvt" Jul 06 01:26:48.350372 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.350304 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 06 01:26:48.350372 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.350320 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/00a6f863-8ae0-4749-8f27-3bdf91cd8775-hosts-file\") pod \"node-resolver-82mwf\" (UID: \"00a6f863-8ae0-4749-8f27-3bdf91cd8775\") " pod="openshift-dns/node-resolver-82mwf" Jul 06 01:26:48.350372 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.350345 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4164d767-f4d0-459d-b0c1-cc8f8ccbaf1a-serviceca\") pod \"node-ca-b46kx\" (UID: \"4164d767-f4d0-459d-b0c1-cc8f8ccbaf1a\") " pod="openshift-image-registry/node-ca-b46kx" Jul 06 01:26:48.350593 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.350376 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ee8fc4f9-9bc1-4b56-a435-f158b76784fb-os-release\") pod \"multus-xszvt\" (UID: \"ee8fc4f9-9bc1-4b56-a435-f158b76784fb\") " pod="openshift-multus/multus-xszvt" Jul 06 01:26:48.350593 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.350406 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ee8fc4f9-9bc1-4b56-a435-f158b76784fb-host-run-k8s-cni-cncf-io\") pod \"multus-xszvt\" (UID: \"ee8fc4f9-9bc1-4b56-a435-f158b76784fb\") " pod="openshift-multus/multus-xszvt" Jul 06 01:26:48.350593 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.350460 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15c8ffd6-1e00-4975-b807-63e00ad36b44-metrics-certs\") pod \"network-metrics-daemon-qnp65\" (UID: \"15c8ffd6-1e00-4975-b807-63e00ad36b44\") " pod="openshift-multus/network-metrics-daemon-qnp65" Jul 06 01:26:48.350593 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.350492 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rmnr\" (UniqueName: \"kubernetes.io/projected/00a6f863-8ae0-4749-8f27-3bdf91cd8775-kube-api-access-7rmnr\") pod \"node-resolver-82mwf\" (UID: \"00a6f863-8ae0-4749-8f27-3bdf91cd8775\") " pod="openshift-dns/node-resolver-82mwf" Jul 06 01:26:48.350593 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.350347 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-x925q\"" Jul 06 01:26:48.350593 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.350552 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ee8fc4f9-9bc1-4b56-a435-f158b76784fb-multus-cni-dir\") pod \"multus-xszvt\" (UID: \"ee8fc4f9-9bc1-4b56-a435-f158b76784fb\") " pod="openshift-multus/multus-xszvt" Jul 06 01:26:48.351680 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.350949 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 06 01:26:48.351680 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.350979 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ee8fc4f9-9bc1-4b56-a435-f158b76784fb-multus-socket-dir-parent\") pod \"multus-xszvt\" (UID: \"ee8fc4f9-9bc1-4b56-a435-f158b76784fb\") " pod="openshift-multus/multus-xszvt" Jul 06 01:26:48.351680 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.351026 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ee8fc4f9-9bc1-4b56-a435-f158b76784fb-host-var-lib-cni-multus\") pod \"multus-xszvt\" (UID: \"ee8fc4f9-9bc1-4b56-a435-f158b76784fb\") " pod="openshift-multus/multus-xszvt" Jul 06 01:26:48.351680 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.351055 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjs7p\" (UniqueName: \"kubernetes.io/projected/ee8fc4f9-9bc1-4b56-a435-f158b76784fb-kube-api-access-mjs7p\") pod \"multus-xszvt\" (UID: \"ee8fc4f9-9bc1-4b56-a435-f158b76784fb\") " pod="openshift-multus/multus-xszvt" Jul 06 01:26:48.351680 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.351080 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ee8fc4f9-9bc1-4b56-a435-f158b76784fb-host-run-netns\") pod \"multus-xszvt\" (UID: \"ee8fc4f9-9bc1-4b56-a435-f158b76784fb\") " pod="openshift-multus/multus-xszvt" Jul 06 01:26:48.351680 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.351140 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e690e9f2-b314-4e44-a211-433d7e8100a1-original-pull-secret\") pod \"global-pull-secret-syncer-xf5mp\" (UID: \"e690e9f2-b314-4e44-a211-433d7e8100a1\") " pod="kube-system/global-pull-secret-syncer-xf5mp" Jul 06 01:26:48.351680 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.351177 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00a6f863-8ae0-4749-8f27-3bdf91cd8775-tmp-dir\") pod \"node-resolver-82mwf\" (UID: \"00a6f863-8ae0-4749-8f27-3bdf91cd8775\") " pod="openshift-dns/node-resolver-82mwf" Jul 06 01:26:48.351680 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.351215 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ee8fc4f9-9bc1-4b56-a435-f158b76784fb-hostroot\") pod \"multus-xszvt\" (UID: \"ee8fc4f9-9bc1-4b56-a435-f158b76784fb\") " pod="openshift-multus/multus-xszvt" Jul 06 01:26:48.351680 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.351255 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ee8fc4f9-9bc1-4b56-a435-f158b76784fb-host-var-lib-cni-bin\") pod \"multus-xszvt\" (UID: \"ee8fc4f9-9bc1-4b56-a435-f158b76784fb\") " pod="openshift-multus/multus-xszvt" Jul 06 01:26:48.351680 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.351344 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e690e9f2-b314-4e44-a211-433d7e8100a1-kubelet-config\") pod \"global-pull-secret-syncer-xf5mp\" (UID: \"e690e9f2-b314-4e44-a211-433d7e8100a1\") " pod="kube-system/global-pull-secret-syncer-xf5mp" Jul 06 01:26:48.351680 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.351366 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4164d767-f4d0-459d-b0c1-cc8f8ccbaf1a-host\") pod \"node-ca-b46kx\" (UID: \"4164d767-f4d0-459d-b0c1-cc8f8ccbaf1a\") " pod="openshift-image-registry/node-ca-b46kx" Jul 06 01:26:48.351680 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.351391 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ee8fc4f9-9bc1-4b56-a435-f158b76784fb-cnibin\") pod \"multus-xszvt\" (UID: \"ee8fc4f9-9bc1-4b56-a435-f158b76784fb\") " pod="openshift-multus/multus-xszvt" Jul 06 01:26:48.351680 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.351412 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ee8fc4f9-9bc1-4b56-a435-f158b76784fb-cni-binary-copy\") pod \"multus-xszvt\" (UID: \"ee8fc4f9-9bc1-4b56-a435-f158b76784fb\") " pod="openshift-multus/multus-xszvt" Jul 06 01:26:48.351680 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.351499 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ee8fc4f9-9bc1-4b56-a435-f158b76784fb-etc-kubernetes\") pod \"multus-xszvt\" (UID: \"ee8fc4f9-9bc1-4b56-a435-f158b76784fb\") " pod="openshift-multus/multus-xszvt" Jul 06 01:26:48.351680 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.351534 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9rqh\" (UniqueName: \"kubernetes.io/projected/15c8ffd6-1e00-4975-b807-63e00ad36b44-kube-api-access-x9rqh\") pod \"network-metrics-daemon-qnp65\" (UID: \"15c8ffd6-1e00-4975-b807-63e00ad36b44\") " pod="openshift-multus/network-metrics-daemon-qnp65" Jul 06 01:26:48.351680 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.351557 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee8fc4f9-9bc1-4b56-a435-f158b76784fb-host-var-lib-kubelet\") pod \"multus-xszvt\" (UID: \"ee8fc4f9-9bc1-4b56-a435-f158b76784fb\") " pod="openshift-multus/multus-xszvt" Jul 06 01:26:48.351680 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.351578 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ee8fc4f9-9bc1-4b56-a435-f158b76784fb-host-run-multus-certs\") pod \"multus-xszvt\" (UID: \"ee8fc4f9-9bc1-4b56-a435-f158b76784fb\") " pod="openshift-multus/multus-xszvt" Jul 06 01:26:48.352502 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.351603 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2j5fp\" (UniqueName: \"kubernetes.io/projected/4164d767-f4d0-459d-b0c1-cc8f8ccbaf1a-kube-api-access-2j5fp\") pod \"node-ca-b46kx\" (UID: \"4164d767-f4d0-459d-b0c1-cc8f8ccbaf1a\") " pod="openshift-image-registry/node-ca-b46kx" Jul 06 01:26:48.352502 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.351626 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ee8fc4f9-9bc1-4b56-a435-f158b76784fb-system-cni-dir\") pod \"multus-xszvt\" (UID: \"ee8fc4f9-9bc1-4b56-a435-f158b76784fb\") " pod="openshift-multus/multus-xszvt" Jul 06 01:26:48.352502 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.352212 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2pmf7" Jul 06 01:26:48.352502 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.352319 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 06 01:26:48.354052 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.354031 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 06 01:26:48.354124 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.354108 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-bj66x\"" Jul 06 01:26:48.354865 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.354843 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9gm9" Jul 06 01:26:48.354966 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.354883 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 06 01:26:48.355287 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.355272 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 06 01:26:48.355614 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.355363 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-sccwj\"" Jul 06 01:26:48.357132 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.357114 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qmxp5" Jul 06 01:26:48.359559 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.359534 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 06 01:26:48.359818 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.359799 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 06 01:26:48.359970 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.359952 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.360098 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.360074 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 06 01:26:48.360557 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.360388 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 06 01:26:48.361240 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.361016 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-k65tm\"" Jul 06 01:26:48.361240 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.361082 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-9blcj\"" Jul 06 01:26:48.361240 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.361096 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 01:26:48.361240 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.361102 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 06 01:26:48.361240 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.361183 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 06 01:26:48.364352 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.364336 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 06 01:26:48.364460 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.364357 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 06 01:26:48.364617 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.364598 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 06 01:26:48.364687 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.364660 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 06 01:26:48.364735 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.364605 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 06 01:26:48.365507 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.365487 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-gzqbf\"" Jul 06 01:26:48.365646 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.365493 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 06 01:26:48.379269 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.379243 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-bhnfq" Jul 06 01:26:48.387429 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.387406 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-bhnfq" Jul 06 01:26:48.440686 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.440652 2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 06 01:26:48.451800 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.451771 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/925f85db-233c-4a3f-9d1c-f8238bce7e4c-run\") pod \"tuned-2pmf7\" (UID: \"925f85db-233c-4a3f-9d1c-f8238bce7e4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2pmf7" Jul 06 01:26:48.451979 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.451835 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/925f85db-233c-4a3f-9d1c-f8238bce7e4c-sys\") pod \"tuned-2pmf7\" (UID: \"925f85db-233c-4a3f-9d1c-f8238bce7e4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2pmf7" Jul 06 01:26:48.451979 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.451860 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-host-slash\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.451979 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.451885 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-etc-openvswitch\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.451979 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.451905 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-host-cni-bin\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.451979 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.451948 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-ovn-node-metrics-cert\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.452244 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.451990 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9fd5\" (UniqueName: \"kubernetes.io/projected/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-kube-api-access-x9fd5\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.452244 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.452033 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c476bace-97d0-4492-adca-efd95306c6de-etc-selinux\") pod \"aws-ebs-csi-driver-node-w9gm9\" (UID: \"c476bace-97d0-4492-adca-efd95306c6de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9gm9" Jul 06 01:26:48.452244 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.452063 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ee8fc4f9-9bc1-4b56-a435-f158b76784fb-host-var-lib-cni-bin\") pod \"multus-xszvt\" (UID: \"ee8fc4f9-9bc1-4b56-a435-f158b76784fb\") " pod="openshift-multus/multus-xszvt" Jul 06 01:26:48.452244 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.452097 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e690e9f2-b314-4e44-a211-433d7e8100a1-kubelet-config\") pod \"global-pull-secret-syncer-xf5mp\" (UID: \"e690e9f2-b314-4e44-a211-433d7e8100a1\") " pod="kube-system/global-pull-secret-syncer-xf5mp" Jul 06 01:26:48.452244 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.452137 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4164d767-f4d0-459d-b0c1-cc8f8ccbaf1a-host\") pod \"node-ca-b46kx\" (UID: \"4164d767-f4d0-459d-b0c1-cc8f8ccbaf1a\") " pod="openshift-image-registry/node-ca-b46kx" Jul 06 01:26:48.452244 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.452161 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ee8fc4f9-9bc1-4b56-a435-f158b76784fb-cnibin\") pod \"multus-xszvt\" (UID: \"ee8fc4f9-9bc1-4b56-a435-f158b76784fb\") " pod="openshift-multus/multus-xszvt" Jul 06 01:26:48.452244 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.452161 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ee8fc4f9-9bc1-4b56-a435-f158b76784fb-host-var-lib-cni-bin\") pod \"multus-xszvt\" (UID: \"ee8fc4f9-9bc1-4b56-a435-f158b76784fb\") " pod="openshift-multus/multus-xszvt" Jul 06 01:26:48.452244 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.452197 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/925f85db-233c-4a3f-9d1c-f8238bce7e4c-etc-systemd\") pod \"tuned-2pmf7\" (UID: \"925f85db-233c-4a3f-9d1c-f8238bce7e4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2pmf7" Jul 06 01:26:48.452244 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.452231 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e690e9f2-b314-4e44-a211-433d7e8100a1-kubelet-config\") pod \"global-pull-secret-syncer-xf5mp\" (UID: \"e690e9f2-b314-4e44-a211-433d7e8100a1\") " pod="kube-system/global-pull-secret-syncer-xf5mp" Jul 06 01:26:48.452632 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.452295 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4164d767-f4d0-459d-b0c1-cc8f8ccbaf1a-host\") pod \"node-ca-b46kx\" (UID: \"4164d767-f4d0-459d-b0c1-cc8f8ccbaf1a\") " pod="openshift-image-registry/node-ca-b46kx" Jul 06 01:26:48.452632 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.452297 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-run-ovn\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.452632 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.452325 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ee8fc4f9-9bc1-4b56-a435-f158b76784fb-cnibin\") pod \"multus-xszvt\" (UID: \"ee8fc4f9-9bc1-4b56-a435-f158b76784fb\") " pod="openshift-multus/multus-xszvt" Jul 06 01:26:48.452632 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.452357 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-env-overrides\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.452632 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.452389 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee8fc4f9-9bc1-4b56-a435-f158b76784fb-host-var-lib-kubelet\") pod \"multus-xszvt\" (UID: \"ee8fc4f9-9bc1-4b56-a435-f158b76784fb\") " pod="openshift-multus/multus-xszvt" Jul 06 01:26:48.452632 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.452413 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ee8fc4f9-9bc1-4b56-a435-f158b76784fb-host-run-multus-certs\") pod \"multus-xszvt\" (UID: \"ee8fc4f9-9bc1-4b56-a435-f158b76784fb\") " pod="openshift-multus/multus-xszvt" Jul 06 01:26:48.452632 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.452431 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-systemd-units\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.452632 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.452471 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-var-lib-openvswitch\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.452632 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.452474 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee8fc4f9-9bc1-4b56-a435-f158b76784fb-host-var-lib-kubelet\") pod \"multus-xszvt\" (UID: \"ee8fc4f9-9bc1-4b56-a435-f158b76784fb\") " pod="openshift-multus/multus-xszvt" Jul 06 01:26:48.452632 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.452494 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.452632 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.452510 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ee8fc4f9-9bc1-4b56-a435-f158b76784fb-host-run-multus-certs\") pod \"multus-xszvt\" (UID: \"ee8fc4f9-9bc1-4b56-a435-f158b76784fb\") " pod="openshift-multus/multus-xszvt" Jul 06 01:26:48.452632 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.452525 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/86824090-3c80-459c-97b6-4e34bcac19c3-system-cni-dir\") pod \"multus-additional-cni-plugins-bdpq8\" (UID: \"86824090-3c80-459c-97b6-4e34bcac19c3\") " pod="openshift-multus/multus-additional-cni-plugins-bdpq8" Jul 06 01:26:48.452632 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.452553 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/86824090-3c80-459c-97b6-4e34bcac19c3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bdpq8\" (UID: \"86824090-3c80-459c-97b6-4e34bcac19c3\") " pod="openshift-multus/multus-additional-cni-plugins-bdpq8" Jul 06 01:26:48.452632 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.452583 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2j5fp\" (UniqueName: \"kubernetes.io/projected/4164d767-f4d0-459d-b0c1-cc8f8ccbaf1a-kube-api-access-2j5fp\") pod \"node-ca-b46kx\" (UID: \"4164d767-f4d0-459d-b0c1-cc8f8ccbaf1a\") " pod="openshift-image-registry/node-ca-b46kx" Jul 06 01:26:48.452632 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.452610 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ee8fc4f9-9bc1-4b56-a435-f158b76784fb-system-cni-dir\") pod \"multus-xszvt\" (UID: \"ee8fc4f9-9bc1-4b56-a435-f158b76784fb\") " pod="openshift-multus/multus-xszvt" Jul 06 01:26:48.453288 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.452689 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ee8fc4f9-9bc1-4b56-a435-f158b76784fb-system-cni-dir\") pod \"multus-xszvt\" (UID: \"ee8fc4f9-9bc1-4b56-a435-f158b76784fb\") " pod="openshift-multus/multus-xszvt" Jul 06 01:26:48.453288 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.452699 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c476bace-97d0-4492-adca-efd95306c6de-device-dir\") pod \"aws-ebs-csi-driver-node-w9gm9\" (UID: \"c476bace-97d0-4492-adca-efd95306c6de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9gm9" Jul 06 01:26:48.453288 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.452732 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/86824090-3c80-459c-97b6-4e34bcac19c3-cnibin\") pod \"multus-additional-cni-plugins-bdpq8\" (UID: \"86824090-3c80-459c-97b6-4e34bcac19c3\") " pod="openshift-multus/multus-additional-cni-plugins-bdpq8" Jul 06 01:26:48.453288 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.452749 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/86824090-3c80-459c-97b6-4e34bcac19c3-os-release\") pod \"multus-additional-cni-plugins-bdpq8\" (UID: \"86824090-3c80-459c-97b6-4e34bcac19c3\") " pod="openshift-multus/multus-additional-cni-plugins-bdpq8" Jul 06 01:26:48.453288 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.452764 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-host-run-ovn-kubernetes\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.453288 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.452781 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ee8fc4f9-9bc1-4b56-a435-f158b76784fb-multus-conf-dir\") pod \"multus-xszvt\" (UID: \"ee8fc4f9-9bc1-4b56-a435-f158b76784fb\") " pod="openshift-multus/multus-xszvt" Jul 06 01:26:48.453288 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.452797 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4164d767-f4d0-459d-b0c1-cc8f8ccbaf1a-serviceca\") pod \"node-ca-b46kx\" (UID: \"4164d767-f4d0-459d-b0c1-cc8f8ccbaf1a\") " pod="openshift-image-registry/node-ca-b46kx" Jul 06 01:26:48.453288 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.452832 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ee8fc4f9-9bc1-4b56-a435-f158b76784fb-multus-conf-dir\") pod \"multus-xszvt\" (UID: \"ee8fc4f9-9bc1-4b56-a435-f158b76784fb\") " pod="openshift-multus/multus-xszvt" Jul 06 01:26:48.453288 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.452870 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ee8fc4f9-9bc1-4b56-a435-f158b76784fb-host-run-k8s-cni-cncf-io\") pod \"multus-xszvt\" (UID: \"ee8fc4f9-9bc1-4b56-a435-f158b76784fb\") " pod="openshift-multus/multus-xszvt" Jul 06 01:26:48.453288 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.452903 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/925f85db-233c-4a3f-9d1c-f8238bce7e4c-etc-sysctl-d\") pod \"tuned-2pmf7\" (UID: \"925f85db-233c-4a3f-9d1c-f8238bce7e4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2pmf7" Jul 06 01:26:48.453288 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.452928 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c476bace-97d0-4492-adca-efd95306c6de-socket-dir\") pod \"aws-ebs-csi-driver-node-w9gm9\" (UID: \"c476bace-97d0-4492-adca-efd95306c6de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9gm9" Jul 06 01:26:48.453288 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.452951 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-ovnkube-script-lib\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.453288 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.452977 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15c8ffd6-1e00-4975-b807-63e00ad36b44-metrics-certs\") pod \"network-metrics-daemon-qnp65\" (UID: \"15c8ffd6-1e00-4975-b807-63e00ad36b44\") " pod="openshift-multus/network-metrics-daemon-qnp65" Jul 06 01:26:48.453288 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.452976 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ee8fc4f9-9bc1-4b56-a435-f158b76784fb-host-run-k8s-cni-cncf-io\") pod \"multus-xszvt\" (UID: \"ee8fc4f9-9bc1-4b56-a435-f158b76784fb\") " pod="openshift-multus/multus-xszvt" Jul 06 01:26:48.453288 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.453001 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ee8fc4f9-9bc1-4b56-a435-f158b76784fb-multus-socket-dir-parent\") pod \"multus-xszvt\" (UID: \"ee8fc4f9-9bc1-4b56-a435-f158b76784fb\") " pod="openshift-multus/multus-xszvt" Jul 06 01:26:48.453288 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.453031 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjs7p\" (UniqueName: \"kubernetes.io/projected/ee8fc4f9-9bc1-4b56-a435-f158b76784fb-kube-api-access-mjs7p\") pod \"multus-xszvt\" (UID: \"ee8fc4f9-9bc1-4b56-a435-f158b76784fb\") " pod="openshift-multus/multus-xszvt" Jul 06 01:26:48.453288 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.453064 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cb081505-8ac0-48a3-be9e-bf770a830127-konnectivity-ca\") pod \"konnectivity-agent-cksd7\" (UID: \"cb081505-8ac0-48a3-be9e-bf770a830127\") " pod="kube-system/konnectivity-agent-cksd7" Jul 06 01:26:48.453999 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:48.453094 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 01:26:48.453999 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.453094 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e43deb0d-17a0-4c43-8120-3edd3732068a-iptables-alerter-script\") pod \"iptables-alerter-qmxp5\" (UID: \"e43deb0d-17a0-4c43-8120-3edd3732068a\") " pod="openshift-network-operator/iptables-alerter-qmxp5" Jul 06 01:26:48.453999 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:48.453179 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15c8ffd6-1e00-4975-b807-63e00ad36b44-metrics-certs podName:15c8ffd6-1e00-4975-b807-63e00ad36b44 nodeName:}" failed. No retries permitted until 2026-07-06 01:26:48.953144577 +0000 UTC m=+2.924396245 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/15c8ffd6-1e00-4975-b807-63e00ad36b44-metrics-certs") pod "network-metrics-daemon-qnp65" (UID: "15c8ffd6-1e00-4975-b807-63e00ad36b44") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 01:26:48.453999 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.453224 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4164d767-f4d0-459d-b0c1-cc8f8ccbaf1a-serviceca\") pod \"node-ca-b46kx\" (UID: \"4164d767-f4d0-459d-b0c1-cc8f8ccbaf1a\") " pod="openshift-image-registry/node-ca-b46kx" Jul 06 01:26:48.453999 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.453231 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ee8fc4f9-9bc1-4b56-a435-f158b76784fb-host-run-netns\") pod \"multus-xszvt\" (UID: \"ee8fc4f9-9bc1-4b56-a435-f158b76784fb\") " pod="openshift-multus/multus-xszvt" Jul 06 01:26:48.453999 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.453258 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/925f85db-233c-4a3f-9d1c-f8238bce7e4c-etc-sysconfig\") pod \"tuned-2pmf7\" (UID: \"925f85db-233c-4a3f-9d1c-f8238bce7e4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2pmf7" Jul 06 01:26:48.453999 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.453282 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-ovnkube-config\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.453999 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.453311 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ee8fc4f9-9bc1-4b56-a435-f158b76784fb-multus-socket-dir-parent\") pod \"multus-xszvt\" (UID: \"ee8fc4f9-9bc1-4b56-a435-f158b76784fb\") " pod="openshift-multus/multus-xszvt" Jul 06 01:26:48.453999 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.453351 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2z4ms\" (UniqueName: \"kubernetes.io/projected/86824090-3c80-459c-97b6-4e34bcac19c3-kube-api-access-2z4ms\") pod \"multus-additional-cni-plugins-bdpq8\" (UID: \"86824090-3c80-459c-97b6-4e34bcac19c3\") " pod="openshift-multus/multus-additional-cni-plugins-bdpq8" Jul 06 01:26:48.453999 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.453379 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e690e9f2-b314-4e44-a211-433d7e8100a1-original-pull-secret\") pod \"global-pull-secret-syncer-xf5mp\" (UID: \"e690e9f2-b314-4e44-a211-433d7e8100a1\") " pod="kube-system/global-pull-secret-syncer-xf5mp" Jul 06 01:26:48.453999 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.453381 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ee8fc4f9-9bc1-4b56-a435-f158b76784fb-host-run-netns\") pod \"multus-xszvt\" (UID: \"ee8fc4f9-9bc1-4b56-a435-f158b76784fb\") " pod="openshift-multus/multus-xszvt" Jul 06 01:26:48.453999 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.453421 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00a6f863-8ae0-4749-8f27-3bdf91cd8775-tmp-dir\") pod \"node-resolver-82mwf\" (UID: \"00a6f863-8ae0-4749-8f27-3bdf91cd8775\") " pod="openshift-dns/node-resolver-82mwf" Jul 06 01:26:48.453999 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:48.453469 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 01:26:48.453999 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.453501 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ee8fc4f9-9bc1-4b56-a435-f158b76784fb-hostroot\") pod \"multus-xszvt\" (UID: \"ee8fc4f9-9bc1-4b56-a435-f158b76784fb\") " pod="openshift-multus/multus-xszvt" Jul 06 01:26:48.453999 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.453527 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/925f85db-233c-4a3f-9d1c-f8238bce7e4c-etc-modprobe-d\") pod \"tuned-2pmf7\" (UID: \"925f85db-233c-4a3f-9d1c-f8238bce7e4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2pmf7" Jul 06 01:26:48.453999 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.453586 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/86824090-3c80-459c-97b6-4e34bcac19c3-cni-binary-copy\") pod \"multus-additional-cni-plugins-bdpq8\" (UID: \"86824090-3c80-459c-97b6-4e34bcac19c3\") " pod="openshift-multus/multus-additional-cni-plugins-bdpq8" Jul 06 01:26:48.453999 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.453608 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ee8fc4f9-9bc1-4b56-a435-f158b76784fb-cni-binary-copy\") pod \"multus-xszvt\" (UID: \"ee8fc4f9-9bc1-4b56-a435-f158b76784fb\") " pod="openshift-multus/multus-xszvt" Jul 06 01:26:48.454822 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.453628 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ee8fc4f9-9bc1-4b56-a435-f158b76784fb-etc-kubernetes\") pod \"multus-xszvt\" (UID: \"ee8fc4f9-9bc1-4b56-a435-f158b76784fb\") " pod="openshift-multus/multus-xszvt" Jul 06 01:26:48.454822 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.453650 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/925f85db-233c-4a3f-9d1c-f8238bce7e4c-etc-tuned\") pod \"tuned-2pmf7\" (UID: \"925f85db-233c-4a3f-9d1c-f8238bce7e4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2pmf7" Jul 06 01:26:48.454822 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.453687 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/86824090-3c80-459c-97b6-4e34bcac19c3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bdpq8\" (UID: \"86824090-3c80-459c-97b6-4e34bcac19c3\") " pod="openshift-multus/multus-additional-cni-plugins-bdpq8" Jul 06 01:26:48.454822 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.453704 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2f6t7\" (UniqueName: \"kubernetes.io/projected/e43deb0d-17a0-4c43-8120-3edd3732068a-kube-api-access-2f6t7\") pod \"iptables-alerter-qmxp5\" (UID: \"e43deb0d-17a0-4c43-8120-3edd3732068a\") " pod="openshift-network-operator/iptables-alerter-qmxp5" Jul 06 01:26:48.454822 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.453723 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9rqh\" (UniqueName: \"kubernetes.io/projected/15c8ffd6-1e00-4975-b807-63e00ad36b44-kube-api-access-x9rqh\") pod \"network-metrics-daemon-qnp65\" (UID: \"15c8ffd6-1e00-4975-b807-63e00ad36b44\") " pod="openshift-multus/network-metrics-daemon-qnp65" Jul 06 01:26:48.454822 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.453742 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-log-socket\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.454822 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.453747 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00a6f863-8ae0-4749-8f27-3bdf91cd8775-tmp-dir\") pod \"node-resolver-82mwf\" (UID: \"00a6f863-8ae0-4749-8f27-3bdf91cd8775\") " pod="openshift-dns/node-resolver-82mwf" Jul 06 01:26:48.454822 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.453764 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/925f85db-233c-4a3f-9d1c-f8238bce7e4c-etc-kubernetes\") pod \"tuned-2pmf7\" (UID: \"925f85db-233c-4a3f-9d1c-f8238bce7e4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2pmf7" Jul 06 01:26:48.454822 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.453807 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/925f85db-233c-4a3f-9d1c-f8238bce7e4c-lib-modules\") pod \"tuned-2pmf7\" (UID: \"925f85db-233c-4a3f-9d1c-f8238bce7e4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2pmf7" Jul 06 01:26:48.454822 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.453824 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c476bace-97d0-4492-adca-efd95306c6de-sys-fs\") pod \"aws-ebs-csi-driver-node-w9gm9\" (UID: \"c476bace-97d0-4492-adca-efd95306c6de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9gm9" Jul 06 01:26:48.454822 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.453874 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdkgm\" (UniqueName: \"kubernetes.io/projected/ab026023-6959-46fd-aaf6-0b145d5a956e-kube-api-access-tdkgm\") pod \"network-check-target-57drk\" (UID: \"ab026023-6959-46fd-aaf6-0b145d5a956e\") " pod="openshift-network-diagnostics/network-check-target-57drk" Jul 06 01:26:48.454822 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.453898 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e690e9f2-b314-4e44-a211-433d7e8100a1-dbus\") pod \"global-pull-secret-syncer-xf5mp\" (UID: \"e690e9f2-b314-4e44-a211-433d7e8100a1\") " pod="kube-system/global-pull-secret-syncer-xf5mp" Jul 06 01:26:48.454822 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.453923 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ee8fc4f9-9bc1-4b56-a435-f158b76784fb-multus-daemon-config\") pod \"multus-xszvt\" (UID: \"ee8fc4f9-9bc1-4b56-a435-f158b76784fb\") " pod="openshift-multus/multus-xszvt" Jul 06 01:26:48.454822 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:48.453946 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e690e9f2-b314-4e44-a211-433d7e8100a1-original-pull-secret podName:e690e9f2-b314-4e44-a211-433d7e8100a1 nodeName:}" failed. No retries permitted until 2026-07-06 01:26:48.953922958 +0000 UTC m=+2.925174638 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e690e9f2-b314-4e44-a211-433d7e8100a1-original-pull-secret") pod "global-pull-secret-syncer-xf5mp" (UID: "e690e9f2-b314-4e44-a211-433d7e8100a1") : object "kube-system"/"original-pull-secret" not registered Jul 06 01:26:48.454822 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.453985 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/925f85db-233c-4a3f-9d1c-f8238bce7e4c-var-lib-kubelet\") pod \"tuned-2pmf7\" (UID: \"925f85db-233c-4a3f-9d1c-f8238bce7e4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2pmf7" Jul 06 01:26:48.454822 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.454000 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ee8fc4f9-9bc1-4b56-a435-f158b76784fb-etc-kubernetes\") pod \"multus-xszvt\" (UID: \"ee8fc4f9-9bc1-4b56-a435-f158b76784fb\") " pod="openshift-multus/multus-xszvt" Jul 06 01:26:48.455485 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.454012 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2jz6\" (UniqueName: \"kubernetes.io/projected/925f85db-233c-4a3f-9d1c-f8238bce7e4c-kube-api-access-x2jz6\") pod \"tuned-2pmf7\" (UID: \"925f85db-233c-4a3f-9d1c-f8238bce7e4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2pmf7" Jul 06 01:26:48.455485 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.454038 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c476bace-97d0-4492-adca-efd95306c6de-kubelet-dir\") pod \"aws-ebs-csi-driver-node-w9gm9\" (UID: \"c476bace-97d0-4492-adca-efd95306c6de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9gm9" Jul 06 01:26:48.455485 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.454063 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-run-openvswitch\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.455485 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.454214 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/00a6f863-8ae0-4749-8f27-3bdf91cd8775-hosts-file\") pod \"node-resolver-82mwf\" (UID: \"00a6f863-8ae0-4749-8f27-3bdf91cd8775\") " pod="openshift-dns/node-resolver-82mwf" Jul 06 01:26:48.455485 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.454236 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ee8fc4f9-9bc1-4b56-a435-f158b76784fb-os-release\") pod \"multus-xszvt\" (UID: \"ee8fc4f9-9bc1-4b56-a435-f158b76784fb\") " pod="openshift-multus/multus-xszvt" Jul 06 01:26:48.455485 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.454257 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-run-systemd\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.455485 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.454274 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-node-log\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.455485 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.454289 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e43deb0d-17a0-4c43-8120-3edd3732068a-host-slash\") pod \"iptables-alerter-qmxp5\" (UID: \"e43deb0d-17a0-4c43-8120-3edd3732068a\") " pod="openshift-network-operator/iptables-alerter-qmxp5" Jul 06 01:26:48.455485 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.454302 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ee8fc4f9-9bc1-4b56-a435-f158b76784fb-cni-binary-copy\") pod \"multus-xszvt\" (UID: \"ee8fc4f9-9bc1-4b56-a435-f158b76784fb\") " pod="openshift-multus/multus-xszvt" Jul 06 01:26:48.455485 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.454306 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rmnr\" (UniqueName: \"kubernetes.io/projected/00a6f863-8ae0-4749-8f27-3bdf91cd8775-kube-api-access-7rmnr\") pod \"node-resolver-82mwf\" (UID: \"00a6f863-8ae0-4749-8f27-3bdf91cd8775\") " pod="openshift-dns/node-resolver-82mwf" Jul 06 01:26:48.455485 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.454349 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ee8fc4f9-9bc1-4b56-a435-f158b76784fb-multus-cni-dir\") pod \"multus-xszvt\" (UID: \"ee8fc4f9-9bc1-4b56-a435-f158b76784fb\") " pod="openshift-multus/multus-xszvt" Jul 06 01:26:48.455485 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.454379 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ee8fc4f9-9bc1-4b56-a435-f158b76784fb-host-var-lib-cni-multus\") pod \"multus-xszvt\" (UID: \"ee8fc4f9-9bc1-4b56-a435-f158b76784fb\") " pod="openshift-multus/multus-xszvt" Jul 06 01:26:48.455485 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.454405 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/925f85db-233c-4a3f-9d1c-f8238bce7e4c-tmp\") pod \"tuned-2pmf7\" (UID: \"925f85db-233c-4a3f-9d1c-f8238bce7e4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2pmf7" Jul 06 01:26:48.455485 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.454430 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-host-kubelet\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.455485 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.454478 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-host-cni-netd\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.455485 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.454503 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cb081505-8ac0-48a3-be9e-bf770a830127-agent-certs\") pod \"konnectivity-agent-cksd7\" (UID: \"cb081505-8ac0-48a3-be9e-bf770a830127\") " pod="kube-system/konnectivity-agent-cksd7" Jul 06 01:26:48.455485 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.454547 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/925f85db-233c-4a3f-9d1c-f8238bce7e4c-host\") pod \"tuned-2pmf7\" (UID: \"925f85db-233c-4a3f-9d1c-f8238bce7e4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2pmf7" Jul 06 01:26:48.456154 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.454562 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ee8fc4f9-9bc1-4b56-a435-f158b76784fb-multus-daemon-config\") pod \"multus-xszvt\" (UID: \"ee8fc4f9-9bc1-4b56-a435-f158b76784fb\") " pod="openshift-multus/multus-xszvt" Jul 06 01:26:48.456154 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.454571 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c476bace-97d0-4492-adca-efd95306c6de-registration-dir\") pod \"aws-ebs-csi-driver-node-w9gm9\" (UID: \"c476bace-97d0-4492-adca-efd95306c6de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9gm9" Jul 06 01:26:48.456154 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.454599 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqphd\" (UniqueName: \"kubernetes.io/projected/c476bace-97d0-4492-adca-efd95306c6de-kube-api-access-jqphd\") pod \"aws-ebs-csi-driver-node-w9gm9\" (UID: \"c476bace-97d0-4492-adca-efd95306c6de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9gm9" Jul 06 01:26:48.456154 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.454637 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e690e9f2-b314-4e44-a211-433d7e8100a1-dbus\") pod \"global-pull-secret-syncer-xf5mp\" (UID: \"e690e9f2-b314-4e44-a211-433d7e8100a1\") " pod="kube-system/global-pull-secret-syncer-xf5mp" Jul 06 01:26:48.456154 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.454684 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ee8fc4f9-9bc1-4b56-a435-f158b76784fb-hostroot\") pod \"multus-xszvt\" (UID: \"ee8fc4f9-9bc1-4b56-a435-f158b76784fb\") " pod="openshift-multus/multus-xszvt" Jul 06 01:26:48.456154 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.454710 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ee8fc4f9-9bc1-4b56-a435-f158b76784fb-multus-cni-dir\") pod \"multus-xszvt\" (UID: \"ee8fc4f9-9bc1-4b56-a435-f158b76784fb\") " pod="openshift-multus/multus-xszvt" Jul 06 01:26:48.456154 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.454724 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ee8fc4f9-9bc1-4b56-a435-f158b76784fb-os-release\") pod \"multus-xszvt\" (UID: \"ee8fc4f9-9bc1-4b56-a435-f158b76784fb\") " pod="openshift-multus/multus-xszvt" Jul 06 01:26:48.456154 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.454746 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-host-run-netns\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.456154 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.454756 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/00a6f863-8ae0-4749-8f27-3bdf91cd8775-hosts-file\") pod \"node-resolver-82mwf\" (UID: \"00a6f863-8ae0-4749-8f27-3bdf91cd8775\") " pod="openshift-dns/node-resolver-82mwf" Jul 06 01:26:48.456154 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.454774 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/86824090-3c80-459c-97b6-4e34bcac19c3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bdpq8\" (UID: \"86824090-3c80-459c-97b6-4e34bcac19c3\") " pod="openshift-multus/multus-additional-cni-plugins-bdpq8" Jul 06 01:26:48.456154 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.454780 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ee8fc4f9-9bc1-4b56-a435-f158b76784fb-host-var-lib-cni-multus\") pod \"multus-xszvt\" (UID: \"ee8fc4f9-9bc1-4b56-a435-f158b76784fb\") " pod="openshift-multus/multus-xszvt" Jul 06 01:26:48.456154 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.454809 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/925f85db-233c-4a3f-9d1c-f8238bce7e4c-etc-sysctl-conf\") pod \"tuned-2pmf7\" (UID: \"925f85db-233c-4a3f-9d1c-f8238bce7e4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2pmf7" Jul 06 01:26:48.458655 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.458607 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-44.ec2.internal" event={"ID":"fa04038e4e7372411052bdcfb35944fb","Type":"ContainerStarted","Data":"ca883c476f17f1ad91fd982241137542547312a01f36100199858cbdd449912d"} Jul 06 01:26:48.459827 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.459799 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-44.ec2.internal" event={"ID":"5c47210a4f94f2c956e2eb14609b8c29","Type":"ContainerStarted","Data":"e61a2915438fb91525d89eaf9bd3f665b31a846cd2ccc6890ffc4bc40928d761"} Jul 06 01:26:48.463805 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.463784 2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 06 01:26:48.467764 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:48.467739 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 01:26:48.467920 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:48.467796 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 01:26:48.467920 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:48.467814 2580 projected.go:196] Error preparing data for projected volume kube-api-access-tdkgm for pod openshift-network-diagnostics/network-check-target-57drk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 01:26:48.467920 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:48.467890 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ab026023-6959-46fd-aaf6-0b145d5a956e-kube-api-access-tdkgm podName:ab026023-6959-46fd-aaf6-0b145d5a956e nodeName:}" failed. No retries permitted until 2026-07-06 01:26:48.967872614 +0000 UTC m=+2.939124298 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-tdkgm" (UniqueName: "kubernetes.io/projected/ab026023-6959-46fd-aaf6-0b145d5a956e-kube-api-access-tdkgm") pod "network-check-target-57drk" (UID: "ab026023-6959-46fd-aaf6-0b145d5a956e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 01:26:48.468104 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.467956 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2j5fp\" (UniqueName: \"kubernetes.io/projected/4164d767-f4d0-459d-b0c1-cc8f8ccbaf1a-kube-api-access-2j5fp\") pod \"node-ca-b46kx\" (UID: \"4164d767-f4d0-459d-b0c1-cc8f8ccbaf1a\") " pod="openshift-image-registry/node-ca-b46kx" Jul 06 01:26:48.469521 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.469424 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjs7p\" (UniqueName: \"kubernetes.io/projected/ee8fc4f9-9bc1-4b56-a435-f158b76784fb-kube-api-access-mjs7p\") pod \"multus-xszvt\" (UID: \"ee8fc4f9-9bc1-4b56-a435-f158b76784fb\") " pod="openshift-multus/multus-xszvt" Jul 06 01:26:48.470930 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.470906 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9rqh\" (UniqueName: \"kubernetes.io/projected/15c8ffd6-1e00-4975-b807-63e00ad36b44-kube-api-access-x9rqh\") pod \"network-metrics-daemon-qnp65\" (UID: \"15c8ffd6-1e00-4975-b807-63e00ad36b44\") " pod="openshift-multus/network-metrics-daemon-qnp65" Jul 06 01:26:48.471337 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.471314 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rmnr\" (UniqueName: \"kubernetes.io/projected/00a6f863-8ae0-4749-8f27-3bdf91cd8775-kube-api-access-7rmnr\") pod \"node-resolver-82mwf\" (UID: \"00a6f863-8ae0-4749-8f27-3bdf91cd8775\") " pod="openshift-dns/node-resolver-82mwf" Jul 06 01:26:48.555683 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.555648 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-log-socket\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.555871 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.555692 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/925f85db-233c-4a3f-9d1c-f8238bce7e4c-etc-kubernetes\") pod \"tuned-2pmf7\" (UID: \"925f85db-233c-4a3f-9d1c-f8238bce7e4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2pmf7" Jul 06 01:26:48.555871 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.555767 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-log-socket\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.555871 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.555792 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/925f85db-233c-4a3f-9d1c-f8238bce7e4c-lib-modules\") pod \"tuned-2pmf7\" (UID: \"925f85db-233c-4a3f-9d1c-f8238bce7e4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2pmf7" Jul 06 01:26:48.555871 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.555775 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/925f85db-233c-4a3f-9d1c-f8238bce7e4c-etc-kubernetes\") pod \"tuned-2pmf7\" (UID: \"925f85db-233c-4a3f-9d1c-f8238bce7e4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2pmf7" Jul 06 01:26:48.555871 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.555834 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c476bace-97d0-4492-adca-efd95306c6de-sys-fs\") pod \"aws-ebs-csi-driver-node-w9gm9\" (UID: \"c476bace-97d0-4492-adca-efd95306c6de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9gm9" Jul 06 01:26:48.555871 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.555874 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/925f85db-233c-4a3f-9d1c-f8238bce7e4c-var-lib-kubelet\") pod \"tuned-2pmf7\" (UID: \"925f85db-233c-4a3f-9d1c-f8238bce7e4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2pmf7" Jul 06 01:26:48.556166 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.555898 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2jz6\" (UniqueName: \"kubernetes.io/projected/925f85db-233c-4a3f-9d1c-f8238bce7e4c-kube-api-access-x2jz6\") pod \"tuned-2pmf7\" (UID: \"925f85db-233c-4a3f-9d1c-f8238bce7e4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2pmf7" Jul 06 01:26:48.556166 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.555917 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/925f85db-233c-4a3f-9d1c-f8238bce7e4c-lib-modules\") pod \"tuned-2pmf7\" (UID: \"925f85db-233c-4a3f-9d1c-f8238bce7e4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2pmf7" Jul 06 01:26:48.556166 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.555930 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c476bace-97d0-4492-adca-efd95306c6de-kubelet-dir\") pod \"aws-ebs-csi-driver-node-w9gm9\" (UID: \"c476bace-97d0-4492-adca-efd95306c6de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9gm9" Jul 06 01:26:48.556166 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.555956 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-run-openvswitch\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.556166 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.555981 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c476bace-97d0-4492-adca-efd95306c6de-sys-fs\") pod \"aws-ebs-csi-driver-node-w9gm9\" (UID: \"c476bace-97d0-4492-adca-efd95306c6de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9gm9" Jul 06 01:26:48.556166 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.555980 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-run-systemd\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.556166 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.555991 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/925f85db-233c-4a3f-9d1c-f8238bce7e4c-var-lib-kubelet\") pod \"tuned-2pmf7\" (UID: \"925f85db-233c-4a3f-9d1c-f8238bce7e4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2pmf7" Jul 06 01:26:48.556166 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.556037 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-run-systemd\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.556166 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.556049 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-run-openvswitch\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.556166 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.556037 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-node-log\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.556166 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.556061 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c476bace-97d0-4492-adca-efd95306c6de-kubelet-dir\") pod \"aws-ebs-csi-driver-node-w9gm9\" (UID: \"c476bace-97d0-4492-adca-efd95306c6de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9gm9" Jul 06 01:26:48.556166 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.556070 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-node-log\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.556166 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.556088 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e43deb0d-17a0-4c43-8120-3edd3732068a-host-slash\") pod \"iptables-alerter-qmxp5\" (UID: \"e43deb0d-17a0-4c43-8120-3edd3732068a\") " pod="openshift-network-operator/iptables-alerter-qmxp5" Jul 06 01:26:48.556166 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.556115 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/925f85db-233c-4a3f-9d1c-f8238bce7e4c-tmp\") pod \"tuned-2pmf7\" (UID: \"925f85db-233c-4a3f-9d1c-f8238bce7e4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2pmf7" Jul 06 01:26:48.556166 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.556129 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-host-kubelet\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.556166 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.556157 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e43deb0d-17a0-4c43-8120-3edd3732068a-host-slash\") pod \"iptables-alerter-qmxp5\" (UID: \"e43deb0d-17a0-4c43-8120-3edd3732068a\") " pod="openshift-network-operator/iptables-alerter-qmxp5" Jul 06 01:26:48.556166 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.556173 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-host-kubelet\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.557128 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.556195 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-host-cni-netd\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.557128 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.556220 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cb081505-8ac0-48a3-be9e-bf770a830127-agent-certs\") pod \"konnectivity-agent-cksd7\" (UID: \"cb081505-8ac0-48a3-be9e-bf770a830127\") " pod="kube-system/konnectivity-agent-cksd7" Jul 06 01:26:48.557128 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.556241 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/925f85db-233c-4a3f-9d1c-f8238bce7e4c-host\") pod \"tuned-2pmf7\" (UID: \"925f85db-233c-4a3f-9d1c-f8238bce7e4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2pmf7" Jul 06 01:26:48.557128 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.556264 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c476bace-97d0-4492-adca-efd95306c6de-registration-dir\") pod \"aws-ebs-csi-driver-node-w9gm9\" (UID: \"c476bace-97d0-4492-adca-efd95306c6de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9gm9" Jul 06 01:26:48.557128 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.556276 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-host-cni-netd\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.557128 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.556289 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqphd\" (UniqueName: \"kubernetes.io/projected/c476bace-97d0-4492-adca-efd95306c6de-kube-api-access-jqphd\") pod \"aws-ebs-csi-driver-node-w9gm9\" (UID: \"c476bace-97d0-4492-adca-efd95306c6de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9gm9" Jul 06 01:26:48.557128 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.556316 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-host-run-netns\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.557128 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.556322 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/925f85db-233c-4a3f-9d1c-f8238bce7e4c-host\") pod \"tuned-2pmf7\" (UID: \"925f85db-233c-4a3f-9d1c-f8238bce7e4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2pmf7" Jul 06 01:26:48.557128 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.556342 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/86824090-3c80-459c-97b6-4e34bcac19c3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bdpq8\" (UID: \"86824090-3c80-459c-97b6-4e34bcac19c3\") " pod="openshift-multus/multus-additional-cni-plugins-bdpq8" Jul 06 01:26:48.557128 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.556356 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c476bace-97d0-4492-adca-efd95306c6de-registration-dir\") pod \"aws-ebs-csi-driver-node-w9gm9\" (UID: \"c476bace-97d0-4492-adca-efd95306c6de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9gm9" Jul 06 01:26:48.557128 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.556367 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/925f85db-233c-4a3f-9d1c-f8238bce7e4c-etc-sysctl-conf\") pod \"tuned-2pmf7\" (UID: \"925f85db-233c-4a3f-9d1c-f8238bce7e4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2pmf7" Jul 06 01:26:48.557128 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.556394 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-host-run-netns\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.557128 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.556403 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/925f85db-233c-4a3f-9d1c-f8238bce7e4c-run\") pod \"tuned-2pmf7\" (UID: \"925f85db-233c-4a3f-9d1c-f8238bce7e4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2pmf7" Jul 06 01:26:48.557128 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.556425 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/925f85db-233c-4a3f-9d1c-f8238bce7e4c-sys\") pod \"tuned-2pmf7\" (UID: \"925f85db-233c-4a3f-9d1c-f8238bce7e4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2pmf7" Jul 06 01:26:48.557128 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.556465 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-host-slash\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.557128 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.556487 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-etc-openvswitch\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.557128 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.556509 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-host-cni-bin\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.557944 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.556538 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-ovn-node-metrics-cert\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.557944 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.556561 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9fd5\" (UniqueName: \"kubernetes.io/projected/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-kube-api-access-x9fd5\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.557944 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.556569 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/925f85db-233c-4a3f-9d1c-f8238bce7e4c-etc-sysctl-conf\") pod \"tuned-2pmf7\" (UID: \"925f85db-233c-4a3f-9d1c-f8238bce7e4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2pmf7" Jul 06 01:26:48.557944 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.556579 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c476bace-97d0-4492-adca-efd95306c6de-etc-selinux\") pod \"aws-ebs-csi-driver-node-w9gm9\" (UID: \"c476bace-97d0-4492-adca-efd95306c6de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9gm9" Jul 06 01:26:48.557944 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.556617 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/925f85db-233c-4a3f-9d1c-f8238bce7e4c-etc-systemd\") pod \"tuned-2pmf7\" (UID: \"925f85db-233c-4a3f-9d1c-f8238bce7e4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2pmf7" Jul 06 01:26:48.557944 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.556644 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-run-ovn\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.557944 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.556649 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c476bace-97d0-4492-adca-efd95306c6de-etc-selinux\") pod \"aws-ebs-csi-driver-node-w9gm9\" (UID: \"c476bace-97d0-4492-adca-efd95306c6de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9gm9" Jul 06 01:26:48.557944 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.556666 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-env-overrides\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.557944 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.556691 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-systemd-units\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.557944 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.556714 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-var-lib-openvswitch\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.557944 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.556721 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/925f85db-233c-4a3f-9d1c-f8238bce7e4c-run\") pod \"tuned-2pmf7\" (UID: \"925f85db-233c-4a3f-9d1c-f8238bce7e4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2pmf7" Jul 06 01:26:48.557944 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.556741 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.557944 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.556751 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/925f85db-233c-4a3f-9d1c-f8238bce7e4c-sys\") pod \"tuned-2pmf7\" (UID: \"925f85db-233c-4a3f-9d1c-f8238bce7e4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2pmf7" Jul 06 01:26:48.557944 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.556771 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/86824090-3c80-459c-97b6-4e34bcac19c3-system-cni-dir\") pod \"multus-additional-cni-plugins-bdpq8\" (UID: \"86824090-3c80-459c-97b6-4e34bcac19c3\") " pod="openshift-multus/multus-additional-cni-plugins-bdpq8" Jul 06 01:26:48.557944 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.556776 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/86824090-3c80-459c-97b6-4e34bcac19c3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bdpq8\" (UID: \"86824090-3c80-459c-97b6-4e34bcac19c3\") " pod="openshift-multus/multus-additional-cni-plugins-bdpq8" Jul 06 01:26:48.557944 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.556791 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-etc-openvswitch\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.557944 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.556798 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/86824090-3c80-459c-97b6-4e34bcac19c3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bdpq8\" (UID: \"86824090-3c80-459c-97b6-4e34bcac19c3\") " pod="openshift-multus/multus-additional-cni-plugins-bdpq8" Jul 06 01:26:48.558798 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.556826 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c476bace-97d0-4492-adca-efd95306c6de-device-dir\") pod \"aws-ebs-csi-driver-node-w9gm9\" (UID: \"c476bace-97d0-4492-adca-efd95306c6de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9gm9" Jul 06 01:26:48.558798 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.556831 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-host-cni-bin\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.558798 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.556858 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/86824090-3c80-459c-97b6-4e34bcac19c3-cnibin\") pod \"multus-additional-cni-plugins-bdpq8\" (UID: \"86824090-3c80-459c-97b6-4e34bcac19c3\") " pod="openshift-multus/multus-additional-cni-plugins-bdpq8" Jul 06 01:26:48.558798 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.556878 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c476bace-97d0-4492-adca-efd95306c6de-device-dir\") pod \"aws-ebs-csi-driver-node-w9gm9\" (UID: \"c476bace-97d0-4492-adca-efd95306c6de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9gm9" Jul 06 01:26:48.558798 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.556887 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/86824090-3c80-459c-97b6-4e34bcac19c3-os-release\") pod \"multus-additional-cni-plugins-bdpq8\" (UID: \"86824090-3c80-459c-97b6-4e34bcac19c3\") " pod="openshift-multus/multus-additional-cni-plugins-bdpq8" Jul 06 01:26:48.558798 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.556914 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-host-run-ovn-kubernetes\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.558798 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.556920 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/925f85db-233c-4a3f-9d1c-f8238bce7e4c-etc-systemd\") pod \"tuned-2pmf7\" (UID: \"925f85db-233c-4a3f-9d1c-f8238bce7e4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2pmf7" Jul 06 01:26:48.558798 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.556949 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/925f85db-233c-4a3f-9d1c-f8238bce7e4c-etc-sysctl-d\") pod \"tuned-2pmf7\" (UID: \"925f85db-233c-4a3f-9d1c-f8238bce7e4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2pmf7" Jul 06 01:26:48.558798 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.556973 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-run-ovn\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.558798 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.556974 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c476bace-97d0-4492-adca-efd95306c6de-socket-dir\") pod \"aws-ebs-csi-driver-node-w9gm9\" (UID: \"c476bace-97d0-4492-adca-efd95306c6de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9gm9" Jul 06 01:26:48.558798 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.557003 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-ovnkube-script-lib\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.558798 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.557043 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cb081505-8ac0-48a3-be9e-bf770a830127-konnectivity-ca\") pod \"konnectivity-agent-cksd7\" (UID: \"cb081505-8ac0-48a3-be9e-bf770a830127\") " pod="kube-system/konnectivity-agent-cksd7" Jul 06 01:26:48.558798 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.557053 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c476bace-97d0-4492-adca-efd95306c6de-socket-dir\") pod \"aws-ebs-csi-driver-node-w9gm9\" (UID: \"c476bace-97d0-4492-adca-efd95306c6de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9gm9" Jul 06 01:26:48.558798 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.557069 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e43deb0d-17a0-4c43-8120-3edd3732068a-iptables-alerter-script\") pod \"iptables-alerter-qmxp5\" (UID: \"e43deb0d-17a0-4c43-8120-3edd3732068a\") " pod="openshift-network-operator/iptables-alerter-qmxp5" Jul 06 01:26:48.558798 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.557095 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/925f85db-233c-4a3f-9d1c-f8238bce7e4c-etc-sysconfig\") pod \"tuned-2pmf7\" (UID: \"925f85db-233c-4a3f-9d1c-f8238bce7e4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2pmf7" Jul 06 01:26:48.558798 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.557118 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-ovnkube-config\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.558798 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.557143 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/86824090-3c80-459c-97b6-4e34bcac19c3-os-release\") pod \"multus-additional-cni-plugins-bdpq8\" (UID: \"86824090-3c80-459c-97b6-4e34bcac19c3\") " pod="openshift-multus/multus-additional-cni-plugins-bdpq8" Jul 06 01:26:48.559548 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.557144 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2z4ms\" (UniqueName: \"kubernetes.io/projected/86824090-3c80-459c-97b6-4e34bcac19c3-kube-api-access-2z4ms\") pod \"multus-additional-cni-plugins-bdpq8\" (UID: \"86824090-3c80-459c-97b6-4e34bcac19c3\") " pod="openshift-multus/multus-additional-cni-plugins-bdpq8" Jul 06 01:26:48.559548 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.557180 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-host-run-ovn-kubernetes\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.559548 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.557189 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/925f85db-233c-4a3f-9d1c-f8238bce7e4c-etc-modprobe-d\") pod \"tuned-2pmf7\" (UID: \"925f85db-233c-4a3f-9d1c-f8238bce7e4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2pmf7" Jul 06 01:26:48.559548 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.557213 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/86824090-3c80-459c-97b6-4e34bcac19c3-cni-binary-copy\") pod \"multus-additional-cni-plugins-bdpq8\" (UID: \"86824090-3c80-459c-97b6-4e34bcac19c3\") " pod="openshift-multus/multus-additional-cni-plugins-bdpq8" Jul 06 01:26:48.559548 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.557237 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/925f85db-233c-4a3f-9d1c-f8238bce7e4c-etc-tuned\") pod \"tuned-2pmf7\" (UID: \"925f85db-233c-4a3f-9d1c-f8238bce7e4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2pmf7" Jul 06 01:26:48.559548 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.557262 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/86824090-3c80-459c-97b6-4e34bcac19c3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bdpq8\" (UID: \"86824090-3c80-459c-97b6-4e34bcac19c3\") " pod="openshift-multus/multus-additional-cni-plugins-bdpq8" Jul 06 01:26:48.559548 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.557269 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/925f85db-233c-4a3f-9d1c-f8238bce7e4c-etc-sysctl-d\") pod \"tuned-2pmf7\" (UID: \"925f85db-233c-4a3f-9d1c-f8238bce7e4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2pmf7" Jul 06 01:26:48.559548 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.557287 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2f6t7\" (UniqueName: \"kubernetes.io/projected/e43deb0d-17a0-4c43-8120-3edd3732068a-kube-api-access-2f6t7\") pod \"iptables-alerter-qmxp5\" (UID: \"e43deb0d-17a0-4c43-8120-3edd3732068a\") " pod="openshift-network-operator/iptables-alerter-qmxp5" Jul 06 01:26:48.559548 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.557803 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cb081505-8ac0-48a3-be9e-bf770a830127-konnectivity-ca\") pod \"konnectivity-agent-cksd7\" (UID: \"cb081505-8ac0-48a3-be9e-bf770a830127\") " pod="kube-system/konnectivity-agent-cksd7" Jul 06 01:26:48.559548 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.558071 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-env-overrides\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.559548 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.558106 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-systemd-units\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.559548 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.558132 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-var-lib-openvswitch\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.559548 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.558156 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.559548 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.558181 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/86824090-3c80-459c-97b6-4e34bcac19c3-system-cni-dir\") pod \"multus-additional-cni-plugins-bdpq8\" (UID: \"86824090-3c80-459c-97b6-4e34bcac19c3\") " pod="openshift-multus/multus-additional-cni-plugins-bdpq8" Jul 06 01:26:48.559548 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.556774 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-host-slash\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.559548 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.558550 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/925f85db-233c-4a3f-9d1c-f8238bce7e4c-etc-sysconfig\") pod \"tuned-2pmf7\" (UID: \"925f85db-233c-4a3f-9d1c-f8238bce7e4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2pmf7" Jul 06 01:26:48.559548 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.557096 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/86824090-3c80-459c-97b6-4e34bcac19c3-cnibin\") pod \"multus-additional-cni-plugins-bdpq8\" (UID: \"86824090-3c80-459c-97b6-4e34bcac19c3\") " pod="openshift-multus/multus-additional-cni-plugins-bdpq8" Jul 06 01:26:48.560215 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.558683 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/86824090-3c80-459c-97b6-4e34bcac19c3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bdpq8\" (UID: \"86824090-3c80-459c-97b6-4e34bcac19c3\") " pod="openshift-multus/multus-additional-cni-plugins-bdpq8" Jul 06 01:26:48.560215 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.558704 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e43deb0d-17a0-4c43-8120-3edd3732068a-iptables-alerter-script\") pod \"iptables-alerter-qmxp5\" (UID: \"e43deb0d-17a0-4c43-8120-3edd3732068a\") " pod="openshift-network-operator/iptables-alerter-qmxp5" Jul 06 01:26:48.560215 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.558902 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/925f85db-233c-4a3f-9d1c-f8238bce7e4c-tmp\") pod \"tuned-2pmf7\" (UID: \"925f85db-233c-4a3f-9d1c-f8238bce7e4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2pmf7" Jul 06 01:26:48.560215 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.559021 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/925f85db-233c-4a3f-9d1c-f8238bce7e4c-etc-modprobe-d\") pod \"tuned-2pmf7\" (UID: \"925f85db-233c-4a3f-9d1c-f8238bce7e4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2pmf7" Jul 06 01:26:48.560215 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.559072 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-ovnkube-script-lib\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.560215 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.559088 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-ovnkube-config\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.560215 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.559160 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/86824090-3c80-459c-97b6-4e34bcac19c3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bdpq8\" (UID: \"86824090-3c80-459c-97b6-4e34bcac19c3\") " pod="openshift-multus/multus-additional-cni-plugins-bdpq8" Jul 06 01:26:48.561083 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.561047 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/925f85db-233c-4a3f-9d1c-f8238bce7e4c-etc-tuned\") pod \"tuned-2pmf7\" (UID: \"925f85db-233c-4a3f-9d1c-f8238bce7e4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2pmf7" Jul 06 01:26:48.561336 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.561314 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-ovn-node-metrics-cert\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.561797 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.561774 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cb081505-8ac0-48a3-be9e-bf770a830127-agent-certs\") pod \"konnectivity-agent-cksd7\" (UID: \"cb081505-8ac0-48a3-be9e-bf770a830127\") " pod="kube-system/konnectivity-agent-cksd7" Jul 06 01:26:48.562916 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.562895 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/86824090-3c80-459c-97b6-4e34bcac19c3-cni-binary-copy\") pod \"multus-additional-cni-plugins-bdpq8\" (UID: \"86824090-3c80-459c-97b6-4e34bcac19c3\") " pod="openshift-multus/multus-additional-cni-plugins-bdpq8" Jul 06 01:26:48.568426 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.568400 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2z4ms\" (UniqueName: \"kubernetes.io/projected/86824090-3c80-459c-97b6-4e34bcac19c3-kube-api-access-2z4ms\") pod \"multus-additional-cni-plugins-bdpq8\" (UID: \"86824090-3c80-459c-97b6-4e34bcac19c3\") " pod="openshift-multus/multus-additional-cni-plugins-bdpq8" Jul 06 01:26:48.568545 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.568473 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2jz6\" (UniqueName: \"kubernetes.io/projected/925f85db-233c-4a3f-9d1c-f8238bce7e4c-kube-api-access-x2jz6\") pod \"tuned-2pmf7\" (UID: \"925f85db-233c-4a3f-9d1c-f8238bce7e4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2pmf7" Jul 06 01:26:48.568653 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.568632 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9fd5\" (UniqueName: \"kubernetes.io/projected/04c8a1d0-bd84-4b0c-934b-efe85acfa81d-kube-api-access-x9fd5\") pod \"ovnkube-node-2sd2m\" (UID: \"04c8a1d0-bd84-4b0c-934b-efe85acfa81d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.569095 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.569073 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqphd\" (UniqueName: \"kubernetes.io/projected/c476bace-97d0-4492-adca-efd95306c6de-kube-api-access-jqphd\") pod \"aws-ebs-csi-driver-node-w9gm9\" (UID: \"c476bace-97d0-4492-adca-efd95306c6de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9gm9" Jul 06 01:26:48.572160 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.572099 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2f6t7\" (UniqueName: \"kubernetes.io/projected/e43deb0d-17a0-4c43-8120-3edd3732068a-kube-api-access-2f6t7\") pod \"iptables-alerter-qmxp5\" (UID: \"e43deb0d-17a0-4c43-8120-3edd3732068a\") " pod="openshift-network-operator/iptables-alerter-qmxp5" Jul 06 01:26:48.650736 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.650698 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-82mwf" Jul 06 01:26:48.659146 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.659129 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-b46kx" Jul 06 01:26:48.670229 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.670202 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-xszvt" Jul 06 01:26:48.676027 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.676007 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bdpq8" Jul 06 01:26:48.683597 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.683575 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-cksd7" Jul 06 01:26:48.694185 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.694162 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2pmf7" Jul 06 01:26:48.699031 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.699009 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9gm9" Jul 06 01:26:48.706758 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.706727 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qmxp5" Jul 06 01:26:48.711476 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.711457 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:26:48.775099 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.775069 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 01:26:48.959174 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.959143 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15c8ffd6-1e00-4975-b807-63e00ad36b44-metrics-certs\") pod \"network-metrics-daemon-qnp65\" (UID: \"15c8ffd6-1e00-4975-b807-63e00ad36b44\") " pod="openshift-multus/network-metrics-daemon-qnp65" Jul 06 01:26:48.959338 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:48.959181 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e690e9f2-b314-4e44-a211-433d7e8100a1-original-pull-secret\") pod \"global-pull-secret-syncer-xf5mp\" (UID: \"e690e9f2-b314-4e44-a211-433d7e8100a1\") " pod="kube-system/global-pull-secret-syncer-xf5mp" Jul 06 01:26:48.959338 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:48.959224 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 01:26:48.959338 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:48.959268 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 01:26:48.959338 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:48.959275 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15c8ffd6-1e00-4975-b807-63e00ad36b44-metrics-certs podName:15c8ffd6-1e00-4975-b807-63e00ad36b44 nodeName:}" failed. No retries permitted until 2026-07-06 01:26:49.959261206 +0000 UTC m=+3.930512874 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/15c8ffd6-1e00-4975-b807-63e00ad36b44-metrics-certs") pod "network-metrics-daemon-qnp65" (UID: "15c8ffd6-1e00-4975-b807-63e00ad36b44") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 01:26:48.959338 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:48.959302 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e690e9f2-b314-4e44-a211-433d7e8100a1-original-pull-secret podName:e690e9f2-b314-4e44-a211-433d7e8100a1 nodeName:}" failed. No retries permitted until 2026-07-06 01:26:49.959292735 +0000 UTC m=+3.930544404 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e690e9f2-b314-4e44-a211-433d7e8100a1-original-pull-secret") pod "global-pull-secret-syncer-xf5mp" (UID: "e690e9f2-b314-4e44-a211-433d7e8100a1") : object "kube-system"/"original-pull-secret" not registered Jul 06 01:26:49.060242 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:49.060193 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdkgm\" (UniqueName: \"kubernetes.io/projected/ab026023-6959-46fd-aaf6-0b145d5a956e-kube-api-access-tdkgm\") pod \"network-check-target-57drk\" (UID: \"ab026023-6959-46fd-aaf6-0b145d5a956e\") " pod="openshift-network-diagnostics/network-check-target-57drk" Jul 06 01:26:49.060426 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:49.060357 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 01:26:49.060426 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:49.060380 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 01:26:49.060426 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:49.060392 2580 projected.go:196] Error preparing data for projected volume kube-api-access-tdkgm for pod openshift-network-diagnostics/network-check-target-57drk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 01:26:49.060597 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:49.060473 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ab026023-6959-46fd-aaf6-0b145d5a956e-kube-api-access-tdkgm podName:ab026023-6959-46fd-aaf6-0b145d5a956e nodeName:}" failed. No retries permitted until 2026-07-06 01:26:50.060454573 +0000 UTC m=+4.031706261 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-tdkgm" (UniqueName: "kubernetes.io/projected/ab026023-6959-46fd-aaf6-0b145d5a956e-kube-api-access-tdkgm") pod "network-check-target-57drk" (UID: "ab026023-6959-46fd-aaf6-0b145d5a956e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 01:26:49.376267 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:49.376229 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode43deb0d_17a0_4c43_8120_3edd3732068a.slice/crio-d048a303438e9f55862f8cb0b81b8b9fc29b903c6ef48b4c38593e633fe80d1e WatchSource:0}: Error finding container d048a303438e9f55862f8cb0b81b8b9fc29b903c6ef48b4c38593e633fe80d1e: Status 404 returned error can't find the container with id d048a303438e9f55862f8cb0b81b8b9fc29b903c6ef48b4c38593e633fe80d1e Jul 06 01:26:49.377785 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:49.377448 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod925f85db_233c_4a3f_9d1c_f8238bce7e4c.slice/crio-aa5ef04e6e7566b367530de22a79da3ef4dcd19a81ec2812a959eefff0f8fbf9 WatchSource:0}: Error finding container aa5ef04e6e7566b367530de22a79da3ef4dcd19a81ec2812a959eefff0f8fbf9: Status 404 returned error can't find the container with id aa5ef04e6e7566b367530de22a79da3ef4dcd19a81ec2812a959eefff0f8fbf9 Jul 06 01:26:49.378853 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:49.378602 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee8fc4f9_9bc1_4b56_a435_f158b76784fb.slice/crio-72605e48f58830ac67528c394edb99eec1ca0a6dc80044a150d82438513edc89 WatchSource:0}: Error finding container 72605e48f58830ac67528c394edb99eec1ca0a6dc80044a150d82438513edc89: Status 404 returned error can't find the container with id 72605e48f58830ac67528c394edb99eec1ca0a6dc80044a150d82438513edc89 Jul 06 01:26:49.379383 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:49.379356 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4164d767_f4d0_459d_b0c1_cc8f8ccbaf1a.slice/crio-3aa2178e43abb6911725b799f04445cb69604682db0d3f85542952b2f68c6efc WatchSource:0}: Error finding container 3aa2178e43abb6911725b799f04445cb69604682db0d3f85542952b2f68c6efc: Status 404 returned error can't find the container with id 3aa2178e43abb6911725b799f04445cb69604682db0d3f85542952b2f68c6efc Jul 06 01:26:49.385755 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:49.385733 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04c8a1d0_bd84_4b0c_934b_efe85acfa81d.slice/crio-bb901cf627a508b721156160b5dbd11480b3060689ec49eba0e17fe8378ef096 WatchSource:0}: Error finding container bb901cf627a508b721156160b5dbd11480b3060689ec49eba0e17fe8378ef096: Status 404 returned error can't find the container with id bb901cf627a508b721156160b5dbd11480b3060689ec49eba0e17fe8378ef096 Jul 06 01:26:49.387846 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:49.387821 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86824090_3c80_459c_97b6_4e34bcac19c3.slice/crio-9f63cbc04b1c85341ed65876236e1c65fa253f6a253eb2edb4bee0b62056aa06 WatchSource:0}: Error finding container 9f63cbc04b1c85341ed65876236e1c65fa253f6a253eb2edb4bee0b62056aa06: Status 404 returned error can't find the container with id 9f63cbc04b1c85341ed65876236e1c65fa253f6a253eb2edb4bee0b62056aa06 Jul 06 01:26:49.389608 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:49.389575 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb081505_8ac0_48a3_be9e_bf770a830127.slice/crio-dce72d4576463893876f2bda10068bdcf315778f45d6c0266c3e3d85ab308e95 WatchSource:0}: Error finding container dce72d4576463893876f2bda10068bdcf315778f45d6c0266c3e3d85ab308e95: Status 404 returned error can't find the container with id dce72d4576463893876f2bda10068bdcf315778f45d6c0266c3e3d85ab308e95 Jul 06 01:26:49.390542 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:49.390513 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00a6f863_8ae0_4749_8f27_3bdf91cd8775.slice/crio-7378d9d0133331d163bb7a58b40a8bbc736ef2a78caae9d85c6a9def51cc7c4b WatchSource:0}: Error finding container 7378d9d0133331d163bb7a58b40a8bbc736ef2a78caae9d85c6a9def51cc7c4b: Status 404 returned error can't find the container with id 7378d9d0133331d163bb7a58b40a8bbc736ef2a78caae9d85c6a9def51cc7c4b Jul 06 01:26:49.392894 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:49.391828 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 01:21:48 +0000 UTC" deadline="2028-02-02 10:54:49.875941935 +0000 UTC" Jul 06 01:26:49.392894 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:49.391870 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13833h28m0.484075178s" Jul 06 01:26:49.392894 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:26:49.391954 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc476bace_97d0_4492_adca_efd95306c6de.slice/crio-39ccaf37953a8531cbf7bb235bd2ad6a7f271524fcd5fd4777aa9477221fe6fa WatchSource:0}: Error finding container 39ccaf37953a8531cbf7bb235bd2ad6a7f271524fcd5fd4777aa9477221fe6fa: Status 404 returned error can't find the container with id 39ccaf37953a8531cbf7bb235bd2ad6a7f271524fcd5fd4777aa9477221fe6fa Jul 06 01:26:49.454033 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:49.454010 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xf5mp" Jul 06 01:26:49.454128 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:49.454111 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xf5mp" podUID="e690e9f2-b314-4e44-a211-433d7e8100a1" Jul 06 01:26:49.462269 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:49.462238 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-82mwf" event={"ID":"00a6f863-8ae0-4749-8f27-3bdf91cd8775","Type":"ContainerStarted","Data":"7378d9d0133331d163bb7a58b40a8bbc736ef2a78caae9d85c6a9def51cc7c4b"} Jul 06 01:26:49.463340 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:49.463317 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-cksd7" event={"ID":"cb081505-8ac0-48a3-be9e-bf770a830127","Type":"ContainerStarted","Data":"dce72d4576463893876f2bda10068bdcf315778f45d6c0266c3e3d85ab308e95"} Jul 06 01:26:49.464831 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:49.464734 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bdpq8" event={"ID":"86824090-3c80-459c-97b6-4e34bcac19c3","Type":"ContainerStarted","Data":"9f63cbc04b1c85341ed65876236e1c65fa253f6a253eb2edb4bee0b62056aa06"} Jul 06 01:26:49.466022 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:49.465983 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qmxp5" event={"ID":"e43deb0d-17a0-4c43-8120-3edd3732068a","Type":"ContainerStarted","Data":"d048a303438e9f55862f8cb0b81b8b9fc29b903c6ef48b4c38593e633fe80d1e"} Jul 06 01:26:49.467182 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:49.467156 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9gm9" event={"ID":"c476bace-97d0-4492-adca-efd95306c6de","Type":"ContainerStarted","Data":"39ccaf37953a8531cbf7bb235bd2ad6a7f271524fcd5fd4777aa9477221fe6fa"} Jul 06 01:26:49.468474 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:49.468427 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" event={"ID":"04c8a1d0-bd84-4b0c-934b-efe85acfa81d","Type":"ContainerStarted","Data":"bb901cf627a508b721156160b5dbd11480b3060689ec49eba0e17fe8378ef096"} Jul 06 01:26:49.469703 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:49.469679 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2pmf7" event={"ID":"925f85db-233c-4a3f-9d1c-f8238bce7e4c","Type":"ContainerStarted","Data":"aa5ef04e6e7566b367530de22a79da3ef4dcd19a81ec2812a959eefff0f8fbf9"} Jul 06 01:26:49.470818 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:49.470794 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-b46kx" event={"ID":"4164d767-f4d0-459d-b0c1-cc8f8ccbaf1a","Type":"ContainerStarted","Data":"3aa2178e43abb6911725b799f04445cb69604682db0d3f85542952b2f68c6efc"} Jul 06 01:26:49.472158 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:49.472138 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-xszvt" event={"ID":"ee8fc4f9-9bc1-4b56-a435-f158b76784fb","Type":"ContainerStarted","Data":"72605e48f58830ac67528c394edb99eec1ca0a6dc80044a150d82438513edc89"} Jul 06 01:26:49.967987 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:49.967947 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15c8ffd6-1e00-4975-b807-63e00ad36b44-metrics-certs\") pod \"network-metrics-daemon-qnp65\" (UID: \"15c8ffd6-1e00-4975-b807-63e00ad36b44\") " pod="openshift-multus/network-metrics-daemon-qnp65" Jul 06 01:26:49.968182 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:49.968001 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e690e9f2-b314-4e44-a211-433d7e8100a1-original-pull-secret\") pod \"global-pull-secret-syncer-xf5mp\" (UID: \"e690e9f2-b314-4e44-a211-433d7e8100a1\") " pod="kube-system/global-pull-secret-syncer-xf5mp" Jul 06 01:26:49.968182 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:49.968174 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 01:26:49.968311 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:49.968239 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e690e9f2-b314-4e44-a211-433d7e8100a1-original-pull-secret podName:e690e9f2-b314-4e44-a211-433d7e8100a1 nodeName:}" failed. No retries permitted until 2026-07-06 01:26:51.968220341 +0000 UTC m=+5.939472011 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e690e9f2-b314-4e44-a211-433d7e8100a1-original-pull-secret") pod "global-pull-secret-syncer-xf5mp" (UID: "e690e9f2-b314-4e44-a211-433d7e8100a1") : object "kube-system"/"original-pull-secret" not registered Jul 06 01:26:49.968681 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:49.968659 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 01:26:49.968782 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:49.968715 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15c8ffd6-1e00-4975-b807-63e00ad36b44-metrics-certs podName:15c8ffd6-1e00-4975-b807-63e00ad36b44 nodeName:}" failed. No retries permitted until 2026-07-06 01:26:51.968700684 +0000 UTC m=+5.939952355 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/15c8ffd6-1e00-4975-b807-63e00ad36b44-metrics-certs") pod "network-metrics-daemon-qnp65" (UID: "15c8ffd6-1e00-4975-b807-63e00ad36b44") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 01:26:50.069029 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:50.068986 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdkgm\" (UniqueName: \"kubernetes.io/projected/ab026023-6959-46fd-aaf6-0b145d5a956e-kube-api-access-tdkgm\") pod \"network-check-target-57drk\" (UID: \"ab026023-6959-46fd-aaf6-0b145d5a956e\") " pod="openshift-network-diagnostics/network-check-target-57drk" Jul 06 01:26:50.069221 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:50.069167 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 01:26:50.069221 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:50.069190 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 01:26:50.069221 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:50.069204 2580 projected.go:196] Error preparing data for projected volume kube-api-access-tdkgm for pod openshift-network-diagnostics/network-check-target-57drk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 01:26:50.069380 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:50.069267 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ab026023-6959-46fd-aaf6-0b145d5a956e-kube-api-access-tdkgm podName:ab026023-6959-46fd-aaf6-0b145d5a956e nodeName:}" failed. No retries permitted until 2026-07-06 01:26:52.06924707 +0000 UTC m=+6.040498740 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-tdkgm" (UniqueName: "kubernetes.io/projected/ab026023-6959-46fd-aaf6-0b145d5a956e-kube-api-access-tdkgm") pod "network-check-target-57drk" (UID: "ab026023-6959-46fd-aaf6-0b145d5a956e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 01:26:50.392886 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:50.392841 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 01:21:48 +0000 UTC" deadline="2027-12-09 20:39:47.378280059 +0000 UTC" Jul 06 01:26:50.392886 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:50.392886 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12523h12m56.985398787s" Jul 06 01:26:50.456323 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:50.456076 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qnp65" Jul 06 01:26:50.456323 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:50.456225 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qnp65" podUID="15c8ffd6-1e00-4975-b807-63e00ad36b44" Jul 06 01:26:50.456646 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:50.456623 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57drk" Jul 06 01:26:50.457057 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:50.456718 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-57drk" podUID="ab026023-6959-46fd-aaf6-0b145d5a956e" Jul 06 01:26:50.497340 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:50.497178 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-44.ec2.internal" event={"ID":"fa04038e4e7372411052bdcfb35944fb","Type":"ContainerStarted","Data":"3284f692543c6919df57fa17e5fe270f9db578fc514e5002affadb6ad7dfa276"} Jul 06 01:26:50.526857 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:50.526815 2580 generic.go:356] "Generic (PLEG): container finished" podID="5c47210a4f94f2c956e2eb14609b8c29" containerID="b3f7300662ed98de5386de7f77e62305b30d3a6cb7b2736a05d22505b4b5c0e4" exitCode=0 Jul 06 01:26:50.527044 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:50.526875 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-44.ec2.internal" event={"ID":"5c47210a4f94f2c956e2eb14609b8c29","Type":"ContainerDied","Data":"b3f7300662ed98de5386de7f77e62305b30d3a6cb7b2736a05d22505b4b5c0e4"} Jul 06 01:26:50.547071 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:50.546269 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-44.ec2.internal" podStartSLOduration=3.546249793 podStartE2EDuration="3.546249793s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 01:26:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:26:50.519318703 +0000 UTC m=+4.490570396" watchObservedRunningTime="2026-07-06 01:26:50.546249793 +0000 UTC m=+4.517501485" Jul 06 01:26:51.453711 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:51.453677 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xf5mp" Jul 06 01:26:51.454253 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:51.453820 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xf5mp" podUID="e690e9f2-b314-4e44-a211-433d7e8100a1" Jul 06 01:26:51.550732 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:51.550686 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-44.ec2.internal" event={"ID":"5c47210a4f94f2c956e2eb14609b8c29","Type":"ContainerStarted","Data":"d8ea5a97f3fd5dc5e6cb1d1387f6ce7b4dba8c8387fbfc9b0b49ad3341c14fd8"} Jul 06 01:26:51.993478 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:51.993422 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15c8ffd6-1e00-4975-b807-63e00ad36b44-metrics-certs\") pod \"network-metrics-daemon-qnp65\" (UID: \"15c8ffd6-1e00-4975-b807-63e00ad36b44\") " pod="openshift-multus/network-metrics-daemon-qnp65" Jul 06 01:26:51.993658 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:51.993496 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e690e9f2-b314-4e44-a211-433d7e8100a1-original-pull-secret\") pod \"global-pull-secret-syncer-xf5mp\" (UID: \"e690e9f2-b314-4e44-a211-433d7e8100a1\") " pod="kube-system/global-pull-secret-syncer-xf5mp" Jul 06 01:26:51.993658 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:51.993636 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 01:26:51.993855 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:51.993696 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e690e9f2-b314-4e44-a211-433d7e8100a1-original-pull-secret podName:e690e9f2-b314-4e44-a211-433d7e8100a1 nodeName:}" failed. No retries permitted until 2026-07-06 01:26:55.993679364 +0000 UTC m=+9.964931039 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e690e9f2-b314-4e44-a211-433d7e8100a1-original-pull-secret") pod "global-pull-secret-syncer-xf5mp" (UID: "e690e9f2-b314-4e44-a211-433d7e8100a1") : object "kube-system"/"original-pull-secret" not registered Jul 06 01:26:51.994123 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:51.994104 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 01:26:51.994199 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:51.994152 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15c8ffd6-1e00-4975-b807-63e00ad36b44-metrics-certs podName:15c8ffd6-1e00-4975-b807-63e00ad36b44 nodeName:}" failed. No retries permitted until 2026-07-06 01:26:55.994142269 +0000 UTC m=+9.965393939 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/15c8ffd6-1e00-4975-b807-63e00ad36b44-metrics-certs") pod "network-metrics-daemon-qnp65" (UID: "15c8ffd6-1e00-4975-b807-63e00ad36b44") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 01:26:52.094555 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:52.094516 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdkgm\" (UniqueName: \"kubernetes.io/projected/ab026023-6959-46fd-aaf6-0b145d5a956e-kube-api-access-tdkgm\") pod \"network-check-target-57drk\" (UID: \"ab026023-6959-46fd-aaf6-0b145d5a956e\") " pod="openshift-network-diagnostics/network-check-target-57drk" Jul 06 01:26:52.094746 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:52.094712 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 01:26:52.094746 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:52.094734 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 01:26:52.094746 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:52.094747 2580 projected.go:196] Error preparing data for projected volume kube-api-access-tdkgm for pod openshift-network-diagnostics/network-check-target-57drk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 01:26:52.094911 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:52.094807 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ab026023-6959-46fd-aaf6-0b145d5a956e-kube-api-access-tdkgm podName:ab026023-6959-46fd-aaf6-0b145d5a956e nodeName:}" failed. No retries permitted until 2026-07-06 01:26:56.094786949 +0000 UTC m=+10.066038663 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-tdkgm" (UniqueName: "kubernetes.io/projected/ab026023-6959-46fd-aaf6-0b145d5a956e-kube-api-access-tdkgm") pod "network-check-target-57drk" (UID: "ab026023-6959-46fd-aaf6-0b145d5a956e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 01:26:52.456721 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:52.456689 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qnp65" Jul 06 01:26:52.457193 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:52.456836 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qnp65" podUID="15c8ffd6-1e00-4975-b807-63e00ad36b44" Jul 06 01:26:52.457333 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:52.457316 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57drk" Jul 06 01:26:52.457470 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:52.457428 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-57drk" podUID="ab026023-6959-46fd-aaf6-0b145d5a956e" Jul 06 01:26:53.453483 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:53.453431 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xf5mp" Jul 06 01:26:53.453692 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:53.453592 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xf5mp" podUID="e690e9f2-b314-4e44-a211-433d7e8100a1" Jul 06 01:26:54.456051 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:54.455557 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qnp65" Jul 06 01:26:54.456051 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:54.455611 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57drk" Jul 06 01:26:54.456051 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:54.455711 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qnp65" podUID="15c8ffd6-1e00-4975-b807-63e00ad36b44" Jul 06 01:26:54.456051 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:54.455793 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-57drk" podUID="ab026023-6959-46fd-aaf6-0b145d5a956e" Jul 06 01:26:55.454208 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:55.454169 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xf5mp" Jul 06 01:26:55.454496 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:55.454428 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xf5mp" podUID="e690e9f2-b314-4e44-a211-433d7e8100a1" Jul 06 01:26:56.028955 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:56.028912 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15c8ffd6-1e00-4975-b807-63e00ad36b44-metrics-certs\") pod \"network-metrics-daemon-qnp65\" (UID: \"15c8ffd6-1e00-4975-b807-63e00ad36b44\") " pod="openshift-multus/network-metrics-daemon-qnp65" Jul 06 01:26:56.029390 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:56.028977 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e690e9f2-b314-4e44-a211-433d7e8100a1-original-pull-secret\") pod \"global-pull-secret-syncer-xf5mp\" (UID: \"e690e9f2-b314-4e44-a211-433d7e8100a1\") " pod="kube-system/global-pull-secret-syncer-xf5mp" Jul 06 01:26:56.029390 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:56.029062 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 01:26:56.029390 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:56.029083 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 01:26:56.029390 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:56.029133 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15c8ffd6-1e00-4975-b807-63e00ad36b44-metrics-certs podName:15c8ffd6-1e00-4975-b807-63e00ad36b44 nodeName:}" failed. No retries permitted until 2026-07-06 01:27:04.02911405 +0000 UTC m=+18.000365723 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/15c8ffd6-1e00-4975-b807-63e00ad36b44-metrics-certs") pod "network-metrics-daemon-qnp65" (UID: "15c8ffd6-1e00-4975-b807-63e00ad36b44") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 01:26:56.029390 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:56.029150 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e690e9f2-b314-4e44-a211-433d7e8100a1-original-pull-secret podName:e690e9f2-b314-4e44-a211-433d7e8100a1 nodeName:}" failed. No retries permitted until 2026-07-06 01:27:04.029142971 +0000 UTC m=+18.000394639 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e690e9f2-b314-4e44-a211-433d7e8100a1-original-pull-secret") pod "global-pull-secret-syncer-xf5mp" (UID: "e690e9f2-b314-4e44-a211-433d7e8100a1") : object "kube-system"/"original-pull-secret" not registered Jul 06 01:26:56.130293 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:56.129641 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdkgm\" (UniqueName: \"kubernetes.io/projected/ab026023-6959-46fd-aaf6-0b145d5a956e-kube-api-access-tdkgm\") pod \"network-check-target-57drk\" (UID: \"ab026023-6959-46fd-aaf6-0b145d5a956e\") " pod="openshift-network-diagnostics/network-check-target-57drk" Jul 06 01:26:56.130293 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:56.129901 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 01:26:56.130293 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:56.129923 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 01:26:56.130293 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:56.129934 2580 projected.go:196] Error preparing data for projected volume kube-api-access-tdkgm for pod openshift-network-diagnostics/network-check-target-57drk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 01:26:56.130293 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:56.129994 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ab026023-6959-46fd-aaf6-0b145d5a956e-kube-api-access-tdkgm podName:ab026023-6959-46fd-aaf6-0b145d5a956e nodeName:}" failed. No retries permitted until 2026-07-06 01:27:04.129975659 +0000 UTC m=+18.101227335 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-tdkgm" (UniqueName: "kubernetes.io/projected/ab026023-6959-46fd-aaf6-0b145d5a956e-kube-api-access-tdkgm") pod "network-check-target-57drk" (UID: "ab026023-6959-46fd-aaf6-0b145d5a956e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 01:26:56.458590 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:56.458556 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57drk" Jul 06 01:26:56.458803 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:56.458674 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-57drk" podUID="ab026023-6959-46fd-aaf6-0b145d5a956e" Jul 06 01:26:56.459104 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:56.459077 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qnp65" Jul 06 01:26:56.459206 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:56.459186 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qnp65" podUID="15c8ffd6-1e00-4975-b807-63e00ad36b44" Jul 06 01:26:57.454134 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:57.454097 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xf5mp" Jul 06 01:26:57.454556 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:57.454236 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xf5mp" podUID="e690e9f2-b314-4e44-a211-433d7e8100a1" Jul 06 01:26:58.455477 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:58.454328 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qnp65" Jul 06 01:26:58.455477 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:58.454495 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qnp65" podUID="15c8ffd6-1e00-4975-b807-63e00ad36b44" Jul 06 01:26:58.455477 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:58.454871 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57drk" Jul 06 01:26:58.455477 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:58.454961 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-57drk" podUID="ab026023-6959-46fd-aaf6-0b145d5a956e" Jul 06 01:26:59.454407 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:26:59.454364 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xf5mp" Jul 06 01:26:59.454589 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:26:59.454519 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xf5mp" podUID="e690e9f2-b314-4e44-a211-433d7e8100a1" Jul 06 01:27:00.457390 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:00.457359 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qnp65" Jul 06 01:27:00.457868 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:00.457508 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qnp65" podUID="15c8ffd6-1e00-4975-b807-63e00ad36b44" Jul 06 01:27:00.457868 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:00.457570 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57drk" Jul 06 01:27:00.457868 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:00.457667 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-57drk" podUID="ab026023-6959-46fd-aaf6-0b145d5a956e" Jul 06 01:27:01.453729 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:01.453696 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xf5mp" Jul 06 01:27:01.453917 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:01.453824 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xf5mp" podUID="e690e9f2-b314-4e44-a211-433d7e8100a1" Jul 06 01:27:02.454467 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:02.454421 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qnp65" Jul 06 01:27:02.454835 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:02.454475 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57drk" Jul 06 01:27:02.454835 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:02.454558 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qnp65" podUID="15c8ffd6-1e00-4975-b807-63e00ad36b44" Jul 06 01:27:02.454835 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:02.454694 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-57drk" podUID="ab026023-6959-46fd-aaf6-0b145d5a956e" Jul 06 01:27:03.454131 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:03.454089 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xf5mp" Jul 06 01:27:03.454377 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:03.454193 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xf5mp" podUID="e690e9f2-b314-4e44-a211-433d7e8100a1" Jul 06 01:27:04.089825 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:04.089728 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15c8ffd6-1e00-4975-b807-63e00ad36b44-metrics-certs\") pod \"network-metrics-daemon-qnp65\" (UID: \"15c8ffd6-1e00-4975-b807-63e00ad36b44\") " pod="openshift-multus/network-metrics-daemon-qnp65" Jul 06 01:27:04.089825 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:04.089787 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e690e9f2-b314-4e44-a211-433d7e8100a1-original-pull-secret\") pod \"global-pull-secret-syncer-xf5mp\" (UID: \"e690e9f2-b314-4e44-a211-433d7e8100a1\") " pod="kube-system/global-pull-secret-syncer-xf5mp" Jul 06 01:27:04.090307 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:04.089884 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 01:27:04.090307 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:04.089919 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 01:27:04.090307 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:04.089964 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15c8ffd6-1e00-4975-b807-63e00ad36b44-metrics-certs podName:15c8ffd6-1e00-4975-b807-63e00ad36b44 nodeName:}" failed. No retries permitted until 2026-07-06 01:27:20.089944331 +0000 UTC m=+34.061196014 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/15c8ffd6-1e00-4975-b807-63e00ad36b44-metrics-certs") pod "network-metrics-daemon-qnp65" (UID: "15c8ffd6-1e00-4975-b807-63e00ad36b44") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 01:27:04.090307 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:04.089985 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e690e9f2-b314-4e44-a211-433d7e8100a1-original-pull-secret podName:e690e9f2-b314-4e44-a211-433d7e8100a1 nodeName:}" failed. No retries permitted until 2026-07-06 01:27:20.089975404 +0000 UTC m=+34.061227073 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e690e9f2-b314-4e44-a211-433d7e8100a1-original-pull-secret") pod "global-pull-secret-syncer-xf5mp" (UID: "e690e9f2-b314-4e44-a211-433d7e8100a1") : object "kube-system"/"original-pull-secret" not registered Jul 06 01:27:04.191162 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:04.191126 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdkgm\" (UniqueName: \"kubernetes.io/projected/ab026023-6959-46fd-aaf6-0b145d5a956e-kube-api-access-tdkgm\") pod \"network-check-target-57drk\" (UID: \"ab026023-6959-46fd-aaf6-0b145d5a956e\") " pod="openshift-network-diagnostics/network-check-target-57drk" Jul 06 01:27:04.191346 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:04.191280 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 01:27:04.191346 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:04.191305 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 01:27:04.191346 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:04.191319 2580 projected.go:196] Error preparing data for projected volume kube-api-access-tdkgm for pod openshift-network-diagnostics/network-check-target-57drk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 01:27:04.191481 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:04.191374 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ab026023-6959-46fd-aaf6-0b145d5a956e-kube-api-access-tdkgm podName:ab026023-6959-46fd-aaf6-0b145d5a956e nodeName:}" failed. No retries permitted until 2026-07-06 01:27:20.191361076 +0000 UTC m=+34.162612746 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-tdkgm" (UniqueName: "kubernetes.io/projected/ab026023-6959-46fd-aaf6-0b145d5a956e-kube-api-access-tdkgm") pod "network-check-target-57drk" (UID: "ab026023-6959-46fd-aaf6-0b145d5a956e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 01:27:04.457344 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:04.457316 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qnp65" Jul 06 01:27:04.457535 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:04.457317 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57drk" Jul 06 01:27:04.457535 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:04.457455 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qnp65" podUID="15c8ffd6-1e00-4975-b807-63e00ad36b44" Jul 06 01:27:04.457535 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:04.457495 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-57drk" podUID="ab026023-6959-46fd-aaf6-0b145d5a956e" Jul 06 01:27:05.454429 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:05.454386 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xf5mp" Jul 06 01:27:05.454913 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:05.454550 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xf5mp" podUID="e690e9f2-b314-4e44-a211-433d7e8100a1" Jul 06 01:27:06.454328 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:06.454294 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57drk" Jul 06 01:27:06.454551 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:06.454403 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-57drk" podUID="ab026023-6959-46fd-aaf6-0b145d5a956e" Jul 06 01:27:06.454909 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:06.454775 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qnp65" Jul 06 01:27:06.454909 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:06.454877 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qnp65" podUID="15c8ffd6-1e00-4975-b807-63e00ad36b44" Jul 06 01:27:07.453859 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:07.453821 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xf5mp" Jul 06 01:27:07.454052 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:07.453958 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xf5mp" podUID="e690e9f2-b314-4e44-a211-433d7e8100a1" Jul 06 01:27:08.454822 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:08.454531 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57drk" Jul 06 01:27:08.456278 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:08.454954 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-57drk" podUID="ab026023-6959-46fd-aaf6-0b145d5a956e" Jul 06 01:27:08.456278 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:08.455045 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qnp65" Jul 06 01:27:08.456278 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:08.455575 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qnp65" podUID="15c8ffd6-1e00-4975-b807-63e00ad36b44" Jul 06 01:27:08.596104 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:08.595855 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-cksd7" event={"ID":"cb081505-8ac0-48a3-be9e-bf770a830127","Type":"ContainerStarted","Data":"de64df17fdc7b29cd32c630fb0194b5d51968e142d75416edf72f663a6dea91e"} Jul 06 01:27:08.597579 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:08.597552 2580 generic.go:356] "Generic (PLEG): container finished" podID="86824090-3c80-459c-97b6-4e34bcac19c3" containerID="f157c951ce5bec3f07744058e1ece72df8c6aa10ce964b9ad9f151fd01f4e144" exitCode=0 Jul 06 01:27:08.597682 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:08.597613 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bdpq8" event={"ID":"86824090-3c80-459c-97b6-4e34bcac19c3","Type":"ContainerDied","Data":"f157c951ce5bec3f07744058e1ece72df8c6aa10ce964b9ad9f151fd01f4e144"} Jul 06 01:27:08.599306 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:08.599242 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9gm9" event={"ID":"c476bace-97d0-4492-adca-efd95306c6de","Type":"ContainerStarted","Data":"d0c82aaa5b35929711711bf686d170790a63e49c7bd2649e49e7d08da3d58d7f"} Jul 06 01:27:08.601755 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:08.601721 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" event={"ID":"04c8a1d0-bd84-4b0c-934b-efe85acfa81d","Type":"ContainerStarted","Data":"d999ad9c4ef49c92de3e73739fbe31a3b96c0aa64dbc3314a06218223fec5f91"} Jul 06 01:27:08.601852 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:08.601783 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" event={"ID":"04c8a1d0-bd84-4b0c-934b-efe85acfa81d","Type":"ContainerStarted","Data":"c67248d684df51fb701d04f498154f3d77ee154d3d9affb591481394c9e32fa6"} Jul 06 01:27:08.601852 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:08.601832 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" event={"ID":"04c8a1d0-bd84-4b0c-934b-efe85acfa81d","Type":"ContainerStarted","Data":"d96ef1cfa7ce49c929fa15ccad84315b3b17d51d490ce71199c303c15323ceba"} Jul 06 01:27:08.603184 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:08.603161 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2pmf7" event={"ID":"925f85db-233c-4a3f-9d1c-f8238bce7e4c","Type":"ContainerStarted","Data":"ff584745d7c775dd4319a7dbe6a524c66a8c7c97ce546232c9b38afb0f9f5e5b"} Jul 06 01:27:08.604880 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:08.604858 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-b46kx" event={"ID":"4164d767-f4d0-459d-b0c1-cc8f8ccbaf1a","Type":"ContainerStarted","Data":"e20a595b30e1674b345db688918b6fdcaa759764e3b7d26d8c5729e6bc141692"} Jul 06 01:27:08.606574 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:08.606549 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-xszvt" event={"ID":"ee8fc4f9-9bc1-4b56-a435-f158b76784fb","Type":"ContainerStarted","Data":"5df8abcea0f1d78e398c4716ac06e88a58c29d5a6ec38d8683f624a22c2f81f4"} Jul 06 01:27:08.611478 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:08.610337 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-44.ec2.internal" podStartSLOduration=21.610302275 podStartE2EDuration="21.610302275s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 01:26:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:26:51.570738499 +0000 UTC m=+5.541990190" watchObservedRunningTime="2026-07-06 01:27:08.610302275 +0000 UTC m=+22.581553960" Jul 06 01:27:08.611478 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:08.610737 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-cksd7" podStartSLOduration=8.239870604 podStartE2EDuration="22.6107226s" totalImagesPullingTime="14.370851996s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:26:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:27:08.610637791 +0000 UTC m=+22.581889462" watchObservedRunningTime="2026-07-06 01:27:08.6107226 +0000 UTC m=+22.581974291" Jul 06 01:27:08.611775 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:08.611753 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-82mwf" event={"ID":"00a6f863-8ae0-4749-8f27-3bdf91cd8775","Type":"ContainerStarted","Data":"202fa62acce575a3d79e458c72b35975cb22a30ab56958d255f45ff758a00621"} Jul 06 01:27:08.645957 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:08.645908 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-xszvt" podStartSLOduration=3.762836327 podStartE2EDuration="22.645890599s" totalImagesPullingTime="18.883054272s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 01:26:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:27:08.645872715 +0000 UTC m=+22.617124406" watchObservedRunningTime="2026-07-06 01:27:08.645890599 +0000 UTC m=+22.617142291" Jul 06 01:27:08.646101 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:08.646082 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-b46kx" podStartSLOduration=3.814651436 podStartE2EDuration="22.646075723s" totalImagesPullingTime="18.831424287s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 01:26:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:27:08.629730573 +0000 UTC m=+22.600982264" watchObservedRunningTime="2026-07-06 01:27:08.646075723 +0000 UTC m=+22.617327431" Jul 06 01:27:08.699080 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:08.699035 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-2pmf7" podStartSLOduration=3.860734236 podStartE2EDuration="22.69902015s" totalImagesPullingTime="18.838285914s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 01:26:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:27:08.685307326 +0000 UTC m=+22.656559019" watchObservedRunningTime="2026-07-06 01:27:08.69902015 +0000 UTC m=+22.670271862" Jul 06 01:27:08.699236 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:08.699170 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-82mwf" podStartSLOduration=8.32919606 podStartE2EDuration="22.699160034s" totalImagesPullingTime="14.369963974s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 01:26:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:27:08.698857116 +0000 UTC m=+22.670108808" watchObservedRunningTime="2026-07-06 01:27:08.699160034 +0000 UTC m=+22.670411717" Jul 06 01:27:08.707154 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:08.707066 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:27:08.773308 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:08.773274 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:27:08.878531 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:08.878383 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:27:09.453454 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:09.453419 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xf5mp" Jul 06 01:27:09.453699 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:09.453534 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xf5mp" podUID="e690e9f2-b314-4e44-a211-433d7e8100a1" Jul 06 01:27:09.575869 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:09.575833 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:27:09.615208 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:09.615171 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qmxp5" event={"ID":"e43deb0d-17a0-4c43-8120-3edd3732068a","Type":"ContainerStarted","Data":"f25f6949abf092b641365cafdb75519a8642aa4b2d72549eadf99a8710551752"} Jul 06 01:27:09.618090 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:09.618063 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" event={"ID":"04c8a1d0-bd84-4b0c-934b-efe85acfa81d","Type":"ContainerStarted","Data":"f001e1e96eafc79c014c5ad51aa5ac09dee2c5519ab91f1d3f840ec2be90c489"} Jul 06 01:27:09.618235 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:09.618139 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" event={"ID":"04c8a1d0-bd84-4b0c-934b-efe85acfa81d","Type":"ContainerStarted","Data":"217d83b7e840faa308136e99b6b013922d9a0a8d22fe713e55e35b42c36ca3ab"} Jul 06 01:27:09.618235 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:09.618193 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" event={"ID":"04c8a1d0-bd84-4b0c-934b-efe85acfa81d","Type":"ContainerStarted","Data":"7fbf1832246cb26a837ed4c7854eb6c7d6cdee83ed215c72d36cc17560c80eef"} Jul 06 01:27:09.778184 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:09.778148 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:27:09.794512 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:09.794485 2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 06 01:27:09.879134 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:09.879063 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:27:10.422221 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:10.421911 2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T01:27:09.794504376Z","UUID":"0f3b4ea2-abe1-4e08-b823-769393d308e1","Handler":null,"Name":"","Endpoint":""} Jul 06 01:27:10.423763 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:10.423738 2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 06 01:27:10.423929 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:10.423782 2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 06 01:27:10.454312 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:10.454285 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qnp65" Jul 06 01:27:10.454512 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:10.454286 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57drk" Jul 06 01:27:10.454512 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:10.454416 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qnp65" podUID="15c8ffd6-1e00-4975-b807-63e00ad36b44" Jul 06 01:27:10.454512 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:10.454492 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-57drk" podUID="ab026023-6959-46fd-aaf6-0b145d5a956e" Jul 06 01:27:10.623112 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:10.623067 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9gm9" event={"ID":"c476bace-97d0-4492-adca-efd95306c6de","Type":"ContainerStarted","Data":"aacbbfe925677c7f1863adfdea575d14d0e82d586a233b60025f05a56e958d71"} Jul 06 01:27:11.247232 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:11.246924 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:27:11.299333 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:11.299291 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:27:11.453578 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:11.453551 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xf5mp" Jul 06 01:27:11.453730 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:11.453656 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xf5mp" podUID="e690e9f2-b314-4e44-a211-433d7e8100a1" Jul 06 01:27:11.628250 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:11.628154 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9gm9" event={"ID":"c476bace-97d0-4492-adca-efd95306c6de","Type":"ContainerStarted","Data":"4e8320b9bd984850fd1f1fca70a7d45a589469ee0641bb318b6772ec56a0d12b"} Jul 06 01:27:11.631931 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:11.631889 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" event={"ID":"04c8a1d0-bd84-4b0c-934b-efe85acfa81d","Type":"ContainerStarted","Data":"2c5773a50e80e4033f58f1355fc83bf1e4b62081bd8b5477d6c7ad23b41e1fd1"} Jul 06 01:27:11.645048 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:11.644997 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-qmxp5" podStartSLOduration=6.805815864 podStartE2EDuration="25.644978931s" totalImagesPullingTime="18.839163067s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 01:26:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:27:09.631956719 +0000 UTC m=+23.603208704" watchObservedRunningTime="2026-07-06 01:27:11.644978931 +0000 UTC m=+25.616230624" Jul 06 01:27:11.645309 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:11.645278 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w9gm9" podStartSLOduration=4.469236684 podStartE2EDuration="25.645268228s" totalImagesPullingTime="21.176031544s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 01:26:46 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:27:11.644823331 +0000 UTC m=+25.616075023" watchObservedRunningTime="2026-07-06 01:27:11.645268228 +0000 UTC m=+25.616519920" Jul 06 01:27:11.952866 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:11.952831 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-cksd7" Jul 06 01:27:11.953679 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:11.953654 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-cksd7" Jul 06 01:27:12.454334 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:12.454300 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57drk" Jul 06 01:27:12.454586 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:12.454309 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qnp65" Jul 06 01:27:12.454586 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:12.454456 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-57drk" podUID="ab026023-6959-46fd-aaf6-0b145d5a956e" Jul 06 01:27:12.454586 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:12.454524 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qnp65" podUID="15c8ffd6-1e00-4975-b807-63e00ad36b44" Jul 06 01:27:12.634296 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:12.634239 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-cksd7" Jul 06 01:27:12.634776 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:12.634620 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-cksd7" Jul 06 01:27:12.690984 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:12.690949 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:27:13.453608 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:13.453578 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xf5mp" Jul 06 01:27:13.453745 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:13.453684 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xf5mp" podUID="e690e9f2-b314-4e44-a211-433d7e8100a1" Jul 06 01:27:13.647039 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:13.645967 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" event={"ID":"04c8a1d0-bd84-4b0c-934b-efe85acfa81d","Type":"ContainerStarted","Data":"c3d364d2ea4a02fbd6d0b6b153e63a7e76982ab63285f8ad38f5b5c2d604f1fa"} Jul 06 01:27:13.647039 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:13.646267 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:27:13.647039 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:13.646320 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:27:13.647039 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:13.646363 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:27:13.663992 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:13.663959 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:27:13.664136 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:13.664080 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:27:13.678424 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:13.678378 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" podStartSLOduration=8.765456613 podStartE2EDuration="27.678362353s" totalImagesPullingTime="18.91290574s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 01:26:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:27:13.677804956 +0000 UTC m=+27.649056660" watchObservedRunningTime="2026-07-06 01:27:13.678362353 +0000 UTC m=+27.649614045" Jul 06 01:27:13.932257 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:13.932227 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:27:13.982802 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:13.982722 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:27:14.454565 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:14.454531 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57drk" Jul 06 01:27:14.454743 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:14.454545 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qnp65" Jul 06 01:27:14.454743 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:14.454633 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-57drk" podUID="ab026023-6959-46fd-aaf6-0b145d5a956e" Jul 06 01:27:14.454810 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:14.454740 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qnp65" podUID="15c8ffd6-1e00-4975-b807-63e00ad36b44" Jul 06 01:27:14.650219 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:14.650186 2580 generic.go:356] "Generic (PLEG): container finished" podID="86824090-3c80-459c-97b6-4e34bcac19c3" containerID="18e2853a94180f9a72efcc6df77725c045b3174b08a444d345a9b31a79b4287e" exitCode=0 Jul 06 01:27:14.650696 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:14.650287 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bdpq8" event={"ID":"86824090-3c80-459c-97b6-4e34bcac19c3","Type":"ContainerDied","Data":"18e2853a94180f9a72efcc6df77725c045b3174b08a444d345a9b31a79b4287e"} Jul 06 01:27:15.453612 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:15.453581 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xf5mp" Jul 06 01:27:15.453848 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:15.453695 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xf5mp" podUID="e690e9f2-b314-4e44-a211-433d7e8100a1" Jul 06 01:27:15.712549 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:15.712471 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xf5mp"] Jul 06 01:27:15.712892 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:15.712570 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xf5mp" Jul 06 01:27:15.712892 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:15.712661 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xf5mp" podUID="e690e9f2-b314-4e44-a211-433d7e8100a1" Jul 06 01:27:15.715477 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:15.715378 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-57drk"] Jul 06 01:27:15.715540 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:15.715492 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57drk" Jul 06 01:27:15.715605 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:15.715582 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-57drk" podUID="ab026023-6959-46fd-aaf6-0b145d5a956e" Jul 06 01:27:15.728657 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:15.728627 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qnp65"] Jul 06 01:27:15.728790 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:15.728766 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qnp65" Jul 06 01:27:15.728914 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:15.728890 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qnp65" podUID="15c8ffd6-1e00-4975-b807-63e00ad36b44" Jul 06 01:27:16.656598 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:16.656361 2580 generic.go:356] "Generic (PLEG): container finished" podID="86824090-3c80-459c-97b6-4e34bcac19c3" containerID="163a0d1e4624f4c620160c328650e790025bf187b628f3cd284e95cc5a313ab7" exitCode=0 Jul 06 01:27:16.656598 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:16.656467 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bdpq8" event={"ID":"86824090-3c80-459c-97b6-4e34bcac19c3","Type":"ContainerDied","Data":"163a0d1e4624f4c620160c328650e790025bf187b628f3cd284e95cc5a313ab7"} Jul 06 01:27:17.454026 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:17.453993 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qnp65" Jul 06 01:27:17.454712 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:17.453999 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57drk" Jul 06 01:27:17.454712 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:17.454133 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qnp65" podUID="15c8ffd6-1e00-4975-b807-63e00ad36b44" Jul 06 01:27:17.454712 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:17.454148 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xf5mp" Jul 06 01:27:17.454712 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:17.454235 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-57drk" podUID="ab026023-6959-46fd-aaf6-0b145d5a956e" Jul 06 01:27:17.454712 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:17.454305 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xf5mp" podUID="e690e9f2-b314-4e44-a211-433d7e8100a1" Jul 06 01:27:18.438563 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:18.438529 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:27:18.542732 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:18.542690 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:27:18.664411 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:18.664375 2580 generic.go:356] "Generic (PLEG): container finished" podID="86824090-3c80-459c-97b6-4e34bcac19c3" containerID="fa2083b76514059161d33af5d3bd947517bcee0aab6d220c2bcbcc87939d1f52" exitCode=0 Jul 06 01:27:18.664576 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:18.664455 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bdpq8" event={"ID":"86824090-3c80-459c-97b6-4e34bcac19c3","Type":"ContainerDied","Data":"fa2083b76514059161d33af5d3bd947517bcee0aab6d220c2bcbcc87939d1f52"} Jul 06 01:27:19.454008 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:19.453974 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57drk" Jul 06 01:27:19.454008 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:19.454002 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qnp65" Jul 06 01:27:19.454266 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:19.454002 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xf5mp" Jul 06 01:27:19.454266 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:19.454086 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-57drk" podUID="ab026023-6959-46fd-aaf6-0b145d5a956e" Jul 06 01:27:19.454266 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:19.454180 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qnp65" podUID="15c8ffd6-1e00-4975-b807-63e00ad36b44" Jul 06 01:27:19.454266 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:19.454253 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xf5mp" podUID="e690e9f2-b314-4e44-a211-433d7e8100a1" Jul 06 01:27:19.875268 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:19.875150 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:27:19.937301 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:19.937267 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:27:20.104679 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:20.104645 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15c8ffd6-1e00-4975-b807-63e00ad36b44-metrics-certs\") pod \"network-metrics-daemon-qnp65\" (UID: \"15c8ffd6-1e00-4975-b807-63e00ad36b44\") " pod="openshift-multus/network-metrics-daemon-qnp65" Jul 06 01:27:20.104866 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:20.104690 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e690e9f2-b314-4e44-a211-433d7e8100a1-original-pull-secret\") pod \"global-pull-secret-syncer-xf5mp\" (UID: \"e690e9f2-b314-4e44-a211-433d7e8100a1\") " pod="kube-system/global-pull-secret-syncer-xf5mp" Jul 06 01:27:20.104866 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:20.104821 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 01:27:20.104967 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:20.104879 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 01:27:20.104967 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:20.104896 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15c8ffd6-1e00-4975-b807-63e00ad36b44-metrics-certs podName:15c8ffd6-1e00-4975-b807-63e00ad36b44 nodeName:}" failed. No retries permitted until 2026-07-06 01:27:52.104876278 +0000 UTC m=+66.076127967 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/15c8ffd6-1e00-4975-b807-63e00ad36b44-metrics-certs") pod "network-metrics-daemon-qnp65" (UID: "15c8ffd6-1e00-4975-b807-63e00ad36b44") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 01:27:20.104967 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:20.104936 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e690e9f2-b314-4e44-a211-433d7e8100a1-original-pull-secret podName:e690e9f2-b314-4e44-a211-433d7e8100a1 nodeName:}" failed. No retries permitted until 2026-07-06 01:27:52.104919701 +0000 UTC m=+66.076171375 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e690e9f2-b314-4e44-a211-433d7e8100a1-original-pull-secret") pod "global-pull-secret-syncer-xf5mp" (UID: "e690e9f2-b314-4e44-a211-433d7e8100a1") : object "kube-system"/"original-pull-secret" not registered Jul 06 01:27:20.205763 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:20.205706 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdkgm\" (UniqueName: \"kubernetes.io/projected/ab026023-6959-46fd-aaf6-0b145d5a956e-kube-api-access-tdkgm\") pod \"network-check-target-57drk\" (UID: \"ab026023-6959-46fd-aaf6-0b145d5a956e\") " pod="openshift-network-diagnostics/network-check-target-57drk" Jul 06 01:27:20.205940 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:20.205905 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 01:27:20.205940 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:20.205929 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 01:27:20.206082 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:20.205942 2580 projected.go:196] Error preparing data for projected volume kube-api-access-tdkgm for pod openshift-network-diagnostics/network-check-target-57drk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 01:27:20.206082 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:20.206012 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ab026023-6959-46fd-aaf6-0b145d5a956e-kube-api-access-tdkgm podName:ab026023-6959-46fd-aaf6-0b145d5a956e nodeName:}" failed. No retries permitted until 2026-07-06 01:27:52.205993492 +0000 UTC m=+66.177245174 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-tdkgm" (UniqueName: "kubernetes.io/projected/ab026023-6959-46fd-aaf6-0b145d5a956e-kube-api-access-tdkgm") pod "network-check-target-57drk" (UID: "ab026023-6959-46fd-aaf6-0b145d5a956e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 01:27:20.830182 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:20.830151 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-44.ec2.internal" event="NodeReady" Jul 06 01:27:20.830362 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:20.830282 2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 06 01:27:20.874947 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:20.874914 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-dxjmj"] Jul 06 01:27:20.879752 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:20.879727 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-v7w8q"] Jul 06 01:27:20.880210 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:20.879854 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dxjmj" Jul 06 01:27:20.882613 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:20.882583 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-58ndz\"" Jul 06 01:27:20.882613 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:20.882585 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 06 01:27:20.882785 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:20.882583 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 06 01:27:20.883705 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:20.883686 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-v7w8q" Jul 06 01:27:20.886452 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:20.886406 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dxjmj"] Jul 06 01:27:20.886769 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:20.886748 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-bszxs\"" Jul 06 01:27:20.887507 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:20.887027 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 06 01:27:20.887507 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:20.887224 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 06 01:27:20.887507 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:20.887411 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 06 01:27:20.890483 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:20.890459 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-v7w8q"] Jul 06 01:27:20.983328 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:20.983286 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:27:21.010744 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:21.010649 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5f07e4d-9391-4b17-8cd2-298dde338112-tmp-dir\") pod \"dns-default-dxjmj\" (UID: \"d5f07e4d-9391-4b17-8cd2-298dde338112\") " pod="openshift-dns/dns-default-dxjmj" Jul 06 01:27:21.010744 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:21.010723 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d90b9b62-2b7e-439a-9d24-4893aa5543d9-cert\") pod \"ingress-canary-v7w8q\" (UID: \"d90b9b62-2b7e-439a-9d24-4893aa5543d9\") " pod="openshift-ingress-canary/ingress-canary-v7w8q" Jul 06 01:27:21.010980 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:21.010751 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d5f07e4d-9391-4b17-8cd2-298dde338112-config-volume\") pod \"dns-default-dxjmj\" (UID: \"d5f07e4d-9391-4b17-8cd2-298dde338112\") " pod="openshift-dns/dns-default-dxjmj" Jul 06 01:27:21.010980 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:21.010788 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgvl5\" (UniqueName: \"kubernetes.io/projected/d5f07e4d-9391-4b17-8cd2-298dde338112-kube-api-access-zgvl5\") pod \"dns-default-dxjmj\" (UID: \"d5f07e4d-9391-4b17-8cd2-298dde338112\") " pod="openshift-dns/dns-default-dxjmj" Jul 06 01:27:21.010980 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:21.010901 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5f07e4d-9391-4b17-8cd2-298dde338112-metrics-tls\") pod \"dns-default-dxjmj\" (UID: \"d5f07e4d-9391-4b17-8cd2-298dde338112\") " pod="openshift-dns/dns-default-dxjmj" Jul 06 01:27:21.010980 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:21.010933 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpf76\" (UniqueName: \"kubernetes.io/projected/d90b9b62-2b7e-439a-9d24-4893aa5543d9-kube-api-access-xpf76\") pod \"ingress-canary-v7w8q\" (UID: \"d90b9b62-2b7e-439a-9d24-4893aa5543d9\") " pod="openshift-ingress-canary/ingress-canary-v7w8q" Jul 06 01:27:21.033743 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:21.033708 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:27:21.111996 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:21.111962 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5f07e4d-9391-4b17-8cd2-298dde338112-tmp-dir\") pod \"dns-default-dxjmj\" (UID: \"d5f07e4d-9391-4b17-8cd2-298dde338112\") " pod="openshift-dns/dns-default-dxjmj" Jul 06 01:27:21.112205 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:21.112010 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d90b9b62-2b7e-439a-9d24-4893aa5543d9-cert\") pod \"ingress-canary-v7w8q\" (UID: \"d90b9b62-2b7e-439a-9d24-4893aa5543d9\") " pod="openshift-ingress-canary/ingress-canary-v7w8q" Jul 06 01:27:21.112205 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:21.112071 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d5f07e4d-9391-4b17-8cd2-298dde338112-config-volume\") pod \"dns-default-dxjmj\" (UID: \"d5f07e4d-9391-4b17-8cd2-298dde338112\") " pod="openshift-dns/dns-default-dxjmj" Jul 06 01:27:21.112205 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:21.112113 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgvl5\" (UniqueName: \"kubernetes.io/projected/d5f07e4d-9391-4b17-8cd2-298dde338112-kube-api-access-zgvl5\") pod \"dns-default-dxjmj\" (UID: \"d5f07e4d-9391-4b17-8cd2-298dde338112\") " pod="openshift-dns/dns-default-dxjmj" Jul 06 01:27:21.112205 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:21.112201 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5f07e4d-9391-4b17-8cd2-298dde338112-metrics-tls\") pod \"dns-default-dxjmj\" (UID: \"d5f07e4d-9391-4b17-8cd2-298dde338112\") " pod="openshift-dns/dns-default-dxjmj" Jul 06 01:27:21.112409 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:21.112229 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpf76\" (UniqueName: \"kubernetes.io/projected/d90b9b62-2b7e-439a-9d24-4893aa5543d9-kube-api-access-xpf76\") pod \"ingress-canary-v7w8q\" (UID: \"d90b9b62-2b7e-439a-9d24-4893aa5543d9\") " pod="openshift-ingress-canary/ingress-canary-v7w8q" Jul 06 01:27:21.112476 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:21.112402 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5f07e4d-9391-4b17-8cd2-298dde338112-tmp-dir\") pod \"dns-default-dxjmj\" (UID: \"d5f07e4d-9391-4b17-8cd2-298dde338112\") " pod="openshift-dns/dns-default-dxjmj" Jul 06 01:27:21.112581 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:21.112562 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 01:27:21.112648 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:21.112637 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5f07e4d-9391-4b17-8cd2-298dde338112-metrics-tls podName:d5f07e4d-9391-4b17-8cd2-298dde338112 nodeName:}" failed. No retries permitted until 2026-07-06 01:27:21.612616951 +0000 UTC m=+35.583868630 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d5f07e4d-9391-4b17-8cd2-298dde338112-metrics-tls") pod "dns-default-dxjmj" (UID: "d5f07e4d-9391-4b17-8cd2-298dde338112") : secret "dns-default-metrics-tls" not found Jul 06 01:27:21.112852 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:21.112836 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d5f07e4d-9391-4b17-8cd2-298dde338112-config-volume\") pod \"dns-default-dxjmj\" (UID: \"d5f07e4d-9391-4b17-8cd2-298dde338112\") " pod="openshift-dns/dns-default-dxjmj" Jul 06 01:27:21.117662 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:21.117459 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d90b9b62-2b7e-439a-9d24-4893aa5543d9-cert\") pod \"ingress-canary-v7w8q\" (UID: \"d90b9b62-2b7e-439a-9d24-4893aa5543d9\") " pod="openshift-ingress-canary/ingress-canary-v7w8q" Jul 06 01:27:21.124102 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:21.124076 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpf76\" (UniqueName: \"kubernetes.io/projected/d90b9b62-2b7e-439a-9d24-4893aa5543d9-kube-api-access-xpf76\") pod \"ingress-canary-v7w8q\" (UID: \"d90b9b62-2b7e-439a-9d24-4893aa5543d9\") " pod="openshift-ingress-canary/ingress-canary-v7w8q" Jul 06 01:27:21.124880 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:21.124838 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgvl5\" (UniqueName: \"kubernetes.io/projected/d5f07e4d-9391-4b17-8cd2-298dde338112-kube-api-access-zgvl5\") pod \"dns-default-dxjmj\" (UID: \"d5f07e4d-9391-4b17-8cd2-298dde338112\") " pod="openshift-dns/dns-default-dxjmj" Jul 06 01:27:21.200985 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:21.200942 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-v7w8q" Jul 06 01:27:21.454427 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:21.454391 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qnp65" Jul 06 01:27:21.454427 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:21.454422 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57drk" Jul 06 01:27:21.454703 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:21.454528 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xf5mp" Jul 06 01:27:21.461505 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:21.458089 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 01:27:21.461505 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:21.458968 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-n24x9\"" Jul 06 01:27:21.461505 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:21.459215 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qszpf\"" Jul 06 01:27:21.461505 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:21.459238 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 01:27:21.461505 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:21.459499 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 01:27:21.461505 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:21.459714 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 01:27:21.587301 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:21.587267 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:27:21.616379 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:21.616346 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5f07e4d-9391-4b17-8cd2-298dde338112-metrics-tls\") pod \"dns-default-dxjmj\" (UID: \"d5f07e4d-9391-4b17-8cd2-298dde338112\") " pod="openshift-dns/dns-default-dxjmj" Jul 06 01:27:21.616581 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:21.616540 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 01:27:21.616648 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:21.616607 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5f07e4d-9391-4b17-8cd2-298dde338112-metrics-tls podName:d5f07e4d-9391-4b17-8cd2-298dde338112 nodeName:}" failed. No retries permitted until 2026-07-06 01:27:22.616590325 +0000 UTC m=+36.587842003 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d5f07e4d-9391-4b17-8cd2-298dde338112-metrics-tls") pod "dns-default-dxjmj" (UID: "d5f07e4d-9391-4b17-8cd2-298dde338112") : secret "dns-default-metrics-tls" not found Jul 06 01:27:22.625639 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:22.625594 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5f07e4d-9391-4b17-8cd2-298dde338112-metrics-tls\") pod \"dns-default-dxjmj\" (UID: \"d5f07e4d-9391-4b17-8cd2-298dde338112\") " pod="openshift-dns/dns-default-dxjmj" Jul 06 01:27:22.626063 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:22.625760 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 01:27:22.626063 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:22.625838 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5f07e4d-9391-4b17-8cd2-298dde338112-metrics-tls podName:d5f07e4d-9391-4b17-8cd2-298dde338112 nodeName:}" failed. No retries permitted until 2026-07-06 01:27:24.625816588 +0000 UTC m=+38.597068268 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d5f07e4d-9391-4b17-8cd2-298dde338112-metrics-tls") pod "dns-default-dxjmj" (UID: "d5f07e4d-9391-4b17-8cd2-298dde338112") : secret "dns-default-metrics-tls" not found Jul 06 01:27:22.857889 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:22.857859 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-v7w8q"] Jul 06 01:27:22.861827 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:27:22.861798 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd90b9b62_2b7e_439a_9d24_4893aa5543d9.slice/crio-c84a70f841daf3b7516a80ff33bff3e4b557b655ac9a0c6b998252df24a8a287 WatchSource:0}: Error finding container c84a70f841daf3b7516a80ff33bff3e4b557b655ac9a0c6b998252df24a8a287: Status 404 returned error can't find the container with id c84a70f841daf3b7516a80ff33bff3e4b557b655ac9a0c6b998252df24a8a287 Jul 06 01:27:23.682343 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:23.682297 2580 generic.go:356] "Generic (PLEG): container finished" podID="86824090-3c80-459c-97b6-4e34bcac19c3" containerID="bd540dd8e177ba15a4eab9469b95ab73a76d10578208698dbb9a37a9bb7698b4" exitCode=0 Jul 06 01:27:23.682763 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:23.682383 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bdpq8" event={"ID":"86824090-3c80-459c-97b6-4e34bcac19c3","Type":"ContainerDied","Data":"bd540dd8e177ba15a4eab9469b95ab73a76d10578208698dbb9a37a9bb7698b4"} Jul 06 01:27:23.683647 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:23.683619 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-v7w8q" event={"ID":"d90b9b62-2b7e-439a-9d24-4893aa5543d9","Type":"ContainerStarted","Data":"c84a70f841daf3b7516a80ff33bff3e4b557b655ac9a0c6b998252df24a8a287"} Jul 06 01:27:24.643941 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:24.643724 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5f07e4d-9391-4b17-8cd2-298dde338112-metrics-tls\") pod \"dns-default-dxjmj\" (UID: \"d5f07e4d-9391-4b17-8cd2-298dde338112\") " pod="openshift-dns/dns-default-dxjmj" Jul 06 01:27:24.644145 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:24.643870 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 01:27:24.644145 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:24.644037 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5f07e4d-9391-4b17-8cd2-298dde338112-metrics-tls podName:d5f07e4d-9391-4b17-8cd2-298dde338112 nodeName:}" failed. No retries permitted until 2026-07-06 01:27:28.64401482 +0000 UTC m=+42.615266507 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d5f07e4d-9391-4b17-8cd2-298dde338112-metrics-tls") pod "dns-default-dxjmj" (UID: "d5f07e4d-9391-4b17-8cd2-298dde338112") : secret "dns-default-metrics-tls" not found Jul 06 01:27:24.689890 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:24.689855 2580 generic.go:356] "Generic (PLEG): container finished" podID="86824090-3c80-459c-97b6-4e34bcac19c3" containerID="ec289fe80cc19d54860600ace124641fe91dd900f15dda62c86d1c1da479ec4f" exitCode=0 Jul 06 01:27:24.690360 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:24.689931 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bdpq8" event={"ID":"86824090-3c80-459c-97b6-4e34bcac19c3","Type":"ContainerDied","Data":"ec289fe80cc19d54860600ace124641fe91dd900f15dda62c86d1c1da479ec4f"} Jul 06 01:27:24.720388 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:24.720348 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:27:24.776755 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:24.776710 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:27:25.443252 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:25.443221 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:27:25.543232 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:25.543199 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:27:25.696339 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:25.696295 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bdpq8" event={"ID":"86824090-3c80-459c-97b6-4e34bcac19c3","Type":"ContainerStarted","Data":"cfd9c052f8523f9d1cdd452db567e44d30872bd4417464ed8f1953917bab1864"} Jul 06 01:27:25.697700 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:25.697677 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-v7w8q" event={"ID":"d90b9b62-2b7e-439a-9d24-4893aa5543d9","Type":"ContainerStarted","Data":"a30af5b287dd28320858eb0b0166b6b774e0af7856270d8cf39417c7793aa3aa"} Jul 06 01:27:25.719340 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:25.719293 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-bdpq8" podStartSLOduration=9.700265432 podStartE2EDuration="39.719278724s" totalImagesPullingTime="30.019013292s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 01:26:46 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:27:25.717751092 +0000 UTC m=+39.689002782" watchObservedRunningTime="2026-07-06 01:27:25.719278724 +0000 UTC m=+39.690530414" Jul 06 01:27:25.732224 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:25.732167 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-v7w8q" podStartSLOduration=3.241480598 podStartE2EDuration="5.732146177s" totalImagesPullingTime="2.490665579s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:27:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:27:25.731399524 +0000 UTC m=+39.702651215" watchObservedRunningTime="2026-07-06 01:27:25.732146177 +0000 UTC m=+39.703397869" Jul 06 01:27:25.804342 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:25.804306 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:27:25.859416 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:25.859386 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:27:25.914674 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:25.914638 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:27:28.205154 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:28.205112 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5fd6d69f-4g6hq"] Jul 06 01:27:28.232283 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:28.232248 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-658bcdcb4f-c6mtj"] Jul 06 01:27:28.232478 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:28.232370 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5fd6d69f-4g6hq" Jul 06 01:27:28.235258 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:28.235227 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 06 01:27:28.235258 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:28.235256 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 06 01:27:28.235472 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:28.235260 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 06 01:27:28.235472 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:28.235260 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 06 01:27:28.252224 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:28.252197 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5fd6d69f-4g6hq"] Jul 06 01:27:28.252349 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:28.252235 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-658bcdcb4f-c6mtj"] Jul 06 01:27:28.252349 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:28.252314 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-658bcdcb4f-c6mtj" Jul 06 01:27:28.254908 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:28.254886 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 06 01:27:28.254908 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:28.254885 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 06 01:27:28.255092 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:28.254885 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 06 01:27:28.255092 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:28.254924 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 06 01:27:28.371653 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:28.371616 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8rdg\" (UniqueName: \"kubernetes.io/projected/235fdd76-2b3a-4698-b9e5-ab05305231c6-kube-api-access-s8rdg\") pod \"klusterlet-addon-workmgr-7c5fd6d69f-4g6hq\" (UID: \"235fdd76-2b3a-4698-b9e5-ab05305231c6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5fd6d69f-4g6hq" Jul 06 01:27:28.371653 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:28.371655 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/1b429a9e-ffde-4404-a881-e2794a4bbd48-ca\") pod \"cluster-proxy-proxy-agent-658bcdcb4f-c6mtj\" (UID: \"1b429a9e-ffde-4404-a881-e2794a4bbd48\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-658bcdcb4f-c6mtj" Jul 06 01:27:28.371858 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:28.371674 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/1b429a9e-ffde-4404-a881-e2794a4bbd48-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-658bcdcb4f-c6mtj\" (UID: \"1b429a9e-ffde-4404-a881-e2794a4bbd48\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-658bcdcb4f-c6mtj" Jul 06 01:27:28.371858 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:28.371758 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b429a9e-ffde-4404-a881-e2794a4bbd48-additional-service-ca\") pod \"cluster-proxy-proxy-agent-658bcdcb4f-c6mtj\" (UID: \"1b429a9e-ffde-4404-a881-e2794a4bbd48\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-658bcdcb4f-c6mtj" Jul 06 01:27:28.371858 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:28.371800 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/235fdd76-2b3a-4698-b9e5-ab05305231c6-tmp\") pod \"klusterlet-addon-workmgr-7c5fd6d69f-4g6hq\" (UID: \"235fdd76-2b3a-4698-b9e5-ab05305231c6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5fd6d69f-4g6hq" Jul 06 01:27:28.371858 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:28.371822 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/235fdd76-2b3a-4698-b9e5-ab05305231c6-klusterlet-config\") pod \"klusterlet-addon-workmgr-7c5fd6d69f-4g6hq\" (UID: \"235fdd76-2b3a-4698-b9e5-ab05305231c6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5fd6d69f-4g6hq" Jul 06 01:27:28.371982 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:28.371866 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/1b429a9e-ffde-4404-a881-e2794a4bbd48-hub\") pod \"cluster-proxy-proxy-agent-658bcdcb4f-c6mtj\" (UID: \"1b429a9e-ffde-4404-a881-e2794a4bbd48\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-658bcdcb4f-c6mtj" Jul 06 01:27:28.371982 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:28.371885 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1b429a9e-ffde-4404-a881-e2794a4bbd48-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-658bcdcb4f-c6mtj\" (UID: \"1b429a9e-ffde-4404-a881-e2794a4bbd48\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-658bcdcb4f-c6mtj" Jul 06 01:27:28.371982 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:28.371923 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77sc8\" (UniqueName: \"kubernetes.io/projected/1b429a9e-ffde-4404-a881-e2794a4bbd48-kube-api-access-77sc8\") pod \"cluster-proxy-proxy-agent-658bcdcb4f-c6mtj\" (UID: \"1b429a9e-ffde-4404-a881-e2794a4bbd48\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-658bcdcb4f-c6mtj" Jul 06 01:27:28.472586 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:28.472491 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77sc8\" (UniqueName: \"kubernetes.io/projected/1b429a9e-ffde-4404-a881-e2794a4bbd48-kube-api-access-77sc8\") pod \"cluster-proxy-proxy-agent-658bcdcb4f-c6mtj\" (UID: \"1b429a9e-ffde-4404-a881-e2794a4bbd48\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-658bcdcb4f-c6mtj" Jul 06 01:27:28.472586 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:28.472546 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8rdg\" (UniqueName: \"kubernetes.io/projected/235fdd76-2b3a-4698-b9e5-ab05305231c6-kube-api-access-s8rdg\") pod \"klusterlet-addon-workmgr-7c5fd6d69f-4g6hq\" (UID: \"235fdd76-2b3a-4698-b9e5-ab05305231c6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5fd6d69f-4g6hq" Jul 06 01:27:28.472782 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:28.472613 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/1b429a9e-ffde-4404-a881-e2794a4bbd48-ca\") pod \"cluster-proxy-proxy-agent-658bcdcb4f-c6mtj\" (UID: \"1b429a9e-ffde-4404-a881-e2794a4bbd48\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-658bcdcb4f-c6mtj" Jul 06 01:27:28.472782 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:28.472643 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/1b429a9e-ffde-4404-a881-e2794a4bbd48-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-658bcdcb4f-c6mtj\" (UID: \"1b429a9e-ffde-4404-a881-e2794a4bbd48\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-658bcdcb4f-c6mtj" Jul 06 01:27:28.472782 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:28.472676 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b429a9e-ffde-4404-a881-e2794a4bbd48-additional-service-ca\") pod \"cluster-proxy-proxy-agent-658bcdcb4f-c6mtj\" (UID: \"1b429a9e-ffde-4404-a881-e2794a4bbd48\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-658bcdcb4f-c6mtj" Jul 06 01:27:28.472782 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:28.472710 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/235fdd76-2b3a-4698-b9e5-ab05305231c6-tmp\") pod \"klusterlet-addon-workmgr-7c5fd6d69f-4g6hq\" (UID: \"235fdd76-2b3a-4698-b9e5-ab05305231c6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5fd6d69f-4g6hq" Jul 06 01:27:28.472782 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:28.472736 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/235fdd76-2b3a-4698-b9e5-ab05305231c6-klusterlet-config\") pod \"klusterlet-addon-workmgr-7c5fd6d69f-4g6hq\" (UID: \"235fdd76-2b3a-4698-b9e5-ab05305231c6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5fd6d69f-4g6hq" Jul 06 01:27:28.473064 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:28.472800 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/1b429a9e-ffde-4404-a881-e2794a4bbd48-hub\") pod \"cluster-proxy-proxy-agent-658bcdcb4f-c6mtj\" (UID: \"1b429a9e-ffde-4404-a881-e2794a4bbd48\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-658bcdcb4f-c6mtj" Jul 06 01:27:28.473064 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:28.472828 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1b429a9e-ffde-4404-a881-e2794a4bbd48-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-658bcdcb4f-c6mtj\" (UID: \"1b429a9e-ffde-4404-a881-e2794a4bbd48\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-658bcdcb4f-c6mtj" Jul 06 01:27:28.473169 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:28.473145 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/235fdd76-2b3a-4698-b9e5-ab05305231c6-tmp\") pod \"klusterlet-addon-workmgr-7c5fd6d69f-4g6hq\" (UID: \"235fdd76-2b3a-4698-b9e5-ab05305231c6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5fd6d69f-4g6hq" Jul 06 01:27:28.473545 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:28.473523 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b429a9e-ffde-4404-a881-e2794a4bbd48-additional-service-ca\") pod \"cluster-proxy-proxy-agent-658bcdcb4f-c6mtj\" (UID: \"1b429a9e-ffde-4404-a881-e2794a4bbd48\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-658bcdcb4f-c6mtj" Jul 06 01:27:28.476876 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:28.476854 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/235fdd76-2b3a-4698-b9e5-ab05305231c6-klusterlet-config\") pod \"klusterlet-addon-workmgr-7c5fd6d69f-4g6hq\" (UID: \"235fdd76-2b3a-4698-b9e5-ab05305231c6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5fd6d69f-4g6hq" Jul 06 01:27:28.477047 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:28.477025 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/1b429a9e-ffde-4404-a881-e2794a4bbd48-hub\") pod \"cluster-proxy-proxy-agent-658bcdcb4f-c6mtj\" (UID: \"1b429a9e-ffde-4404-a881-e2794a4bbd48\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-658bcdcb4f-c6mtj" Jul 06 01:27:28.477095 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:28.477033 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/1b429a9e-ffde-4404-a881-e2794a4bbd48-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-658bcdcb4f-c6mtj\" (UID: \"1b429a9e-ffde-4404-a881-e2794a4bbd48\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-658bcdcb4f-c6mtj" Jul 06 01:27:28.477095 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:28.477033 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/1b429a9e-ffde-4404-a881-e2794a4bbd48-ca\") pod \"cluster-proxy-proxy-agent-658bcdcb4f-c6mtj\" (UID: \"1b429a9e-ffde-4404-a881-e2794a4bbd48\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-658bcdcb4f-c6mtj" Jul 06 01:27:28.477164 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:28.477153 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1b429a9e-ffde-4404-a881-e2794a4bbd48-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-658bcdcb4f-c6mtj\" (UID: \"1b429a9e-ffde-4404-a881-e2794a4bbd48\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-658bcdcb4f-c6mtj" Jul 06 01:27:28.481289 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:28.481264 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8rdg\" (UniqueName: \"kubernetes.io/projected/235fdd76-2b3a-4698-b9e5-ab05305231c6-kube-api-access-s8rdg\") pod \"klusterlet-addon-workmgr-7c5fd6d69f-4g6hq\" (UID: \"235fdd76-2b3a-4698-b9e5-ab05305231c6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5fd6d69f-4g6hq" Jul 06 01:27:28.482131 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:28.482114 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-77sc8\" (UniqueName: \"kubernetes.io/projected/1b429a9e-ffde-4404-a881-e2794a4bbd48-kube-api-access-77sc8\") pod \"cluster-proxy-proxy-agent-658bcdcb4f-c6mtj\" (UID: \"1b429a9e-ffde-4404-a881-e2794a4bbd48\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-658bcdcb4f-c6mtj" Jul 06 01:27:28.541217 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:28.541175 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5fd6d69f-4g6hq" Jul 06 01:27:28.576190 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:28.576158 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-658bcdcb4f-c6mtj" Jul 06 01:27:28.674902 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:28.674866 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5f07e4d-9391-4b17-8cd2-298dde338112-metrics-tls\") pod \"dns-default-dxjmj\" (UID: \"d5f07e4d-9391-4b17-8cd2-298dde338112\") " pod="openshift-dns/dns-default-dxjmj" Jul 06 01:27:28.675067 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:28.675044 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 01:27:28.675149 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:28.675136 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5f07e4d-9391-4b17-8cd2-298dde338112-metrics-tls podName:d5f07e4d-9391-4b17-8cd2-298dde338112 nodeName:}" failed. No retries permitted until 2026-07-06 01:27:36.675108173 +0000 UTC m=+50.646359852 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d5f07e4d-9391-4b17-8cd2-298dde338112-metrics-tls") pod "dns-default-dxjmj" (UID: "d5f07e4d-9391-4b17-8cd2-298dde338112") : secret "dns-default-metrics-tls" not found Jul 06 01:27:28.724727 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:28.724651 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5fd6d69f-4g6hq"] Jul 06 01:27:28.728112 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:27:28.728082 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod235fdd76_2b3a_4698_b9e5_ab05305231c6.slice/crio-6589c717c162c6d0177e239c5f5d4247eb27dc19c13b2098850e33dc5b228d7f WatchSource:0}: Error finding container 6589c717c162c6d0177e239c5f5d4247eb27dc19c13b2098850e33dc5b228d7f: Status 404 returned error can't find the container with id 6589c717c162c6d0177e239c5f5d4247eb27dc19c13b2098850e33dc5b228d7f Jul 06 01:27:28.768591 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:28.768558 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-658bcdcb4f-c6mtj"] Jul 06 01:27:28.773142 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:27:28.773114 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1b429a9e_ffde_4404_a881_e2794a4bbd48.slice/crio-8fff2dfae808189ce491565cf3f4b2dcb9faedb188d98da0a875e66d38e31465 WatchSource:0}: Error finding container 8fff2dfae808189ce491565cf3f4b2dcb9faedb188d98da0a875e66d38e31465: Status 404 returned error can't find the container with id 8fff2dfae808189ce491565cf3f4b2dcb9faedb188d98da0a875e66d38e31465 Jul 06 01:27:29.710056 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:29.710011 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5fd6d69f-4g6hq" event={"ID":"235fdd76-2b3a-4698-b9e5-ab05305231c6","Type":"ContainerStarted","Data":"6589c717c162c6d0177e239c5f5d4247eb27dc19c13b2098850e33dc5b228d7f"} Jul 06 01:27:29.711490 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:29.711420 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-658bcdcb4f-c6mtj" event={"ID":"1b429a9e-ffde-4404-a881-e2794a4bbd48","Type":"ContainerStarted","Data":"8fff2dfae808189ce491565cf3f4b2dcb9faedb188d98da0a875e66d38e31465"} Jul 06 01:27:34.013947 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:34.013906 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:27:34.111837 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:34.111793 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:27:35.728714 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:35.728675 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-658bcdcb4f-c6mtj" event={"ID":"1b429a9e-ffde-4404-a881-e2794a4bbd48","Type":"ContainerStarted","Data":"eba29e88acd2c97789380944b511c19025496812c6cf69a7ca06bc81fa274fc2"} Jul 06 01:27:35.728714 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:35.728721 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-658bcdcb4f-c6mtj" event={"ID":"1b429a9e-ffde-4404-a881-e2794a4bbd48","Type":"ContainerStarted","Data":"e5d90854bd1b3a2578fa1bfbe0f31b7a028a303585ddbe2368779b7bb9cec459"} Jul 06 01:27:35.729224 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:35.728741 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-658bcdcb4f-c6mtj" event={"ID":"1b429a9e-ffde-4404-a881-e2794a4bbd48","Type":"ContainerStarted","Data":"cc0aa864650c55f2261dd160d4015d6776eba163e23e5e312ae6f9f65d718f46"} Jul 06 01:27:35.730225 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:35.730196 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5fd6d69f-4g6hq" event={"ID":"235fdd76-2b3a-4698-b9e5-ab05305231c6","Type":"ContainerStarted","Data":"9b03f4ae5175a7f2890d6ab0062631b15365fb31216602b63315233188365873"} Jul 06 01:27:35.730323 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:35.730303 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5fd6d69f-4g6hq" Jul 06 01:27:35.731896 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:35.731874 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5fd6d69f-4g6hq" Jul 06 01:27:35.746857 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:35.746804 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-658bcdcb4f-c6mtj" podStartSLOduration=1.745201888 podStartE2EDuration="7.746786176s" totalImagesPullingTime="6.001584288s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:27:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:27:35.746173389 +0000 UTC m=+49.717425080" watchObservedRunningTime="2026-07-06 01:27:35.746786176 +0000 UTC m=+49.718037868" Jul 06 01:27:35.762770 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:35.762711 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5fd6d69f-4g6hq" podStartSLOduration=1.701712955 podStartE2EDuration="7.762693092s" totalImagesPullingTime="6.060980137s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:27:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:27:35.761671655 +0000 UTC m=+49.732923346" watchObservedRunningTime="2026-07-06 01:27:35.762693092 +0000 UTC m=+49.733944782" Jul 06 01:27:36.740113 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:36.740072 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5f07e4d-9391-4b17-8cd2-298dde338112-metrics-tls\") pod \"dns-default-dxjmj\" (UID: \"d5f07e4d-9391-4b17-8cd2-298dde338112\") " pod="openshift-dns/dns-default-dxjmj" Jul 06 01:27:36.740533 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:36.740211 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 01:27:36.740533 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:36.740275 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5f07e4d-9391-4b17-8cd2-298dde338112-metrics-tls podName:d5f07e4d-9391-4b17-8cd2-298dde338112 nodeName:}" failed. No retries permitted until 2026-07-06 01:27:52.740255844 +0000 UTC m=+66.711507533 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d5f07e4d-9391-4b17-8cd2-298dde338112-metrics-tls") pod "dns-default-dxjmj" (UID: "d5f07e4d-9391-4b17-8cd2-298dde338112") : secret "dns-default-metrics-tls" not found Jul 06 01:27:44.720732 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:44.720690 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:27:44.794568 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:44.794530 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:27:45.664299 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:45.664271 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2sd2m" Jul 06 01:27:45.774951 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:45.774904 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:27:45.828540 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:45.828501 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:27:52.145278 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:52.145238 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15c8ffd6-1e00-4975-b807-63e00ad36b44-metrics-certs\") pod \"network-metrics-daemon-qnp65\" (UID: \"15c8ffd6-1e00-4975-b807-63e00ad36b44\") " pod="openshift-multus/network-metrics-daemon-qnp65" Jul 06 01:27:52.145278 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:52.145288 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e690e9f2-b314-4e44-a211-433d7e8100a1-original-pull-secret\") pod \"global-pull-secret-syncer-xf5mp\" (UID: \"e690e9f2-b314-4e44-a211-433d7e8100a1\") " pod="kube-system/global-pull-secret-syncer-xf5mp" Jul 06 01:27:52.148188 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:52.148163 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 01:27:52.148300 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:52.148196 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 01:27:52.156329 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:52.156306 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 06 01:27:52.156449 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:52.156365 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15c8ffd6-1e00-4975-b807-63e00ad36b44-metrics-certs podName:15c8ffd6-1e00-4975-b807-63e00ad36b44 nodeName:}" failed. No retries permitted until 2026-07-06 01:28:56.156349573 +0000 UTC m=+130.127601243 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/15c8ffd6-1e00-4975-b807-63e00ad36b44-metrics-certs") pod "network-metrics-daemon-qnp65" (UID: "15c8ffd6-1e00-4975-b807-63e00ad36b44") : secret "metrics-daemon-secret" not found Jul 06 01:27:52.158607 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:52.158585 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e690e9f2-b314-4e44-a211-433d7e8100a1-original-pull-secret\") pod \"global-pull-secret-syncer-xf5mp\" (UID: \"e690e9f2-b314-4e44-a211-433d7e8100a1\") " pod="kube-system/global-pull-secret-syncer-xf5mp" Jul 06 01:27:52.246462 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:52.246395 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdkgm\" (UniqueName: \"kubernetes.io/projected/ab026023-6959-46fd-aaf6-0b145d5a956e-kube-api-access-tdkgm\") pod \"network-check-target-57drk\" (UID: \"ab026023-6959-46fd-aaf6-0b145d5a956e\") " pod="openshift-network-diagnostics/network-check-target-57drk" Jul 06 01:27:52.249332 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:52.249307 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 01:27:52.259800 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:52.259773 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 01:27:52.270884 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:52.270850 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdkgm\" (UniqueName: \"kubernetes.io/projected/ab026023-6959-46fd-aaf6-0b145d5a956e-kube-api-access-tdkgm\") pod \"network-check-target-57drk\" (UID: \"ab026023-6959-46fd-aaf6-0b145d5a956e\") " pod="openshift-network-diagnostics/network-check-target-57drk" Jul 06 01:27:52.378127 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:52.378090 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-n24x9\"" Jul 06 01:27:52.382378 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:52.382349 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xf5mp" Jul 06 01:27:52.385412 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:52.385397 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57drk" Jul 06 01:27:52.535037 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:52.535000 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-57drk"] Jul 06 01:27:52.544340 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:27:52.544310 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab026023_6959_46fd_aaf6_0b145d5a956e.slice/crio-ae716f5b7a5278e46108825ac977a246a033f8d05b280ce27f5edcce62a3ab66 WatchSource:0}: Error finding container ae716f5b7a5278e46108825ac977a246a033f8d05b280ce27f5edcce62a3ab66: Status 404 returned error can't find the container with id ae716f5b7a5278e46108825ac977a246a033f8d05b280ce27f5edcce62a3ab66 Jul 06 01:27:52.552639 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:52.552547 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xf5mp"] Jul 06 01:27:52.556022 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:27:52.556001 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode690e9f2_b314_4e44_a211_433d7e8100a1.slice/crio-07c3a2bb9150ea54ce1129e6e1bca461bf44ca5402ba2aa818392d2209efa195 WatchSource:0}: Error finding container 07c3a2bb9150ea54ce1129e6e1bca461bf44ca5402ba2aa818392d2209efa195: Status 404 returned error can't find the container with id 07c3a2bb9150ea54ce1129e6e1bca461bf44ca5402ba2aa818392d2209efa195 Jul 06 01:27:52.751008 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:52.750917 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5f07e4d-9391-4b17-8cd2-298dde338112-metrics-tls\") pod \"dns-default-dxjmj\" (UID: \"d5f07e4d-9391-4b17-8cd2-298dde338112\") " pod="openshift-dns/dns-default-dxjmj" Jul 06 01:27:52.751153 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:52.751082 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 01:27:52.751153 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:27:52.751148 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5f07e4d-9391-4b17-8cd2-298dde338112-metrics-tls podName:d5f07e4d-9391-4b17-8cd2-298dde338112 nodeName:}" failed. No retries permitted until 2026-07-06 01:28:24.751133117 +0000 UTC m=+98.722384789 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d5f07e4d-9391-4b17-8cd2-298dde338112-metrics-tls") pod "dns-default-dxjmj" (UID: "d5f07e4d-9391-4b17-8cd2-298dde338112") : secret "dns-default-metrics-tls" not found Jul 06 01:27:52.772792 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:52.772758 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xf5mp" event={"ID":"e690e9f2-b314-4e44-a211-433d7e8100a1","Type":"ContainerStarted","Data":"07c3a2bb9150ea54ce1129e6e1bca461bf44ca5402ba2aa818392d2209efa195"} Jul 06 01:27:52.773840 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:52.773815 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-57drk" event={"ID":"ab026023-6959-46fd-aaf6-0b145d5a956e","Type":"ContainerStarted","Data":"ae716f5b7a5278e46108825ac977a246a033f8d05b280ce27f5edcce62a3ab66"} Jul 06 01:27:57.792260 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:57.792217 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xf5mp" event={"ID":"e690e9f2-b314-4e44-a211-433d7e8100a1","Type":"ContainerStarted","Data":"a5ce936212f43c8a298a02b5ee84e768d9e676d4bdc7846b037dc4abae10f447"} Jul 06 01:27:57.793621 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:57.793599 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-57drk" event={"ID":"ab026023-6959-46fd-aaf6-0b145d5a956e","Type":"ContainerStarted","Data":"4cd4440ceeb7e1b43d4989d2e0e0f4d9c4880b6cdd1cce5740245c2624f24bcf"} Jul 06 01:27:57.793739 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:57.793722 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-57drk" Jul 06 01:27:57.806224 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:57.806178 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-xf5mp" podStartSLOduration=67.056203917 podStartE2EDuration="1m11.806162057s" totalImagesPullingTime="4.74995814s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 01:26:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:27:57.806105178 +0000 UTC m=+71.777356871" watchObservedRunningTime="2026-07-06 01:27:57.806162057 +0000 UTC m=+71.777413745" Jul 06 01:27:57.822179 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:27:57.822131 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-57drk" podStartSLOduration=67.065073221 podStartE2EDuration="1m11.822111582s" totalImagesPullingTime="4.757038361s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:26:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:27:57.821588744 +0000 UTC m=+71.792840435" watchObservedRunningTime="2026-07-06 01:27:57.822111582 +0000 UTC m=+71.793363273" Jul 06 01:28:16.498501 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:28:16.498306 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:28:16.551892 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:28:16.551864 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:28:24.775330 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:28:24.775286 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5f07e4d-9391-4b17-8cd2-298dde338112-metrics-tls\") pod \"dns-default-dxjmj\" (UID: \"d5f07e4d-9391-4b17-8cd2-298dde338112\") " pod="openshift-dns/dns-default-dxjmj" Jul 06 01:28:24.775765 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:28:24.775419 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 01:28:24.775765 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:28:24.775507 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5f07e4d-9391-4b17-8cd2-298dde338112-metrics-tls podName:d5f07e4d-9391-4b17-8cd2-298dde338112 nodeName:}" failed. No retries permitted until 2026-07-06 01:29:28.775492835 +0000 UTC m=+162.746744505 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d5f07e4d-9391-4b17-8cd2-298dde338112-metrics-tls") pod "dns-default-dxjmj" (UID: "d5f07e4d-9391-4b17-8cd2-298dde338112") : secret "dns-default-metrics-tls" not found Jul 06 01:28:28.800003 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:28:28.799971 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-57drk" Jul 06 01:28:56.178388 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:28:56.178347 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15c8ffd6-1e00-4975-b807-63e00ad36b44-metrics-certs\") pod \"network-metrics-daemon-qnp65\" (UID: \"15c8ffd6-1e00-4975-b807-63e00ad36b44\") " pod="openshift-multus/network-metrics-daemon-qnp65" Jul 06 01:28:56.178902 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:28:56.178536 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 06 01:28:56.178902 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:28:56.178611 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15c8ffd6-1e00-4975-b807-63e00ad36b44-metrics-certs podName:15c8ffd6-1e00-4975-b807-63e00ad36b44 nodeName:}" failed. No retries permitted until 2026-07-06 01:30:58.178591786 +0000 UTC m=+252.149843454 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/15c8ffd6-1e00-4975-b807-63e00ad36b44-metrics-certs") pod "network-metrics-daemon-qnp65" (UID: "15c8ffd6-1e00-4975-b807-63e00ad36b44") : secret "metrics-daemon-secret" not found Jul 06 01:29:23.893789 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:29:23.893728 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-dxjmj" podUID="d5f07e4d-9391-4b17-8cd2-298dde338112" Jul 06 01:29:24.045516 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:29:24.045485 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dxjmj" Jul 06 01:29:24.468108 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:29:24.468068 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-qnp65" podUID="15c8ffd6-1e00-4975-b807-63e00ad36b44" Jul 06 01:29:28.805885 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:29:28.805838 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5f07e4d-9391-4b17-8cd2-298dde338112-metrics-tls\") pod \"dns-default-dxjmj\" (UID: \"d5f07e4d-9391-4b17-8cd2-298dde338112\") " pod="openshift-dns/dns-default-dxjmj" Jul 06 01:29:28.806360 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:29:28.805967 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 01:29:28.806360 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:29:28.806036 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5f07e4d-9391-4b17-8cd2-298dde338112-metrics-tls podName:d5f07e4d-9391-4b17-8cd2-298dde338112 nodeName:}" failed. No retries permitted until 2026-07-06 01:31:30.806016773 +0000 UTC m=+284.777268459 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d5f07e4d-9391-4b17-8cd2-298dde338112-metrics-tls") pod "dns-default-dxjmj" (UID: "d5f07e4d-9391-4b17-8cd2-298dde338112") : secret "dns-default-metrics-tls" not found Jul 06 01:29:35.076301 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:29:35.076262 2580 generic.go:356] "Generic (PLEG): container finished" podID="235fdd76-2b3a-4698-b9e5-ab05305231c6" containerID="9b03f4ae5175a7f2890d6ab0062631b15365fb31216602b63315233188365873" exitCode=1 Jul 06 01:29:35.076718 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:29:35.076317 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5fd6d69f-4g6hq" event={"ID":"235fdd76-2b3a-4698-b9e5-ab05305231c6","Type":"ContainerDied","Data":"9b03f4ae5175a7f2890d6ab0062631b15365fb31216602b63315233188365873"} Jul 06 01:29:35.076718 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:29:35.076633 2580 scope.go:117] "RemoveContainer" containerID="9b03f4ae5175a7f2890d6ab0062631b15365fb31216602b63315233188365873" Jul 06 01:29:35.453516 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:29:35.453482 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qnp65" Jul 06 01:29:35.731410 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:29:35.731325 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5fd6d69f-4g6hq" Jul 06 01:29:36.080512 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:29:36.080402 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5fd6d69f-4g6hq" event={"ID":"235fdd76-2b3a-4698-b9e5-ab05305231c6","Type":"ContainerStarted","Data":"aa09d4b463b9e383c06864e2b918c3984efc818031acd3038e32f24ae732e2f2"} Jul 06 01:29:36.080915 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:29:36.080731 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5fd6d69f-4g6hq" Jul 06 01:29:36.081252 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:29:36.081234 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5fd6d69f-4g6hq" Jul 06 01:30:27.798929 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:30:27.798890 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:30:54.457781 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:30:54.457742 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:30:54.544046 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:30:54.544010 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:30:56.419753 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:30:56.419713 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:30:56.577927 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:30:56.577862 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:30:58.220421 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:30:58.220364 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15c8ffd6-1e00-4975-b807-63e00ad36b44-metrics-certs\") pod \"network-metrics-daemon-qnp65\" (UID: \"15c8ffd6-1e00-4975-b807-63e00ad36b44\") " pod="openshift-multus/network-metrics-daemon-qnp65" Jul 06 01:30:58.220938 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:30:58.220566 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 06 01:30:58.220938 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:30:58.220658 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15c8ffd6-1e00-4975-b807-63e00ad36b44-metrics-certs podName:15c8ffd6-1e00-4975-b807-63e00ad36b44 nodeName:}" failed. No retries permitted until 2026-07-06 01:33:00.220636649 +0000 UTC m=+374.191888319 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/15c8ffd6-1e00-4975-b807-63e00ad36b44-metrics-certs") pod "network-metrics-daemon-qnp65" (UID: "15c8ffd6-1e00-4975-b807-63e00ad36b44") : secret "metrics-daemon-secret" not found Jul 06 01:30:58.317238 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:30:58.317209 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-fcdfg"] Jul 06 01:30:58.320110 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:30:58.320087 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fcdfg" Jul 06 01:30:58.323463 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:30:58.323431 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 06 01:30:58.324049 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:30:58.324031 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 06 01:30:58.324146 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:30:58.324135 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 06 01:30:58.324468 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:30:58.324452 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 06 01:30:58.327170 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:30:58.327151 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9xpc9\"" Jul 06 01:30:58.337096 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:30:58.337070 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fcdfg"] Jul 06 01:30:58.370709 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:30:58.370675 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:30:58.421606 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:30:58.421563 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/227d7764-6d83-4e82-953d-e33128c2c10a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fcdfg\" (UID: \"227d7764-6d83-4e82-953d-e33128c2c10a\") " pod="openshift-insights/insights-runtime-extractor-fcdfg" Jul 06 01:30:58.421606 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:30:58.421601 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j85pt\" (UniqueName: \"kubernetes.io/projected/227d7764-6d83-4e82-953d-e33128c2c10a-kube-api-access-j85pt\") pod \"insights-runtime-extractor-fcdfg\" (UID: \"227d7764-6d83-4e82-953d-e33128c2c10a\") " pod="openshift-insights/insights-runtime-extractor-fcdfg" Jul 06 01:30:58.421806 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:30:58.421687 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/227d7764-6d83-4e82-953d-e33128c2c10a-crio-socket\") pod \"insights-runtime-extractor-fcdfg\" (UID: \"227d7764-6d83-4e82-953d-e33128c2c10a\") " pod="openshift-insights/insights-runtime-extractor-fcdfg" Jul 06 01:30:58.421806 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:30:58.421717 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/227d7764-6d83-4e82-953d-e33128c2c10a-data-volume\") pod \"insights-runtime-extractor-fcdfg\" (UID: \"227d7764-6d83-4e82-953d-e33128c2c10a\") " pod="openshift-insights/insights-runtime-extractor-fcdfg" Jul 06 01:30:58.421806 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:30:58.421733 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/227d7764-6d83-4e82-953d-e33128c2c10a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fcdfg\" (UID: \"227d7764-6d83-4e82-953d-e33128c2c10a\") " pod="openshift-insights/insights-runtime-extractor-fcdfg" Jul 06 01:30:58.452580 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:30:58.452542 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:30:58.523020 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:30:58.522915 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/227d7764-6d83-4e82-953d-e33128c2c10a-crio-socket\") pod \"insights-runtime-extractor-fcdfg\" (UID: \"227d7764-6d83-4e82-953d-e33128c2c10a\") " pod="openshift-insights/insights-runtime-extractor-fcdfg" Jul 06 01:30:58.523020 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:30:58.522956 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/227d7764-6d83-4e82-953d-e33128c2c10a-data-volume\") pod \"insights-runtime-extractor-fcdfg\" (UID: \"227d7764-6d83-4e82-953d-e33128c2c10a\") " pod="openshift-insights/insights-runtime-extractor-fcdfg" Jul 06 01:30:58.523020 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:30:58.522973 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/227d7764-6d83-4e82-953d-e33128c2c10a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fcdfg\" (UID: \"227d7764-6d83-4e82-953d-e33128c2c10a\") " pod="openshift-insights/insights-runtime-extractor-fcdfg" Jul 06 01:30:58.523371 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:30:58.523030 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/227d7764-6d83-4e82-953d-e33128c2c10a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fcdfg\" (UID: \"227d7764-6d83-4e82-953d-e33128c2c10a\") " pod="openshift-insights/insights-runtime-extractor-fcdfg" Jul 06 01:30:58.523371 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:30:58.523047 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j85pt\" (UniqueName: \"kubernetes.io/projected/227d7764-6d83-4e82-953d-e33128c2c10a-kube-api-access-j85pt\") pod \"insights-runtime-extractor-fcdfg\" (UID: \"227d7764-6d83-4e82-953d-e33128c2c10a\") " pod="openshift-insights/insights-runtime-extractor-fcdfg" Jul 06 01:30:58.523371 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:30:58.523050 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/227d7764-6d83-4e82-953d-e33128c2c10a-crio-socket\") pod \"insights-runtime-extractor-fcdfg\" (UID: \"227d7764-6d83-4e82-953d-e33128c2c10a\") " pod="openshift-insights/insights-runtime-extractor-fcdfg" Jul 06 01:30:58.523371 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:30:58.523095 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 01:30:58.523371 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:30:58.523156 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 01:30:58.523371 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:30:58.523170 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/227d7764-6d83-4e82-953d-e33128c2c10a-insights-runtime-extractor-tls podName:227d7764-6d83-4e82-953d-e33128c2c10a nodeName:}" failed. No retries permitted until 2026-07-06 01:30:59.02314797 +0000 UTC m=+252.994399638 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/227d7764-6d83-4e82-953d-e33128c2c10a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fcdfg" (UID: "227d7764-6d83-4e82-953d-e33128c2c10a") : secret "insights-runtime-extractor-tls" not found Jul 06 01:30:58.523371 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:30:58.523213 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/227d7764-6d83-4e82-953d-e33128c2c10a-kube-rbac-proxy-cm podName:227d7764-6d83-4e82-953d-e33128c2c10a nodeName:}" failed. No retries permitted until 2026-07-06 01:30:59.023193062 +0000 UTC m=+252.994444732 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/227d7764-6d83-4e82-953d-e33128c2c10a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fcdfg" (UID: "227d7764-6d83-4e82-953d-e33128c2c10a") : configmap "kube-rbac-proxy" not found Jul 06 01:30:58.523371 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:30:58.523305 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/227d7764-6d83-4e82-953d-e33128c2c10a-data-volume\") pod \"insights-runtime-extractor-fcdfg\" (UID: \"227d7764-6d83-4e82-953d-e33128c2c10a\") " pod="openshift-insights/insights-runtime-extractor-fcdfg" Jul 06 01:30:58.533718 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:30:58.533695 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j85pt\" (UniqueName: \"kubernetes.io/projected/227d7764-6d83-4e82-953d-e33128c2c10a-kube-api-access-j85pt\") pod \"insights-runtime-extractor-fcdfg\" (UID: \"227d7764-6d83-4e82-953d-e33128c2c10a\") " pod="openshift-insights/insights-runtime-extractor-fcdfg" Jul 06 01:30:58.841112 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:30:58.841028 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:30:58.977849 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:30:58.977811 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:30:59.026816 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:30:59.026777 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/227d7764-6d83-4e82-953d-e33128c2c10a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fcdfg\" (UID: \"227d7764-6d83-4e82-953d-e33128c2c10a\") " pod="openshift-insights/insights-runtime-extractor-fcdfg" Jul 06 01:30:59.026973 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:30:59.026847 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/227d7764-6d83-4e82-953d-e33128c2c10a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fcdfg\" (UID: \"227d7764-6d83-4e82-953d-e33128c2c10a\") " pod="openshift-insights/insights-runtime-extractor-fcdfg" Jul 06 01:30:59.026973 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:30:59.026938 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 01:30:59.026973 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:30:59.026959 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 01:30:59.027114 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:30:59.027015 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/227d7764-6d83-4e82-953d-e33128c2c10a-insights-runtime-extractor-tls podName:227d7764-6d83-4e82-953d-e33128c2c10a nodeName:}" failed. No retries permitted until 2026-07-06 01:31:00.026996495 +0000 UTC m=+253.998248184 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/227d7764-6d83-4e82-953d-e33128c2c10a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fcdfg" (UID: "227d7764-6d83-4e82-953d-e33128c2c10a") : secret "insights-runtime-extractor-tls" not found Jul 06 01:30:59.027114 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:30:59.027043 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/227d7764-6d83-4e82-953d-e33128c2c10a-kube-rbac-proxy-cm podName:227d7764-6d83-4e82-953d-e33128c2c10a nodeName:}" failed. No retries permitted until 2026-07-06 01:31:00.027024627 +0000 UTC m=+253.998276310 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/227d7764-6d83-4e82-953d-e33128c2c10a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fcdfg" (UID: "227d7764-6d83-4e82-953d-e33128c2c10a") : configmap "kube-rbac-proxy" not found Jul 06 01:30:59.078641 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:30:59.078603 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:30:59.162121 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:30:59.162087 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:30:59.709878 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:30:59.709832 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:30:59.810745 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:30:59.810708 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:00.033689 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:00.033601 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/227d7764-6d83-4e82-953d-e33128c2c10a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fcdfg\" (UID: \"227d7764-6d83-4e82-953d-e33128c2c10a\") " pod="openshift-insights/insights-runtime-extractor-fcdfg" Jul 06 01:31:00.033689 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:00.033649 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/227d7764-6d83-4e82-953d-e33128c2c10a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fcdfg\" (UID: \"227d7764-6d83-4e82-953d-e33128c2c10a\") " pod="openshift-insights/insights-runtime-extractor-fcdfg" Jul 06 01:31:00.033882 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:31:00.033718 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 01:31:00.033882 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:31:00.033751 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 01:31:00.033882 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:31:00.033771 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/227d7764-6d83-4e82-953d-e33128c2c10a-kube-rbac-proxy-cm podName:227d7764-6d83-4e82-953d-e33128c2c10a nodeName:}" failed. No retries permitted until 2026-07-06 01:31:02.033758119 +0000 UTC m=+256.005009788 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/227d7764-6d83-4e82-953d-e33128c2c10a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fcdfg" (UID: "227d7764-6d83-4e82-953d-e33128c2c10a") : configmap "kube-rbac-proxy" not found Jul 06 01:31:00.033882 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:31:00.033804 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/227d7764-6d83-4e82-953d-e33128c2c10a-insights-runtime-extractor-tls podName:227d7764-6d83-4e82-953d-e33128c2c10a nodeName:}" failed. No retries permitted until 2026-07-06 01:31:02.033791738 +0000 UTC m=+256.005043408 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/227d7764-6d83-4e82-953d-e33128c2c10a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fcdfg" (UID: "227d7764-6d83-4e82-953d-e33128c2c10a") : secret "insights-runtime-extractor-tls" not found Jul 06 01:31:00.289175 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:00.289102 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-82mwf_00a6f863-8ae0-4749-8f27-3bdf91cd8775/dns-node-resolver/0.log" Jul 06 01:31:01.292137 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:01.292110 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-b46kx_4164d767-f4d0-459d-b0c1-cc8f8ccbaf1a/node-ca/0.log" Jul 06 01:31:01.339721 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:01.339687 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:01.401927 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:01.401891 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:01.502931 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:01.502891 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:02.048251 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:02.048208 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/227d7764-6d83-4e82-953d-e33128c2c10a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fcdfg\" (UID: \"227d7764-6d83-4e82-953d-e33128c2c10a\") " pod="openshift-insights/insights-runtime-extractor-fcdfg" Jul 06 01:31:02.048459 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:02.048263 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/227d7764-6d83-4e82-953d-e33128c2c10a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fcdfg\" (UID: \"227d7764-6d83-4e82-953d-e33128c2c10a\") " pod="openshift-insights/insights-runtime-extractor-fcdfg" Jul 06 01:31:02.048459 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:31:02.048330 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 01:31:02.048459 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:31:02.048367 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 01:31:02.048459 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:31:02.048386 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/227d7764-6d83-4e82-953d-e33128c2c10a-kube-rbac-proxy-cm podName:227d7764-6d83-4e82-953d-e33128c2c10a nodeName:}" failed. No retries permitted until 2026-07-06 01:31:06.048372811 +0000 UTC m=+260.019624480 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/227d7764-6d83-4e82-953d-e33128c2c10a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fcdfg" (UID: "227d7764-6d83-4e82-953d-e33128c2c10a") : configmap "kube-rbac-proxy" not found Jul 06 01:31:02.048459 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:31:02.048461 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/227d7764-6d83-4e82-953d-e33128c2c10a-insights-runtime-extractor-tls podName:227d7764-6d83-4e82-953d-e33128c2c10a nodeName:}" failed. No retries permitted until 2026-07-06 01:31:06.048428478 +0000 UTC m=+260.019680147 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/227d7764-6d83-4e82-953d-e33128c2c10a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fcdfg" (UID: "227d7764-6d83-4e82-953d-e33128c2c10a") : secret "insights-runtime-extractor-tls" not found Jul 06 01:31:02.207591 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:02.207560 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:02.291280 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:02.291244 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-v7w8q_d90b9b62-2b7e-439a-9d24-4893aa5543d9/serve-healthcheck-canary/0.log" Jul 06 01:31:03.170851 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:03.170820 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:03.401225 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:03.401184 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:03.532224 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:03.532139 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:03.633209 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:03.633157 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:03.959335 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:03.959296 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:04.085230 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:04.085190 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:04.194805 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:04.194764 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:04.308601 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:04.308517 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:04.603333 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:04.603240 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:04.706167 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:04.706129 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:05.665691 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:05.665651 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:06.080341 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:06.080243 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/227d7764-6d83-4e82-953d-e33128c2c10a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fcdfg\" (UID: \"227d7764-6d83-4e82-953d-e33128c2c10a\") " pod="openshift-insights/insights-runtime-extractor-fcdfg" Jul 06 01:31:06.080341 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:06.080295 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/227d7764-6d83-4e82-953d-e33128c2c10a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fcdfg\" (UID: \"227d7764-6d83-4e82-953d-e33128c2c10a\") " pod="openshift-insights/insights-runtime-extractor-fcdfg" Jul 06 01:31:06.080623 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:31:06.080411 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 01:31:06.080623 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:31:06.080496 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 01:31:06.080623 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:31:06.080536 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/227d7764-6d83-4e82-953d-e33128c2c10a-insights-runtime-extractor-tls podName:227d7764-6d83-4e82-953d-e33128c2c10a nodeName:}" failed. No retries permitted until 2026-07-06 01:31:14.080516068 +0000 UTC m=+268.051767743 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/227d7764-6d83-4e82-953d-e33128c2c10a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fcdfg" (UID: "227d7764-6d83-4e82-953d-e33128c2c10a") : secret "insights-runtime-extractor-tls" not found Jul 06 01:31:06.080750 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:31:06.080627 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/227d7764-6d83-4e82-953d-e33128c2c10a-kube-rbac-proxy-cm podName:227d7764-6d83-4e82-953d-e33128c2c10a nodeName:}" failed. No retries permitted until 2026-07-06 01:31:14.080613372 +0000 UTC m=+268.051865047 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/227d7764-6d83-4e82-953d-e33128c2c10a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fcdfg" (UID: "227d7764-6d83-4e82-953d-e33128c2c10a") : configmap "kube-rbac-proxy" not found Jul 06 01:31:06.090817 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:06.090774 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:06.270703 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:06.270671 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:06.464569 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:06.464529 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:06.910770 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:06.910689 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:07.064417 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:07.064375 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:07.256925 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:07.256806 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:07.666993 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:07.666930 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:08.098548 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:08.098460 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:08.187279 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:08.187246 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:08.287373 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:08.287335 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:08.471237 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:08.471194 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:08.668340 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:08.668302 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:09.473569 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:09.473532 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:09.770302 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:09.770211 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:09.871948 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:09.871912 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:10.071632 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:10.071544 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:10.668609 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:10.668578 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:11.067250 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:11.067164 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:11.266743 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:11.266700 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:11.571638 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:11.571603 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:11.672353 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:11.672324 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:12.263923 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:12.263879 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:12.863032 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:12.862992 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:13.064244 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:13.064215 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:13.263687 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:13.263653 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:13.664229 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:13.664200 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:13.864523 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:13.864485 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:14.143390 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:14.143354 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/227d7764-6d83-4e82-953d-e33128c2c10a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fcdfg\" (UID: \"227d7764-6d83-4e82-953d-e33128c2c10a\") " pod="openshift-insights/insights-runtime-extractor-fcdfg" Jul 06 01:31:14.143608 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:14.143418 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/227d7764-6d83-4e82-953d-e33128c2c10a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fcdfg\" (UID: \"227d7764-6d83-4e82-953d-e33128c2c10a\") " pod="openshift-insights/insights-runtime-extractor-fcdfg" Jul 06 01:31:14.143955 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:14.143930 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/227d7764-6d83-4e82-953d-e33128c2c10a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fcdfg\" (UID: \"227d7764-6d83-4e82-953d-e33128c2c10a\") " pod="openshift-insights/insights-runtime-extractor-fcdfg" Jul 06 01:31:14.145920 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:14.145902 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/227d7764-6d83-4e82-953d-e33128c2c10a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fcdfg\" (UID: \"227d7764-6d83-4e82-953d-e33128c2c10a\") " pod="openshift-insights/insights-runtime-extractor-fcdfg" Jul 06 01:31:14.229195 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:14.229154 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fcdfg" Jul 06 01:31:14.356648 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:14.356610 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fcdfg"] Jul 06 01:31:14.360137 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:31:14.360095 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod227d7764_6d83_4e82_953d_e33128c2c10a.slice/crio-dc46502b665b230f21e6d71626ab8682af4358090d977fa8b05555a3f3540432 WatchSource:0}: Error finding container dc46502b665b230f21e6d71626ab8682af4358090d977fa8b05555a3f3540432: Status 404 returned error can't find the container with id dc46502b665b230f21e6d71626ab8682af4358090d977fa8b05555a3f3540432 Jul 06 01:31:15.362935 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:15.362849 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fcdfg" event={"ID":"227d7764-6d83-4e82-953d-e33128c2c10a","Type":"ContainerStarted","Data":"dc46502b665b230f21e6d71626ab8682af4358090d977fa8b05555a3f3540432"} Jul 06 01:31:18.372614 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:18.372574 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fcdfg" event={"ID":"227d7764-6d83-4e82-953d-e33128c2c10a","Type":"ContainerStarted","Data":"120b3b728303d05d50f34bc63540489ed776834474dc00404aeb8c934cb27d3c"} Jul 06 01:31:19.051877 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:19.051784 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:19.153066 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:19.153020 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:19.214614 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:19.214577 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:19.353605 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:19.353516 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:19.378324 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:19.378280 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fcdfg" event={"ID":"227d7764-6d83-4e82-953d-e33128c2c10a","Type":"ContainerStarted","Data":"c1718e30c9f01460767da1a00a426d7803e7afd1a4aa586edf1d527cd7103dae"} Jul 06 01:31:19.724347 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:19.724298 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:19.826602 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:19.826560 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:19.926637 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:19.926582 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:19.996955 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:19.996814 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:20.134872 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:20.134836 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:20.234578 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:20.234539 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:20.335106 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:20.335014 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:20.393409 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:20.393331 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:20.537533 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:20.537484 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:20.740148 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:20.740112 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:21.095274 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:21.095180 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:21.386312 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:21.386276 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fcdfg" event={"ID":"227d7764-6d83-4e82-953d-e33128c2c10a","Type":"ContainerStarted","Data":"e3ce4dfe7dca7c79ce911b9142aeb28b05894a47b6e100daa38e2d5ad2e79329"} Jul 06 01:31:21.406176 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:21.406124 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-fcdfg" podStartSLOduration=16.658560945 podStartE2EDuration="23.406106253s" totalImagesPullingTime="6.747545308s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 01:30:58 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:31:21.405133972 +0000 UTC m=+275.376385663" watchObservedRunningTime="2026-07-06 01:31:21.406106253 +0000 UTC m=+275.377357943" Jul 06 01:31:22.552304 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:22.552272 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:22.653288 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:22.653254 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:24.134276 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:24.134243 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-68kkp"] Jul 06 01:31:24.137332 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:24.137316 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-68kkp" Jul 06 01:31:24.140827 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:24.140796 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-ppqh5\"" Jul 06 01:31:24.141056 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:24.141039 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 06 01:31:24.141108 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:24.141079 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 06 01:31:24.149297 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:24.149100 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-68kkp"] Jul 06 01:31:24.197488 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:24.197450 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-67887675c-sjf6n"] Jul 06 01:31:24.200419 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:24.200401 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67887675c-sjf6n" Jul 06 01:31:24.203270 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:24.203250 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-qzthl\"" Jul 06 01:31:24.203389 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:24.203324 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 06 01:31:24.203613 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:24.203596 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 06 01:31:24.203742 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:24.203726 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 06 01:31:24.213819 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:24.213794 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bcdcad6b-89cf-4818-9365-9b521ed419f3-registry-tls\") pod \"image-registry-67887675c-sjf6n\" (UID: \"bcdcad6b-89cf-4818-9365-9b521ed419f3\") " pod="openshift-image-registry/image-registry-67887675c-sjf6n" Jul 06 01:31:24.213962 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:24.213823 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bcdcad6b-89cf-4818-9365-9b521ed419f3-registry-certificates\") pod \"image-registry-67887675c-sjf6n\" (UID: \"bcdcad6b-89cf-4818-9365-9b521ed419f3\") " pod="openshift-image-registry/image-registry-67887675c-sjf6n" Jul 06 01:31:24.213962 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:24.213847 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bcdcad6b-89cf-4818-9365-9b521ed419f3-ca-trust-extracted\") pod \"image-registry-67887675c-sjf6n\" (UID: \"bcdcad6b-89cf-4818-9365-9b521ed419f3\") " pod="openshift-image-registry/image-registry-67887675c-sjf6n" Jul 06 01:31:24.213962 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:24.213864 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bcdcad6b-89cf-4818-9365-9b521ed419f3-trusted-ca\") pod \"image-registry-67887675c-sjf6n\" (UID: \"bcdcad6b-89cf-4818-9365-9b521ed419f3\") " pod="openshift-image-registry/image-registry-67887675c-sjf6n" Jul 06 01:31:24.213962 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:24.213931 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bcdcad6b-89cf-4818-9365-9b521ed419f3-bound-sa-token\") pod \"image-registry-67887675c-sjf6n\" (UID: \"bcdcad6b-89cf-4818-9365-9b521ed419f3\") " pod="openshift-image-registry/image-registry-67887675c-sjf6n" Jul 06 01:31:24.214177 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:24.213991 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qg2zg\" (UniqueName: \"kubernetes.io/projected/bcdcad6b-89cf-4818-9365-9b521ed419f3-kube-api-access-qg2zg\") pod \"image-registry-67887675c-sjf6n\" (UID: \"bcdcad6b-89cf-4818-9365-9b521ed419f3\") " pod="openshift-image-registry/image-registry-67887675c-sjf6n" Jul 06 01:31:24.214177 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:24.214017 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 06 01:31:24.214177 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:24.214020 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8f9gp\" (UniqueName: \"kubernetes.io/projected/108029ee-5264-4c60-91b5-a390aea7fe33-kube-api-access-8f9gp\") pod \"downloads-685bfb5f64-68kkp\" (UID: \"108029ee-5264-4c60-91b5-a390aea7fe33\") " pod="openshift-console/downloads-685bfb5f64-68kkp" Jul 06 01:31:24.214177 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:24.214064 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bcdcad6b-89cf-4818-9365-9b521ed419f3-installation-pull-secrets\") pod \"image-registry-67887675c-sjf6n\" (UID: \"bcdcad6b-89cf-4818-9365-9b521ed419f3\") " pod="openshift-image-registry/image-registry-67887675c-sjf6n" Jul 06 01:31:24.214177 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:24.214122 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bcdcad6b-89cf-4818-9365-9b521ed419f3-image-registry-private-configuration\") pod \"image-registry-67887675c-sjf6n\" (UID: \"bcdcad6b-89cf-4818-9365-9b521ed419f3\") " pod="openshift-image-registry/image-registry-67887675c-sjf6n" Jul 06 01:31:24.218611 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:24.218588 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-67887675c-sjf6n"] Jul 06 01:31:24.246204 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:24.246164 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:24.314619 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:24.314576 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bcdcad6b-89cf-4818-9365-9b521ed419f3-bound-sa-token\") pod \"image-registry-67887675c-sjf6n\" (UID: \"bcdcad6b-89cf-4818-9365-9b521ed419f3\") " pod="openshift-image-registry/image-registry-67887675c-sjf6n" Jul 06 01:31:24.314812 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:24.314632 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qg2zg\" (UniqueName: \"kubernetes.io/projected/bcdcad6b-89cf-4818-9365-9b521ed419f3-kube-api-access-qg2zg\") pod \"image-registry-67887675c-sjf6n\" (UID: \"bcdcad6b-89cf-4818-9365-9b521ed419f3\") " pod="openshift-image-registry/image-registry-67887675c-sjf6n" Jul 06 01:31:24.314812 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:24.314652 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8f9gp\" (UniqueName: \"kubernetes.io/projected/108029ee-5264-4c60-91b5-a390aea7fe33-kube-api-access-8f9gp\") pod \"downloads-685bfb5f64-68kkp\" (UID: \"108029ee-5264-4c60-91b5-a390aea7fe33\") " pod="openshift-console/downloads-685bfb5f64-68kkp" Jul 06 01:31:24.314812 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:24.314668 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bcdcad6b-89cf-4818-9365-9b521ed419f3-installation-pull-secrets\") pod \"image-registry-67887675c-sjf6n\" (UID: \"bcdcad6b-89cf-4818-9365-9b521ed419f3\") " pod="openshift-image-registry/image-registry-67887675c-sjf6n" Jul 06 01:31:24.314812 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:24.314704 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bcdcad6b-89cf-4818-9365-9b521ed419f3-image-registry-private-configuration\") pod \"image-registry-67887675c-sjf6n\" (UID: \"bcdcad6b-89cf-4818-9365-9b521ed419f3\") " pod="openshift-image-registry/image-registry-67887675c-sjf6n" Jul 06 01:31:24.314812 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:24.314726 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bcdcad6b-89cf-4818-9365-9b521ed419f3-registry-tls\") pod \"image-registry-67887675c-sjf6n\" (UID: \"bcdcad6b-89cf-4818-9365-9b521ed419f3\") " pod="openshift-image-registry/image-registry-67887675c-sjf6n" Jul 06 01:31:24.314812 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:24.314740 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bcdcad6b-89cf-4818-9365-9b521ed419f3-registry-certificates\") pod \"image-registry-67887675c-sjf6n\" (UID: \"bcdcad6b-89cf-4818-9365-9b521ed419f3\") " pod="openshift-image-registry/image-registry-67887675c-sjf6n" Jul 06 01:31:24.314812 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:24.314776 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bcdcad6b-89cf-4818-9365-9b521ed419f3-ca-trust-extracted\") pod \"image-registry-67887675c-sjf6n\" (UID: \"bcdcad6b-89cf-4818-9365-9b521ed419f3\") " pod="openshift-image-registry/image-registry-67887675c-sjf6n" Jul 06 01:31:24.314812 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:24.314800 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bcdcad6b-89cf-4818-9365-9b521ed419f3-trusted-ca\") pod \"image-registry-67887675c-sjf6n\" (UID: \"bcdcad6b-89cf-4818-9365-9b521ed419f3\") " pod="openshift-image-registry/image-registry-67887675c-sjf6n" Jul 06 01:31:24.315320 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:24.315298 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bcdcad6b-89cf-4818-9365-9b521ed419f3-ca-trust-extracted\") pod \"image-registry-67887675c-sjf6n\" (UID: \"bcdcad6b-89cf-4818-9365-9b521ed419f3\") " pod="openshift-image-registry/image-registry-67887675c-sjf6n" Jul 06 01:31:24.315666 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:24.315645 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bcdcad6b-89cf-4818-9365-9b521ed419f3-registry-certificates\") pod \"image-registry-67887675c-sjf6n\" (UID: \"bcdcad6b-89cf-4818-9365-9b521ed419f3\") " pod="openshift-image-registry/image-registry-67887675c-sjf6n" Jul 06 01:31:24.315806 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:24.315785 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bcdcad6b-89cf-4818-9365-9b521ed419f3-trusted-ca\") pod \"image-registry-67887675c-sjf6n\" (UID: \"bcdcad6b-89cf-4818-9365-9b521ed419f3\") " pod="openshift-image-registry/image-registry-67887675c-sjf6n" Jul 06 01:31:24.317240 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:24.317221 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bcdcad6b-89cf-4818-9365-9b521ed419f3-installation-pull-secrets\") pod \"image-registry-67887675c-sjf6n\" (UID: \"bcdcad6b-89cf-4818-9365-9b521ed419f3\") " pod="openshift-image-registry/image-registry-67887675c-sjf6n" Jul 06 01:31:24.317855 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:24.317834 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bcdcad6b-89cf-4818-9365-9b521ed419f3-registry-tls\") pod \"image-registry-67887675c-sjf6n\" (UID: \"bcdcad6b-89cf-4818-9365-9b521ed419f3\") " pod="openshift-image-registry/image-registry-67887675c-sjf6n" Jul 06 01:31:24.317976 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:24.317955 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bcdcad6b-89cf-4818-9365-9b521ed419f3-image-registry-private-configuration\") pod \"image-registry-67887675c-sjf6n\" (UID: \"bcdcad6b-89cf-4818-9365-9b521ed419f3\") " pod="openshift-image-registry/image-registry-67887675c-sjf6n" Jul 06 01:31:24.325838 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:24.325807 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bcdcad6b-89cf-4818-9365-9b521ed419f3-bound-sa-token\") pod \"image-registry-67887675c-sjf6n\" (UID: \"bcdcad6b-89cf-4818-9365-9b521ed419f3\") " pod="openshift-image-registry/image-registry-67887675c-sjf6n" Jul 06 01:31:24.326100 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:24.326083 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8f9gp\" (UniqueName: \"kubernetes.io/projected/108029ee-5264-4c60-91b5-a390aea7fe33-kube-api-access-8f9gp\") pod \"downloads-685bfb5f64-68kkp\" (UID: \"108029ee-5264-4c60-91b5-a390aea7fe33\") " pod="openshift-console/downloads-685bfb5f64-68kkp" Jul 06 01:31:24.329288 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:24.329262 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qg2zg\" (UniqueName: \"kubernetes.io/projected/bcdcad6b-89cf-4818-9365-9b521ed419f3-kube-api-access-qg2zg\") pod \"image-registry-67887675c-sjf6n\" (UID: \"bcdcad6b-89cf-4818-9365-9b521ed419f3\") " pod="openshift-image-registry/image-registry-67887675c-sjf6n" Jul 06 01:31:24.340178 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:24.340141 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:24.446379 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:24.446343 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-68kkp" Jul 06 01:31:24.509650 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:24.509594 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67887675c-sjf6n" Jul 06 01:31:24.586589 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:24.586548 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-68kkp"] Jul 06 01:31:24.590825 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:31:24.590796 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod108029ee_5264_4c60_91b5_a390aea7fe33.slice/crio-9d6923e76e1162470268cd00554404de32fbc8588deb5613ee1a30efefa5f944 WatchSource:0}: Error finding container 9d6923e76e1162470268cd00554404de32fbc8588deb5613ee1a30efefa5f944: Status 404 returned error can't find the container with id 9d6923e76e1162470268cd00554404de32fbc8588deb5613ee1a30efefa5f944 Jul 06 01:31:24.648663 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:24.648631 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-67887675c-sjf6n"] Jul 06 01:31:24.651291 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:31:24.651268 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcdcad6b_89cf_4818_9365_9b521ed419f3.slice/crio-611881b456c7c7d930388ff9249717228a25ca044672fb413910eb19d38678f5 WatchSource:0}: Error finding container 611881b456c7c7d930388ff9249717228a25ca044672fb413910eb19d38678f5: Status 404 returned error can't find the container with id 611881b456c7c7d930388ff9249717228a25ca044672fb413910eb19d38678f5 Jul 06 01:31:25.399902 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:25.399850 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67887675c-sjf6n" event={"ID":"bcdcad6b-89cf-4818-9365-9b521ed419f3","Type":"ContainerStarted","Data":"682147d6d367c70dce91f3af44c8f8783c7edd1047411dd6deb21279224636f6"} Jul 06 01:31:25.400404 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:25.399916 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67887675c-sjf6n" event={"ID":"bcdcad6b-89cf-4818-9365-9b521ed419f3","Type":"ContainerStarted","Data":"611881b456c7c7d930388ff9249717228a25ca044672fb413910eb19d38678f5"} Jul 06 01:31:25.400404 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:25.399973 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-67887675c-sjf6n" Jul 06 01:31:25.401157 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:25.401131 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-68kkp" event={"ID":"108029ee-5264-4c60-91b5-a390aea7fe33","Type":"ContainerStarted","Data":"9d6923e76e1162470268cd00554404de32fbc8588deb5613ee1a30efefa5f944"} Jul 06 01:31:25.422761 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:25.422706 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-67887675c-sjf6n" podStartSLOduration=1.422685128 podStartE2EDuration="1.422685128s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:31:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:31:25.422209438 +0000 UTC m=+279.393461130" watchObservedRunningTime="2026-07-06 01:31:25.422685128 +0000 UTC m=+279.393936820" Jul 06 01:31:27.046875 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:31:27.046824 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-dxjmj" podUID="d5f07e4d-9391-4b17-8cd2-298dde338112" Jul 06 01:31:27.407818 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:27.407786 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dxjmj" Jul 06 01:31:28.577843 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:28.577796 2580 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-658bcdcb4f-c6mtj" podUID="1b429a9e-ffde-4404-a881-e2794a4bbd48" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 06 01:31:28.597972 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:28.597930 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:28.654030 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:28.653994 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:28.732394 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:28.732356 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:28.938389 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:28.938348 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:30.864761 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:30.864723 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5f07e4d-9391-4b17-8cd2-298dde338112-metrics-tls\") pod \"dns-default-dxjmj\" (UID: \"d5f07e4d-9391-4b17-8cd2-298dde338112\") " pod="openshift-dns/dns-default-dxjmj" Jul 06 01:31:30.867621 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:30.867595 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5f07e4d-9391-4b17-8cd2-298dde338112-metrics-tls\") pod \"dns-default-dxjmj\" (UID: \"d5f07e4d-9391-4b17-8cd2-298dde338112\") " pod="openshift-dns/dns-default-dxjmj" Jul 06 01:31:31.011593 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:31.011556 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-58ndz\"" Jul 06 01:31:31.020001 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:31.019970 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dxjmj" Jul 06 01:31:31.171172 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:31.171136 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dxjmj"] Jul 06 01:31:31.174505 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:31:31.174468 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5f07e4d_9391_4b17_8cd2_298dde338112.slice/crio-cec56e182b694679bb40128412ba7539cbd2eee380ba6a818008e9e16ab387fb WatchSource:0}: Error finding container cec56e182b694679bb40128412ba7539cbd2eee380ba6a818008e9e16ab387fb: Status 404 returned error can't find the container with id cec56e182b694679bb40128412ba7539cbd2eee380ba6a818008e9e16ab387fb Jul 06 01:31:31.423567 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:31.423522 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dxjmj" event={"ID":"d5f07e4d-9391-4b17-8cd2-298dde338112","Type":"ContainerStarted","Data":"cec56e182b694679bb40128412ba7539cbd2eee380ba6a818008e9e16ab387fb"} Jul 06 01:31:31.756740 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:31.756619 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:31.832387 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:31.832352 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:32.124191 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:32.124089 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:33.433577 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:33.433535 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dxjmj" event={"ID":"d5f07e4d-9391-4b17-8cd2-298dde338112","Type":"ContainerStarted","Data":"f6337bfe904ca5a2ab30a9e75875857f2b80a38012433c553290d748b58004cd"} Jul 06 01:31:33.433999 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:33.433601 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dxjmj" event={"ID":"d5f07e4d-9391-4b17-8cd2-298dde338112","Type":"ContainerStarted","Data":"ef1f6bf325a1b41c2b7e9c33a04e191a5b133b954b43adccc8f86a8b2b453aca"} Jul 06 01:31:33.433999 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:33.433657 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-dxjmj" Jul 06 01:31:33.450582 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:33.450531 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-dxjmj" podStartSLOduration=251.77865483 podStartE2EDuration="4m13.45051405s" totalImagesPullingTime="1.67185922s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:27:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:31:33.449203617 +0000 UTC m=+287.420455309" watchObservedRunningTime="2026-07-06 01:31:33.45051405 +0000 UTC m=+287.421765740" Jul 06 01:31:34.514979 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:34.514918 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:35.994592 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:35.994554 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:36.081979 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:36.081938 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5fd6d69f-4g6hq" podUID="235fdd76-2b3a-4698-b9e5-ab05305231c6" containerName="acm-agent" probeResult="failure" output="Get \"http://10.134.0.8:8000/readyz\": dial tcp 10.134.0.8:8000: connect: connection refused" Jul 06 01:31:36.093071 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:36.092984 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:36.127333 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:36.127294 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-szz4p"] Jul 06 01:31:36.132515 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:36.132488 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-szz4p" Jul 06 01:31:36.135003 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:36.134969 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 06 01:31:36.135157 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:36.135133 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 06 01:31:36.135590 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:36.135569 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 06 01:31:36.135719 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:36.135678 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 06 01:31:36.136345 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:36.136326 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 06 01:31:36.136511 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:36.136332 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 06 01:31:36.136511 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:36.136377 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-tppmq\"" Jul 06 01:31:36.191839 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:36.191798 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:36.210722 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:36.210670 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a2b629f3-f3fc-40ef-af43-bfbdc4be7442-node-exporter-textfile\") pod \"node-exporter-szz4p\" (UID: \"a2b629f3-f3fc-40ef-af43-bfbdc4be7442\") " pod="openshift-monitoring/node-exporter-szz4p" Jul 06 01:31:36.210931 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:36.210744 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a2b629f3-f3fc-40ef-af43-bfbdc4be7442-root\") pod \"node-exporter-szz4p\" (UID: \"a2b629f3-f3fc-40ef-af43-bfbdc4be7442\") " pod="openshift-monitoring/node-exporter-szz4p" Jul 06 01:31:36.210931 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:36.210809 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blhbx\" (UniqueName: \"kubernetes.io/projected/a2b629f3-f3fc-40ef-af43-bfbdc4be7442-kube-api-access-blhbx\") pod \"node-exporter-szz4p\" (UID: \"a2b629f3-f3fc-40ef-af43-bfbdc4be7442\") " pod="openshift-monitoring/node-exporter-szz4p" Jul 06 01:31:36.210931 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:36.210841 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a2b629f3-f3fc-40ef-af43-bfbdc4be7442-node-exporter-wtmp\") pod \"node-exporter-szz4p\" (UID: \"a2b629f3-f3fc-40ef-af43-bfbdc4be7442\") " pod="openshift-monitoring/node-exporter-szz4p" Jul 06 01:31:36.210931 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:36.210891 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a2b629f3-f3fc-40ef-af43-bfbdc4be7442-node-exporter-tls\") pod \"node-exporter-szz4p\" (UID: \"a2b629f3-f3fc-40ef-af43-bfbdc4be7442\") " pod="openshift-monitoring/node-exporter-szz4p" Jul 06 01:31:36.210931 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:36.210916 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a2b629f3-f3fc-40ef-af43-bfbdc4be7442-sys\") pod \"node-exporter-szz4p\" (UID: \"a2b629f3-f3fc-40ef-af43-bfbdc4be7442\") " pod="openshift-monitoring/node-exporter-szz4p" Jul 06 01:31:36.211149 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:36.210938 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a2b629f3-f3fc-40ef-af43-bfbdc4be7442-metrics-client-ca\") pod \"node-exporter-szz4p\" (UID: \"a2b629f3-f3fc-40ef-af43-bfbdc4be7442\") " pod="openshift-monitoring/node-exporter-szz4p" Jul 06 01:31:36.211149 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:36.210984 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a2b629f3-f3fc-40ef-af43-bfbdc4be7442-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-szz4p\" (UID: \"a2b629f3-f3fc-40ef-af43-bfbdc4be7442\") " pod="openshift-monitoring/node-exporter-szz4p" Jul 06 01:31:36.211149 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:36.211063 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a2b629f3-f3fc-40ef-af43-bfbdc4be7442-node-exporter-accelerators-collector-config\") pod \"node-exporter-szz4p\" (UID: \"a2b629f3-f3fc-40ef-af43-bfbdc4be7442\") " pod="openshift-monitoring/node-exporter-szz4p" Jul 06 01:31:36.289177 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:36.289057 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:36.312150 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:36.312102 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a2b629f3-f3fc-40ef-af43-bfbdc4be7442-root\") pod \"node-exporter-szz4p\" (UID: \"a2b629f3-f3fc-40ef-af43-bfbdc4be7442\") " pod="openshift-monitoring/node-exporter-szz4p" Jul 06 01:31:36.312358 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:36.312191 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blhbx\" (UniqueName: \"kubernetes.io/projected/a2b629f3-f3fc-40ef-af43-bfbdc4be7442-kube-api-access-blhbx\") pod \"node-exporter-szz4p\" (UID: \"a2b629f3-f3fc-40ef-af43-bfbdc4be7442\") " pod="openshift-monitoring/node-exporter-szz4p" Jul 06 01:31:36.312358 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:36.312228 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a2b629f3-f3fc-40ef-af43-bfbdc4be7442-node-exporter-wtmp\") pod \"node-exporter-szz4p\" (UID: \"a2b629f3-f3fc-40ef-af43-bfbdc4be7442\") " pod="openshift-monitoring/node-exporter-szz4p" Jul 06 01:31:36.312358 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:36.312234 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a2b629f3-f3fc-40ef-af43-bfbdc4be7442-root\") pod \"node-exporter-szz4p\" (UID: \"a2b629f3-f3fc-40ef-af43-bfbdc4be7442\") " pod="openshift-monitoring/node-exporter-szz4p" Jul 06 01:31:36.312358 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:36.312247 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a2b629f3-f3fc-40ef-af43-bfbdc4be7442-node-exporter-tls\") pod \"node-exporter-szz4p\" (UID: \"a2b629f3-f3fc-40ef-af43-bfbdc4be7442\") " pod="openshift-monitoring/node-exporter-szz4p" Jul 06 01:31:36.312358 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:36.312302 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a2b629f3-f3fc-40ef-af43-bfbdc4be7442-sys\") pod \"node-exporter-szz4p\" (UID: \"a2b629f3-f3fc-40ef-af43-bfbdc4be7442\") " pod="openshift-monitoring/node-exporter-szz4p" Jul 06 01:31:36.312358 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:36.312325 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a2b629f3-f3fc-40ef-af43-bfbdc4be7442-metrics-client-ca\") pod \"node-exporter-szz4p\" (UID: \"a2b629f3-f3fc-40ef-af43-bfbdc4be7442\") " pod="openshift-monitoring/node-exporter-szz4p" Jul 06 01:31:36.312358 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:31:36.312330 2580 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 06 01:31:36.312358 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:36.312351 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a2b629f3-f3fc-40ef-af43-bfbdc4be7442-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-szz4p\" (UID: \"a2b629f3-f3fc-40ef-af43-bfbdc4be7442\") " pod="openshift-monitoring/node-exporter-szz4p" Jul 06 01:31:36.312865 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:31:36.312380 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2b629f3-f3fc-40ef-af43-bfbdc4be7442-node-exporter-tls podName:a2b629f3-f3fc-40ef-af43-bfbdc4be7442 nodeName:}" failed. No retries permitted until 2026-07-06 01:31:36.812364944 +0000 UTC m=+290.783616614 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/a2b629f3-f3fc-40ef-af43-bfbdc4be7442-node-exporter-tls") pod "node-exporter-szz4p" (UID: "a2b629f3-f3fc-40ef-af43-bfbdc4be7442") : secret "node-exporter-tls" not found Jul 06 01:31:36.312865 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:36.312412 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a2b629f3-f3fc-40ef-af43-bfbdc4be7442-node-exporter-accelerators-collector-config\") pod \"node-exporter-szz4p\" (UID: \"a2b629f3-f3fc-40ef-af43-bfbdc4be7442\") " pod="openshift-monitoring/node-exporter-szz4p" Jul 06 01:31:36.312865 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:36.312475 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a2b629f3-f3fc-40ef-af43-bfbdc4be7442-node-exporter-textfile\") pod \"node-exporter-szz4p\" (UID: \"a2b629f3-f3fc-40ef-af43-bfbdc4be7442\") " pod="openshift-monitoring/node-exporter-szz4p" Jul 06 01:31:36.312865 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:36.312856 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a2b629f3-f3fc-40ef-af43-bfbdc4be7442-node-exporter-textfile\") pod \"node-exporter-szz4p\" (UID: \"a2b629f3-f3fc-40ef-af43-bfbdc4be7442\") " pod="openshift-monitoring/node-exporter-szz4p" Jul 06 01:31:36.313057 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:36.312898 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a2b629f3-f3fc-40ef-af43-bfbdc4be7442-node-exporter-wtmp\") pod \"node-exporter-szz4p\" (UID: \"a2b629f3-f3fc-40ef-af43-bfbdc4be7442\") " pod="openshift-monitoring/node-exporter-szz4p" Jul 06 01:31:36.313057 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:36.312935 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a2b629f3-f3fc-40ef-af43-bfbdc4be7442-sys\") pod \"node-exporter-szz4p\" (UID: \"a2b629f3-f3fc-40ef-af43-bfbdc4be7442\") " pod="openshift-monitoring/node-exporter-szz4p" Jul 06 01:31:36.313420 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:36.313394 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a2b629f3-f3fc-40ef-af43-bfbdc4be7442-node-exporter-accelerators-collector-config\") pod \"node-exporter-szz4p\" (UID: \"a2b629f3-f3fc-40ef-af43-bfbdc4be7442\") " pod="openshift-monitoring/node-exporter-szz4p" Jul 06 01:31:36.313527 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:36.313509 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a2b629f3-f3fc-40ef-af43-bfbdc4be7442-metrics-client-ca\") pod \"node-exporter-szz4p\" (UID: \"a2b629f3-f3fc-40ef-af43-bfbdc4be7442\") " pod="openshift-monitoring/node-exporter-szz4p" Jul 06 01:31:36.315170 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:36.315141 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a2b629f3-f3fc-40ef-af43-bfbdc4be7442-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-szz4p\" (UID: \"a2b629f3-f3fc-40ef-af43-bfbdc4be7442\") " pod="openshift-monitoring/node-exporter-szz4p" Jul 06 01:31:36.321839 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:36.321813 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-blhbx\" (UniqueName: \"kubernetes.io/projected/a2b629f3-f3fc-40ef-af43-bfbdc4be7442-kube-api-access-blhbx\") pod \"node-exporter-szz4p\" (UID: \"a2b629f3-f3fc-40ef-af43-bfbdc4be7442\") " pod="openshift-monitoring/node-exporter-szz4p" Jul 06 01:31:36.389569 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:36.389524 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:36.489543 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:36.489500 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:36.594775 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:36.591407 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:36.691360 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:36.691320 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:36.791246 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:36.791211 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:36.817253 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:36.817211 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a2b629f3-f3fc-40ef-af43-bfbdc4be7442-node-exporter-tls\") pod \"node-exporter-szz4p\" (UID: \"a2b629f3-f3fc-40ef-af43-bfbdc4be7442\") " pod="openshift-monitoring/node-exporter-szz4p" Jul 06 01:31:36.820010 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:36.819980 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a2b629f3-f3fc-40ef-af43-bfbdc4be7442-node-exporter-tls\") pod \"node-exporter-szz4p\" (UID: \"a2b629f3-f3fc-40ef-af43-bfbdc4be7442\") " pod="openshift-monitoring/node-exporter-szz4p" Jul 06 01:31:37.044143 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:37.044098 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-szz4p" Jul 06 01:31:37.095608 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:37.095533 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:37.197992 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:37.197947 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:37.296207 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:37.296107 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:38.370212 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:38.370172 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:38.454384 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:31:38.454343 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-qnp65" podUID="15c8ffd6-1e00-4975-b807-63e00ad36b44" Jul 06 01:31:38.470715 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:38.470684 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:38.541888 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:38.541846 2580 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5fd6d69f-4g6hq" podUID="235fdd76-2b3a-4698-b9e5-ab05305231c6" containerName="acm-agent" probeResult="failure" output="Get \"http://10.134.0.8:8000/healthz\": dial tcp 10.134.0.8:8000: connect: connection refused" Jul 06 01:31:38.577608 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:38.577568 2580 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-658bcdcb4f-c6mtj" podUID="1b429a9e-ffde-4404-a881-e2794a4bbd48" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 06 01:31:39.086018 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:39.085981 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:39.197004 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:39.196963 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:39.287843 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:39.287808 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:39.658887 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:39.658841 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:39.755770 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:39.755739 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:40.898128 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:40.898075 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:40.992759 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:40.992715 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:41.363659 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:41.363565 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:41.455491 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:41.455420 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:41.555707 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:41.555675 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:41.683881 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:31:41.683856 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2b629f3_f3fc_40ef_af43_bfbdc4be7442.slice/crio-b4ead1cd65b59caddf1311bcf15d4c00bebe39b44aa02444af1b632016af5598 WatchSource:0}: Error finding container b4ead1cd65b59caddf1311bcf15d4c00bebe39b44aa02444af1b632016af5598: Status 404 returned error can't find the container with id b4ead1cd65b59caddf1311bcf15d4c00bebe39b44aa02444af1b632016af5598 Jul 06 01:31:42.189654 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:42.189615 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:42.289102 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:42.289011 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:42.375506 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:42.375473 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:42.469745 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:42.469705 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-68kkp" event={"ID":"108029ee-5264-4c60-91b5-a390aea7fe33","Type":"ContainerStarted","Data":"2cb01fa0944bbc61c79f63247772ae7004f6aa7917733f5c37702b5c0357c74a"} Jul 06 01:31:42.470247 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:42.470197 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-68kkp" Jul 06 01:31:42.471767 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:42.471730 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-szz4p" event={"ID":"a2b629f3-f3fc-40ef-af43-bfbdc4be7442","Type":"ContainerStarted","Data":"b4ead1cd65b59caddf1311bcf15d4c00bebe39b44aa02444af1b632016af5598"} Jul 06 01:31:42.474258 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:42.474205 2580 generic.go:356] "Generic (PLEG): container finished" podID="235fdd76-2b3a-4698-b9e5-ab05305231c6" containerID="aa09d4b463b9e383c06864e2b918c3984efc818031acd3038e32f24ae732e2f2" exitCode=1 Jul 06 01:31:42.474640 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:42.474282 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5fd6d69f-4g6hq" event={"ID":"235fdd76-2b3a-4698-b9e5-ab05305231c6","Type":"ContainerDied","Data":"aa09d4b463b9e383c06864e2b918c3984efc818031acd3038e32f24ae732e2f2"} Jul 06 01:31:42.474743 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:42.474672 2580 scope.go:117] "RemoveContainer" containerID="9b03f4ae5175a7f2890d6ab0062631b15365fb31216602b63315233188365873" Jul 06 01:31:42.475069 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:42.475049 2580 scope.go:117] "RemoveContainer" containerID="aa09d4b463b9e383c06864e2b918c3984efc818031acd3038e32f24ae732e2f2" Jul 06 01:31:42.475267 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:31:42.475240 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"acm-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=acm-agent pod=klusterlet-addon-workmgr-7c5fd6d69f-4g6hq_open-cluster-management-agent-addon(235fdd76-2b3a-4698-b9e5-ab05305231c6)\"" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5fd6d69f-4g6hq" podUID="235fdd76-2b3a-4698-b9e5-ab05305231c6" Jul 06 01:31:42.481658 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:42.481610 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-68kkp" Jul 06 01:31:42.487516 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:42.487471 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-68kkp" podStartSLOduration=1.084111014 podStartE2EDuration="18.487457297s" totalImagesPullingTime="17.403346283s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:31:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:31:42.486579409 +0000 UTC m=+296.457831102" watchObservedRunningTime="2026-07-06 01:31:42.487457297 +0000 UTC m=+296.458708982" Jul 06 01:31:42.491624 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:42.491591 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:42.591740 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:42.591656 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:43.442532 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:43.442492 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-dxjmj" Jul 06 01:31:43.490321 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:43.490282 2580 generic.go:356] "Generic (PLEG): container finished" podID="a2b629f3-f3fc-40ef-af43-bfbdc4be7442" containerID="1d69e110252e179317d3e50f45f8693d5f5e23723f1fe7551f7258950092fd5e" exitCode=0 Jul 06 01:31:43.490524 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:43.490384 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-szz4p" event={"ID":"a2b629f3-f3fc-40ef-af43-bfbdc4be7442","Type":"ContainerDied","Data":"1d69e110252e179317d3e50f45f8693d5f5e23723f1fe7551f7258950092fd5e"} Jul 06 01:31:43.571792 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:43.571763 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:43.603612 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:43.603573 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6d6dc997c5-ng9mx"] Jul 06 01:31:43.625952 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:43.625910 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d6dc997c5-ng9mx"] Jul 06 01:31:43.626097 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:43.626060 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d6dc997c5-ng9mx" Jul 06 01:31:43.629303 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:43.629266 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 06 01:31:43.629465 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:43.629266 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 06 01:31:43.629465 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:43.629396 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-cgnmd\"" Jul 06 01:31:43.629602 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:43.629269 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 06 01:31:43.629658 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:43.629630 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 06 01:31:43.630107 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:43.630074 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 06 01:31:43.634547 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:43.634478 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 06 01:31:43.644948 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:43.644905 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:43.678312 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:43.678278 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fe893ea8-ebec-4e52-95db-ef9285059853-oauth-serving-cert\") pod \"console-6d6dc997c5-ng9mx\" (UID: \"fe893ea8-ebec-4e52-95db-ef9285059853\") " pod="openshift-console/console-6d6dc997c5-ng9mx" Jul 06 01:31:43.678491 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:43.678315 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mp657\" (UniqueName: \"kubernetes.io/projected/fe893ea8-ebec-4e52-95db-ef9285059853-kube-api-access-mp657\") pod \"console-6d6dc997c5-ng9mx\" (UID: \"fe893ea8-ebec-4e52-95db-ef9285059853\") " pod="openshift-console/console-6d6dc997c5-ng9mx" Jul 06 01:31:43.678491 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:43.678373 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fe893ea8-ebec-4e52-95db-ef9285059853-console-oauth-config\") pod \"console-6d6dc997c5-ng9mx\" (UID: \"fe893ea8-ebec-4e52-95db-ef9285059853\") " pod="openshift-console/console-6d6dc997c5-ng9mx" Jul 06 01:31:43.678491 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:43.678416 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fe893ea8-ebec-4e52-95db-ef9285059853-console-config\") pod \"console-6d6dc997c5-ng9mx\" (UID: \"fe893ea8-ebec-4e52-95db-ef9285059853\") " pod="openshift-console/console-6d6dc997c5-ng9mx" Jul 06 01:31:43.678672 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:43.678495 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fe893ea8-ebec-4e52-95db-ef9285059853-console-serving-cert\") pod \"console-6d6dc997c5-ng9mx\" (UID: \"fe893ea8-ebec-4e52-95db-ef9285059853\") " pod="openshift-console/console-6d6dc997c5-ng9mx" Jul 06 01:31:43.678672 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:43.678531 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe893ea8-ebec-4e52-95db-ef9285059853-trusted-ca-bundle\") pod \"console-6d6dc997c5-ng9mx\" (UID: \"fe893ea8-ebec-4e52-95db-ef9285059853\") " pod="openshift-console/console-6d6dc997c5-ng9mx" Jul 06 01:31:43.678672 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:43.678599 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fe893ea8-ebec-4e52-95db-ef9285059853-service-ca\") pod \"console-6d6dc997c5-ng9mx\" (UID: \"fe893ea8-ebec-4e52-95db-ef9285059853\") " pod="openshift-console/console-6d6dc997c5-ng9mx" Jul 06 01:31:43.779560 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:43.779526 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fe893ea8-ebec-4e52-95db-ef9285059853-console-config\") pod \"console-6d6dc997c5-ng9mx\" (UID: \"fe893ea8-ebec-4e52-95db-ef9285059853\") " pod="openshift-console/console-6d6dc997c5-ng9mx" Jul 06 01:31:43.779749 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:43.779589 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fe893ea8-ebec-4e52-95db-ef9285059853-console-serving-cert\") pod \"console-6d6dc997c5-ng9mx\" (UID: \"fe893ea8-ebec-4e52-95db-ef9285059853\") " pod="openshift-console/console-6d6dc997c5-ng9mx" Jul 06 01:31:43.779820 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:43.779773 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe893ea8-ebec-4e52-95db-ef9285059853-trusted-ca-bundle\") pod \"console-6d6dc997c5-ng9mx\" (UID: \"fe893ea8-ebec-4e52-95db-ef9285059853\") " pod="openshift-console/console-6d6dc997c5-ng9mx" Jul 06 01:31:43.779872 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:43.779831 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fe893ea8-ebec-4e52-95db-ef9285059853-service-ca\") pod \"console-6d6dc997c5-ng9mx\" (UID: \"fe893ea8-ebec-4e52-95db-ef9285059853\") " pod="openshift-console/console-6d6dc997c5-ng9mx" Jul 06 01:31:43.780077 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:43.780043 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fe893ea8-ebec-4e52-95db-ef9285059853-oauth-serving-cert\") pod \"console-6d6dc997c5-ng9mx\" (UID: \"fe893ea8-ebec-4e52-95db-ef9285059853\") " pod="openshift-console/console-6d6dc997c5-ng9mx" Jul 06 01:31:43.780227 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:43.780097 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mp657\" (UniqueName: \"kubernetes.io/projected/fe893ea8-ebec-4e52-95db-ef9285059853-kube-api-access-mp657\") pod \"console-6d6dc997c5-ng9mx\" (UID: \"fe893ea8-ebec-4e52-95db-ef9285059853\") " pod="openshift-console/console-6d6dc997c5-ng9mx" Jul 06 01:31:43.780227 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:43.780160 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fe893ea8-ebec-4e52-95db-ef9285059853-console-oauth-config\") pod \"console-6d6dc997c5-ng9mx\" (UID: \"fe893ea8-ebec-4e52-95db-ef9285059853\") " pod="openshift-console/console-6d6dc997c5-ng9mx" Jul 06 01:31:43.780935 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:43.780531 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fe893ea8-ebec-4e52-95db-ef9285059853-service-ca\") pod \"console-6d6dc997c5-ng9mx\" (UID: \"fe893ea8-ebec-4e52-95db-ef9285059853\") " pod="openshift-console/console-6d6dc997c5-ng9mx" Jul 06 01:31:43.780935 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:43.780541 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fe893ea8-ebec-4e52-95db-ef9285059853-console-config\") pod \"console-6d6dc997c5-ng9mx\" (UID: \"fe893ea8-ebec-4e52-95db-ef9285059853\") " pod="openshift-console/console-6d6dc997c5-ng9mx" Jul 06 01:31:43.780935 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:43.780727 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe893ea8-ebec-4e52-95db-ef9285059853-trusted-ca-bundle\") pod \"console-6d6dc997c5-ng9mx\" (UID: \"fe893ea8-ebec-4e52-95db-ef9285059853\") " pod="openshift-console/console-6d6dc997c5-ng9mx" Jul 06 01:31:43.781750 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:43.781727 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fe893ea8-ebec-4e52-95db-ef9285059853-oauth-serving-cert\") pod \"console-6d6dc997c5-ng9mx\" (UID: \"fe893ea8-ebec-4e52-95db-ef9285059853\") " pod="openshift-console/console-6d6dc997c5-ng9mx" Jul 06 01:31:43.783431 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:43.782723 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fe893ea8-ebec-4e52-95db-ef9285059853-console-serving-cert\") pod \"console-6d6dc997c5-ng9mx\" (UID: \"fe893ea8-ebec-4e52-95db-ef9285059853\") " pod="openshift-console/console-6d6dc997c5-ng9mx" Jul 06 01:31:43.783431 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:43.783124 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fe893ea8-ebec-4e52-95db-ef9285059853-console-oauth-config\") pod \"console-6d6dc997c5-ng9mx\" (UID: \"fe893ea8-ebec-4e52-95db-ef9285059853\") " pod="openshift-console/console-6d6dc997c5-ng9mx" Jul 06 01:31:43.792776 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:43.792745 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mp657\" (UniqueName: \"kubernetes.io/projected/fe893ea8-ebec-4e52-95db-ef9285059853-kube-api-access-mp657\") pod \"console-6d6dc997c5-ng9mx\" (UID: \"fe893ea8-ebec-4e52-95db-ef9285059853\") " pod="openshift-console/console-6d6dc997c5-ng9mx" Jul 06 01:31:43.939679 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:43.939636 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d6dc997c5-ng9mx" Jul 06 01:31:44.156736 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:44.156163 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d6dc997c5-ng9mx"] Jul 06 01:31:44.164290 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:31:44.161638 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe893ea8_ebec_4e52_95db_ef9285059853.slice/crio-a136256d4766dc58fe8fd5a2c99345b6f605da5902cfeef9e5d9e23ee7641c96 WatchSource:0}: Error finding container a136256d4766dc58fe8fd5a2c99345b6f605da5902cfeef9e5d9e23ee7641c96: Status 404 returned error can't find the container with id a136256d4766dc58fe8fd5a2c99345b6f605da5902cfeef9e5d9e23ee7641c96 Jul 06 01:31:44.497412 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:44.497364 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d6dc997c5-ng9mx" event={"ID":"fe893ea8-ebec-4e52-95db-ef9285059853","Type":"ContainerStarted","Data":"a136256d4766dc58fe8fd5a2c99345b6f605da5902cfeef9e5d9e23ee7641c96"} Jul 06 01:31:44.501579 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:44.501541 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-szz4p" event={"ID":"a2b629f3-f3fc-40ef-af43-bfbdc4be7442","Type":"ContainerStarted","Data":"6d8a5614cd3eb86b97c4ac9dc78174eb39a359940555c508168d495b283a83c0"} Jul 06 01:31:44.501742 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:44.501605 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-szz4p" event={"ID":"a2b629f3-f3fc-40ef-af43-bfbdc4be7442","Type":"ContainerStarted","Data":"b3f53d69b35b353afea4c3396fe0325cba4b2ce9bae0d7a682fd43fa10e4b4b5"} Jul 06 01:31:44.549519 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:44.549469 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-szz4p" podStartSLOduration=7.009053659 podStartE2EDuration="8.549449575s" totalImagesPullingTime="1.425294054s" totalInitContainerRuntime="115.101862ms" isStatefulPod=true podCreationTimestamp="2026-07-06 01:31:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:31:44.547462701 +0000 UTC m=+298.518714390" watchObservedRunningTime="2026-07-06 01:31:44.549449575 +0000 UTC m=+298.520701257" Jul 06 01:31:44.806772 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:44.806687 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:44.907170 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:44.907126 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:45.962557 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:45.962521 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:46.047384 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:46.047346 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d6dc997c5-ng9mx"] Jul 06 01:31:46.050799 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:46.050712 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:46.075862 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:46.074698 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-84f857cd9d-pj9h4"] Jul 06 01:31:46.094467 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:46.094407 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5fd6d69f-4g6hq" Jul 06 01:31:46.094639 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:46.094475 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84f857cd9d-pj9h4"] Jul 06 01:31:46.094822 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:46.094800 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84f857cd9d-pj9h4" Jul 06 01:31:46.094924 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:46.094907 2580 scope.go:117] "RemoveContainer" containerID="aa09d4b463b9e383c06864e2b918c3984efc818031acd3038e32f24ae732e2f2" Jul 06 01:31:46.205809 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:46.205778 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f-console-config\") pod \"console-84f857cd9d-pj9h4\" (UID: \"58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f\") " pod="openshift-console/console-84f857cd9d-pj9h4" Jul 06 01:31:46.205934 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:46.205889 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f-trusted-ca-bundle\") pod \"console-84f857cd9d-pj9h4\" (UID: \"58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f\") " pod="openshift-console/console-84f857cd9d-pj9h4" Jul 06 01:31:46.206022 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:46.205995 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cz2jw\" (UniqueName: \"kubernetes.io/projected/58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f-kube-api-access-cz2jw\") pod \"console-84f857cd9d-pj9h4\" (UID: \"58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f\") " pod="openshift-console/console-84f857cd9d-pj9h4" Jul 06 01:31:46.206238 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:46.206217 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f-console-serving-cert\") pod \"console-84f857cd9d-pj9h4\" (UID: \"58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f\") " pod="openshift-console/console-84f857cd9d-pj9h4" Jul 06 01:31:46.206298 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:46.206251 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f-service-ca\") pod \"console-84f857cd9d-pj9h4\" (UID: \"58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f\") " pod="openshift-console/console-84f857cd9d-pj9h4" Jul 06 01:31:46.206353 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:46.206322 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f-console-oauth-config\") pod \"console-84f857cd9d-pj9h4\" (UID: \"58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f\") " pod="openshift-console/console-84f857cd9d-pj9h4" Jul 06 01:31:46.206408 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:46.206359 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f-oauth-serving-cert\") pod \"console-84f857cd9d-pj9h4\" (UID: \"58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f\") " pod="openshift-console/console-84f857cd9d-pj9h4" Jul 06 01:31:46.307411 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:46.307321 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f-console-config\") pod \"console-84f857cd9d-pj9h4\" (UID: \"58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f\") " pod="openshift-console/console-84f857cd9d-pj9h4" Jul 06 01:31:46.307411 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:46.307366 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f-trusted-ca-bundle\") pod \"console-84f857cd9d-pj9h4\" (UID: \"58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f\") " pod="openshift-console/console-84f857cd9d-pj9h4" Jul 06 01:31:46.307411 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:46.307391 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cz2jw\" (UniqueName: \"kubernetes.io/projected/58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f-kube-api-access-cz2jw\") pod \"console-84f857cd9d-pj9h4\" (UID: \"58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f\") " pod="openshift-console/console-84f857cd9d-pj9h4" Jul 06 01:31:46.307700 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:46.307430 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f-console-serving-cert\") pod \"console-84f857cd9d-pj9h4\" (UID: \"58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f\") " pod="openshift-console/console-84f857cd9d-pj9h4" Jul 06 01:31:46.307700 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:46.307473 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f-service-ca\") pod \"console-84f857cd9d-pj9h4\" (UID: \"58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f\") " pod="openshift-console/console-84f857cd9d-pj9h4" Jul 06 01:31:46.315062 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:46.308094 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f-console-oauth-config\") pod \"console-84f857cd9d-pj9h4\" (UID: \"58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f\") " pod="openshift-console/console-84f857cd9d-pj9h4" Jul 06 01:31:46.315062 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:46.308173 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f-oauth-serving-cert\") pod \"console-84f857cd9d-pj9h4\" (UID: \"58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f\") " pod="openshift-console/console-84f857cd9d-pj9h4" Jul 06 01:31:46.315062 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:46.308692 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f-trusted-ca-bundle\") pod \"console-84f857cd9d-pj9h4\" (UID: \"58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f\") " pod="openshift-console/console-84f857cd9d-pj9h4" Jul 06 01:31:46.315062 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:46.309161 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f-oauth-serving-cert\") pod \"console-84f857cd9d-pj9h4\" (UID: \"58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f\") " pod="openshift-console/console-84f857cd9d-pj9h4" Jul 06 01:31:46.315062 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:46.309375 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f-console-config\") pod \"console-84f857cd9d-pj9h4\" (UID: \"58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f\") " pod="openshift-console/console-84f857cd9d-pj9h4" Jul 06 01:31:46.315062 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:46.310122 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f-service-ca\") pod \"console-84f857cd9d-pj9h4\" (UID: \"58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f\") " pod="openshift-console/console-84f857cd9d-pj9h4" Jul 06 01:31:46.315062 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:46.312836 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f-console-serving-cert\") pod \"console-84f857cd9d-pj9h4\" (UID: \"58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f\") " pod="openshift-console/console-84f857cd9d-pj9h4" Jul 06 01:31:46.316537 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:46.316308 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f-console-oauth-config\") pod \"console-84f857cd9d-pj9h4\" (UID: \"58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f\") " pod="openshift-console/console-84f857cd9d-pj9h4" Jul 06 01:31:46.318950 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:46.318925 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cz2jw\" (UniqueName: \"kubernetes.io/projected/58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f-kube-api-access-cz2jw\") pod \"console-84f857cd9d-pj9h4\" (UID: \"58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f\") " pod="openshift-console/console-84f857cd9d-pj9h4" Jul 06 01:31:46.409815 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:46.409705 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84f857cd9d-pj9h4" Jul 06 01:31:46.410811 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:46.410789 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-67887675c-sjf6n" Jul 06 01:31:46.511856 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:46.511711 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5fd6d69f-4g6hq" event={"ID":"235fdd76-2b3a-4698-b9e5-ab05305231c6","Type":"ContainerStarted","Data":"90558ad34821f87f688d3b5cd05de51ecf0353a78afae5b0b7526ac87d51b033"} Jul 06 01:31:46.512668 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:46.512232 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5fd6d69f-4g6hq" Jul 06 01:31:46.513528 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:46.513510 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5fd6d69f-4g6hq" Jul 06 01:31:46.537383 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:46.537344 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:46.786785 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:46.786729 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:46.888467 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:46.888379 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:47.772518 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:47.772452 2580 kubelet.go:1648] "Image garbage collection succeeded" Jul 06 01:31:47.892754 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:47.892716 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84f857cd9d-pj9h4"] Jul 06 01:31:47.899632 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:47.899595 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 01:31:48.521801 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:48.521766 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d6dc997c5-ng9mx" event={"ID":"fe893ea8-ebec-4e52-95db-ef9285059853","Type":"ContainerStarted","Data":"093ab5ca45c541b46ddf52d477746648693d3ecb72d18b154e8f30ca9e784c5b"} Jul 06 01:31:48.523095 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:48.523069 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84f857cd9d-pj9h4" event={"ID":"58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f","Type":"ContainerStarted","Data":"1fad576fa2923b8fa6f38e827301cff6662c2fdc4df15cd7d77a17e73de18d97"} Jul 06 01:31:48.578333 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:48.578296 2580 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-658bcdcb4f-c6mtj" podUID="1b429a9e-ffde-4404-a881-e2794a4bbd48" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 06 01:31:48.578512 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:48.578400 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-658bcdcb4f-c6mtj" Jul 06 01:31:48.579037 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:48.579000 2580 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"eba29e88acd2c97789380944b511c19025496812c6cf69a7ca06bc81fa274fc2"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-658bcdcb4f-c6mtj" containerMessage="Container service-proxy failed liveness probe, will be restarted" Jul 06 01:31:48.579141 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:48.579057 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-658bcdcb4f-c6mtj" podUID="1b429a9e-ffde-4404-a881-e2794a4bbd48" containerName="service-proxy" containerID="cri-o://eba29e88acd2c97789380944b511c19025496812c6cf69a7ca06bc81fa274fc2" gracePeriod=30 Jul 06 01:31:49.530539 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:49.530498 2580 generic.go:356] "Generic (PLEG): container finished" podID="1b429a9e-ffde-4404-a881-e2794a4bbd48" containerID="eba29e88acd2c97789380944b511c19025496812c6cf69a7ca06bc81fa274fc2" exitCode=2 Jul 06 01:31:49.531116 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:49.530561 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-658bcdcb4f-c6mtj" event={"ID":"1b429a9e-ffde-4404-a881-e2794a4bbd48","Type":"ContainerDied","Data":"eba29e88acd2c97789380944b511c19025496812c6cf69a7ca06bc81fa274fc2"} Jul 06 01:31:49.531116 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:49.530609 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-658bcdcb4f-c6mtj" event={"ID":"1b429a9e-ffde-4404-a881-e2794a4bbd48","Type":"ContainerStarted","Data":"695c75a46e09a026a546981132bf0dd86532acea1a53647e41dec513c13877fd"} Jul 06 01:31:49.532548 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:49.532508 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84f857cd9d-pj9h4" event={"ID":"58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f","Type":"ContainerStarted","Data":"5a9477df5de9a11b442899e41dd514f088238f333a5968ab71aa30980872b594"} Jul 06 01:31:49.551960 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:49.551891 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6d6dc997c5-ng9mx" podStartSLOduration=2.587347389 podStartE2EDuration="6.551872175s" totalImagesPullingTime="3.964524786s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:31:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:31:48.557621638 +0000 UTC m=+302.528873328" watchObservedRunningTime="2026-07-06 01:31:49.551872175 +0000 UTC m=+303.523123865" Jul 06 01:31:49.570836 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:49.570769 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-84f857cd9d-pj9h4" podStartSLOduration=2.952596777 podStartE2EDuration="3.570743881s" totalImagesPullingTime="618.147104ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:31:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:31:49.568926782 +0000 UTC m=+303.540178472" watchObservedRunningTime="2026-07-06 01:31:49.570743881 +0000 UTC m=+303.541995574" Jul 06 01:31:50.454360 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:50.454313 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qnp65" Jul 06 01:31:50.605979 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:50.605942 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:50.684428 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:50.684383 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:53.940320 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:53.940277 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6d6dc997c5-ng9mx" Jul 06 01:31:56.409936 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:56.409905 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-84f857cd9d-pj9h4" Jul 06 01:31:56.410383 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:56.409986 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-84f857cd9d-pj9h4" Jul 06 01:31:56.414733 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:56.414704 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-84f857cd9d-pj9h4" Jul 06 01:31:56.564213 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:56.564185 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-84f857cd9d-pj9h4" Jul 06 01:31:56.750632 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:56.750537 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:56.852552 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:56.852523 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:56.953562 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:56.953522 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:59.580563 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:59.580524 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:31:59.681541 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:31:59.681506 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:32:04.866993 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:32:04.866956 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:32:04.914807 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:32:04.914763 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:32:13.563278 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:32:13.563238 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-6d6dc997c5-ng9mx" podUID="fe893ea8-ebec-4e52-95db-ef9285059853" containerName="console" containerID="cri-o://093ab5ca45c541b46ddf52d477746648693d3ecb72d18b154e8f30ca9e784c5b" gracePeriod=15 Jul 06 01:32:13.849050 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:32:13.849007 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d6dc997c5-ng9mx" Jul 06 01:32:14.033619 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:32:14.033586 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fe893ea8-ebec-4e52-95db-ef9285059853-console-oauth-config\") pod \"fe893ea8-ebec-4e52-95db-ef9285059853\" (UID: \"fe893ea8-ebec-4e52-95db-ef9285059853\") " Jul 06 01:32:14.033619 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:32:14.033629 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fe893ea8-ebec-4e52-95db-ef9285059853-oauth-serving-cert\") pod \"fe893ea8-ebec-4e52-95db-ef9285059853\" (UID: \"fe893ea8-ebec-4e52-95db-ef9285059853\") " Jul 06 01:32:14.033873 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:32:14.033663 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mp657\" (UniqueName: \"kubernetes.io/projected/fe893ea8-ebec-4e52-95db-ef9285059853-kube-api-access-mp657\") pod \"fe893ea8-ebec-4e52-95db-ef9285059853\" (UID: \"fe893ea8-ebec-4e52-95db-ef9285059853\") " Jul 06 01:32:14.033873 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:32:14.033687 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fe893ea8-ebec-4e52-95db-ef9285059853-service-ca\") pod \"fe893ea8-ebec-4e52-95db-ef9285059853\" (UID: \"fe893ea8-ebec-4e52-95db-ef9285059853\") " Jul 06 01:32:14.033873 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:32:14.033716 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fe893ea8-ebec-4e52-95db-ef9285059853-console-config\") pod \"fe893ea8-ebec-4e52-95db-ef9285059853\" (UID: \"fe893ea8-ebec-4e52-95db-ef9285059853\") " Jul 06 01:32:14.033873 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:32:14.033773 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fe893ea8-ebec-4e52-95db-ef9285059853-console-serving-cert\") pod \"fe893ea8-ebec-4e52-95db-ef9285059853\" (UID: \"fe893ea8-ebec-4e52-95db-ef9285059853\") " Jul 06 01:32:14.034073 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:32:14.033910 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe893ea8-ebec-4e52-95db-ef9285059853-trusted-ca-bundle\") pod \"fe893ea8-ebec-4e52-95db-ef9285059853\" (UID: \"fe893ea8-ebec-4e52-95db-ef9285059853\") " Jul 06 01:32:14.034181 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:32:14.034153 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe893ea8-ebec-4e52-95db-ef9285059853-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "fe893ea8-ebec-4e52-95db-ef9285059853" (UID: "fe893ea8-ebec-4e52-95db-ef9285059853"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 01:32:14.034244 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:32:14.034196 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe893ea8-ebec-4e52-95db-ef9285059853-service-ca" (OuterVolumeSpecName: "service-ca") pod "fe893ea8-ebec-4e52-95db-ef9285059853" (UID: "fe893ea8-ebec-4e52-95db-ef9285059853"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 01:32:14.034350 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:32:14.034318 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe893ea8-ebec-4e52-95db-ef9285059853-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "fe893ea8-ebec-4e52-95db-ef9285059853" (UID: "fe893ea8-ebec-4e52-95db-ef9285059853"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 01:32:14.034923 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:32:14.034902 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe893ea8-ebec-4e52-95db-ef9285059853-console-config" (OuterVolumeSpecName: "console-config") pod "fe893ea8-ebec-4e52-95db-ef9285059853" (UID: "fe893ea8-ebec-4e52-95db-ef9285059853"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 01:32:14.036147 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:32:14.036112 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe893ea8-ebec-4e52-95db-ef9285059853-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "fe893ea8-ebec-4e52-95db-ef9285059853" (UID: "fe893ea8-ebec-4e52-95db-ef9285059853"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 01:32:14.036147 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:32:14.036127 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe893ea8-ebec-4e52-95db-ef9285059853-kube-api-access-mp657" (OuterVolumeSpecName: "kube-api-access-mp657") pod "fe893ea8-ebec-4e52-95db-ef9285059853" (UID: "fe893ea8-ebec-4e52-95db-ef9285059853"). InnerVolumeSpecName "kube-api-access-mp657". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 01:32:14.036288 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:32:14.036160 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe893ea8-ebec-4e52-95db-ef9285059853-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "fe893ea8-ebec-4e52-95db-ef9285059853" (UID: "fe893ea8-ebec-4e52-95db-ef9285059853"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 01:32:14.135388 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:32:14.135302 2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fe893ea8-ebec-4e52-95db-ef9285059853-service-ca\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 01:32:14.135388 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:32:14.135331 2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fe893ea8-ebec-4e52-95db-ef9285059853-console-config\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 01:32:14.135388 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:32:14.135347 2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fe893ea8-ebec-4e52-95db-ef9285059853-console-serving-cert\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 01:32:14.135388 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:32:14.135358 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe893ea8-ebec-4e52-95db-ef9285059853-trusted-ca-bundle\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 01:32:14.135388 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:32:14.135366 2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fe893ea8-ebec-4e52-95db-ef9285059853-console-oauth-config\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 01:32:14.135388 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:32:14.135377 2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fe893ea8-ebec-4e52-95db-ef9285059853-oauth-serving-cert\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 01:32:14.135388 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:32:14.135385 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mp657\" (UniqueName: \"kubernetes.io/projected/fe893ea8-ebec-4e52-95db-ef9285059853-kube-api-access-mp657\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 01:32:14.620403 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:32:14.620369 2580 generic.go:356] "Generic (PLEG): container finished" podID="fe893ea8-ebec-4e52-95db-ef9285059853" containerID="093ab5ca45c541b46ddf52d477746648693d3ecb72d18b154e8f30ca9e784c5b" exitCode=0 Jul 06 01:32:14.620854 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:32:14.620410 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d6dc997c5-ng9mx" event={"ID":"fe893ea8-ebec-4e52-95db-ef9285059853","Type":"ContainerDied","Data":"093ab5ca45c541b46ddf52d477746648693d3ecb72d18b154e8f30ca9e784c5b"} Jul 06 01:32:14.620854 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:32:14.620418 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d6dc997c5-ng9mx" Jul 06 01:32:14.620854 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:32:14.620466 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d6dc997c5-ng9mx" event={"ID":"fe893ea8-ebec-4e52-95db-ef9285059853","Type":"ContainerDied","Data":"a136256d4766dc58fe8fd5a2c99345b6f605da5902cfeef9e5d9e23ee7641c96"} Jul 06 01:32:14.620854 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:32:14.620487 2580 scope.go:117] "RemoveContainer" containerID="093ab5ca45c541b46ddf52d477746648693d3ecb72d18b154e8f30ca9e784c5b" Jul 06 01:32:14.628585 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:32:14.628566 2580 scope.go:117] "RemoveContainer" containerID="093ab5ca45c541b46ddf52d477746648693d3ecb72d18b154e8f30ca9e784c5b" Jul 06 01:32:14.628906 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:32:14.628874 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"093ab5ca45c541b46ddf52d477746648693d3ecb72d18b154e8f30ca9e784c5b\": container with ID starting with 093ab5ca45c541b46ddf52d477746648693d3ecb72d18b154e8f30ca9e784c5b not found: ID does not exist" containerID="093ab5ca45c541b46ddf52d477746648693d3ecb72d18b154e8f30ca9e784c5b" Jul 06 01:32:14.628971 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:32:14.628921 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"093ab5ca45c541b46ddf52d477746648693d3ecb72d18b154e8f30ca9e784c5b"} err="failed to get container status \"093ab5ca45c541b46ddf52d477746648693d3ecb72d18b154e8f30ca9e784c5b\": rpc error: code = NotFound desc = could not find container \"093ab5ca45c541b46ddf52d477746648693d3ecb72d18b154e8f30ca9e784c5b\": container with ID starting with 093ab5ca45c541b46ddf52d477746648693d3ecb72d18b154e8f30ca9e784c5b not found: ID does not exist" Jul 06 01:32:14.637520 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:32:14.637481 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d6dc997c5-ng9mx"] Jul 06 01:32:14.639685 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:32:14.639661 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6d6dc997c5-ng9mx"] Jul 06 01:32:16.458012 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:32:16.457981 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe893ea8-ebec-4e52-95db-ef9285059853" path="/var/lib/kubelet/pods/fe893ea8-ebec-4e52-95db-ef9285059853/volumes" Jul 06 01:32:49.028010 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:32:49.027973 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:32:49.099252 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:32:49.099220 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:32:52.200246 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:32:52.200211 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:32:52.250250 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:32:52.250215 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:32:54.834643 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:32:54.834611 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:32:54.938113 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:32:54.938082 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:32:55.230099 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:32:55.230062 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:32:55.500701 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:32:55.500597 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:32:55.557407 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:32:55.557372 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:32:58.757567 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:32:58.757533 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:32:58.873064 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:32:58.873013 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:33:00.283865 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:33:00.283823 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15c8ffd6-1e00-4975-b807-63e00ad36b44-metrics-certs\") pod \"network-metrics-daemon-qnp65\" (UID: \"15c8ffd6-1e00-4975-b807-63e00ad36b44\") " pod="openshift-multus/network-metrics-daemon-qnp65" Jul 06 01:33:00.286268 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:33:00.286236 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15c8ffd6-1e00-4975-b807-63e00ad36b44-metrics-certs\") pod \"network-metrics-daemon-qnp65\" (UID: \"15c8ffd6-1e00-4975-b807-63e00ad36b44\") " pod="openshift-multus/network-metrics-daemon-qnp65" Jul 06 01:33:00.357750 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:33:00.357717 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qszpf\"" Jul 06 01:33:00.365815 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:33:00.365788 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qnp65" Jul 06 01:33:00.498148 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:33:00.498043 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qnp65"] Jul 06 01:33:00.500889 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:33:00.500862 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15c8ffd6_1e00_4975_b807_63e00ad36b44.slice/crio-e5b2ded0c081c4e92c8ee756f8249773fd84979708eefb28ef10382cd575f71a WatchSource:0}: Error finding container e5b2ded0c081c4e92c8ee756f8249773fd84979708eefb28ef10382cd575f71a: Status 404 returned error can't find the container with id e5b2ded0c081c4e92c8ee756f8249773fd84979708eefb28ef10382cd575f71a Jul 06 01:33:00.781571 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:33:00.781531 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qnp65" event={"ID":"15c8ffd6-1e00-4975-b807-63e00ad36b44","Type":"ContainerStarted","Data":"e5b2ded0c081c4e92c8ee756f8249773fd84979708eefb28ef10382cd575f71a"} Jul 06 01:33:01.786313 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:33:01.786224 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qnp65" event={"ID":"15c8ffd6-1e00-4975-b807-63e00ad36b44","Type":"ContainerStarted","Data":"7070aadbee1a37475f080b921c3ae706b0d61c35c8b3e27ff821dcecc91e3ab5"} Jul 06 01:33:01.786313 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:33:01.786277 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qnp65" event={"ID":"15c8ffd6-1e00-4975-b807-63e00ad36b44","Type":"ContainerStarted","Data":"e55e36c23ec5f3e37faf598af934369e83827f271422e628ec50602d1b6c913d"} Jul 06 01:33:01.803313 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:33:01.803261 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-qnp65" podStartSLOduration=374.849560128 podStartE2EDuration="6m15.803246917s" totalImagesPullingTime="953.686789ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:26:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:33:01.802993854 +0000 UTC m=+375.774245550" watchObservedRunningTime="2026-07-06 01:33:01.803246917 +0000 UTC m=+375.774498608" Jul 06 01:33:01.900826 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:33:01.900781 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:33:01.965276 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:33:01.965246 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:33:04.078998 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:33:04.078960 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:33:04.133626 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:33:04.133580 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:33:05.740202 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:33:05.740167 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-84f857cd9d-pj9h4"] Jul 06 01:33:05.838899 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:33:05.838864 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:33:05.895876 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:33:05.895844 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:33:05.947134 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:33:05.947103 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:33:19.888210 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:33:19.888174 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:33:19.988614 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:33:19.988579 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:33:30.760694 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:33:30.760636 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-84f857cd9d-pj9h4" podUID="58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f" containerName="console" containerID="cri-o://5a9477df5de9a11b442899e41dd514f088238f333a5968ab71aa30980872b594" gracePeriod=15 Jul 06 01:33:30.883660 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:33:30.883630 2580 generic.go:356] "Generic (PLEG): container finished" podID="58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f" containerID="5a9477df5de9a11b442899e41dd514f088238f333a5968ab71aa30980872b594" exitCode=0 Jul 06 01:33:30.883794 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:33:30.883709 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84f857cd9d-pj9h4" event={"ID":"58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f","Type":"ContainerDied","Data":"5a9477df5de9a11b442899e41dd514f088238f333a5968ab71aa30980872b594"} Jul 06 01:33:31.006871 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:33:31.006843 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84f857cd9d-pj9h4" Jul 06 01:33:31.118144 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:33:31.118058 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f-oauth-serving-cert\") pod \"58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f\" (UID: \"58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f\") " Jul 06 01:33:31.118144 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:33:31.118123 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f-trusted-ca-bundle\") pod \"58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f\" (UID: \"58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f\") " Jul 06 01:33:31.118356 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:33:31.118153 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cz2jw\" (UniqueName: \"kubernetes.io/projected/58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f-kube-api-access-cz2jw\") pod \"58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f\" (UID: \"58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f\") " Jul 06 01:33:31.118356 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:33:31.118325 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f-console-config\") pod \"58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f\" (UID: \"58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f\") " Jul 06 01:33:31.118491 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:33:31.118371 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f-console-oauth-config\") pod \"58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f\" (UID: \"58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f\") " Jul 06 01:33:31.118491 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:33:31.118403 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f" (UID: "58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 01:33:31.118491 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:33:31.118413 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f-service-ca\") pod \"58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f\" (UID: \"58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f\") " Jul 06 01:33:31.118655 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:33:31.118535 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f-console-serving-cert\") pod \"58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f\" (UID: \"58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f\") " Jul 06 01:33:31.118655 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:33:31.118623 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f" (UID: "58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 01:33:31.118763 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:33:31.118677 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f-console-config" (OuterVolumeSpecName: "console-config") pod "58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f" (UID: "58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 01:33:31.118763 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:33:31.118737 2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f-oauth-serving-cert\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 01:33:31.118763 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:33:31.118756 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f-trusted-ca-bundle\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 01:33:31.118871 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:33:31.118771 2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f-console-config\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 01:33:31.118871 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:33:31.118769 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f-service-ca" (OuterVolumeSpecName: "service-ca") pod "58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f" (UID: "58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 01:33:31.120643 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:33:31.120608 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f" (UID: "58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 01:33:31.120754 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:33:31.120661 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f-kube-api-access-cz2jw" (OuterVolumeSpecName: "kube-api-access-cz2jw") pod "58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f" (UID: "58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f"). InnerVolumeSpecName "kube-api-access-cz2jw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 01:33:31.120754 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:33:31.120693 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f" (UID: "58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 01:33:31.219914 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:33:31.219861 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cz2jw\" (UniqueName: \"kubernetes.io/projected/58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f-kube-api-access-cz2jw\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 01:33:31.219914 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:33:31.219907 2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f-console-oauth-config\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 01:33:31.219914 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:33:31.219918 2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f-service-ca\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 01:33:31.219914 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:33:31.219927 2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f-console-serving-cert\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 01:33:31.888864 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:33:31.888832 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84f857cd9d-pj9h4" Jul 06 01:33:31.888864 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:33:31.888844 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84f857cd9d-pj9h4" event={"ID":"58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f","Type":"ContainerDied","Data":"1fad576fa2923b8fa6f38e827301cff6662c2fdc4df15cd7d77a17e73de18d97"} Jul 06 01:33:31.889312 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:33:31.888896 2580 scope.go:117] "RemoveContainer" containerID="5a9477df5de9a11b442899e41dd514f088238f333a5968ab71aa30980872b594" Jul 06 01:33:31.909332 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:33:31.909302 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-84f857cd9d-pj9h4"] Jul 06 01:33:31.915715 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:33:31.915689 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-84f857cd9d-pj9h4"] Jul 06 01:33:32.461624 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:33:32.459886 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f" path="/var/lib/kubelet/pods/58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f/volumes" Jul 06 01:33:49.192063 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:33:49.192025 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:33:49.243968 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:33:49.243934 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:34:03.741890 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:34:03.741853 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:34:03.841282 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:34:03.841248 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:34:30.368921 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:34:30.368888 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:34:31.149078 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:34:31.149045 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:34:33.983636 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:34:33.983601 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:34:34.044025 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:34:34.043992 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:34:35.467503 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:34:35.467466 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:34:35.574936 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:34:35.574902 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:34:35.771054 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:34:35.770974 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:34:35.869966 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:34:35.869925 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:34:38.146958 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:34:38.146918 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:34:38.365885 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:34:38.365847 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:34:40.762314 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:34:40.762278 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:34:40.812070 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:34:40.812035 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:34:45.767268 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:34:45.767231 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:34:45.867963 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:34:45.867925 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:34:48.260726 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:34:48.260690 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:34:48.358904 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:34:48.358866 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:34:54.193062 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:34:54.193024 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:34:54.294649 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:34:54.294618 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:34:55.975428 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:34:55.975395 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:35:12.209215 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:12.209174 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:35:12.349484 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:12.349419 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:35:12.439483 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:12.439426 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:35:12.811007 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:12.810969 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:35:13.018483 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:13.018425 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:35:13.109920 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:13.109832 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:35:13.151522 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:13.151493 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-656987b587-lfrb4"] Jul 06 01:35:13.151805 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:13.151789 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f" containerName="console" Jul 06 01:35:13.151883 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:13.151809 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f" containerName="console" Jul 06 01:35:13.151883 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:13.151827 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe893ea8-ebec-4e52-95db-ef9285059853" containerName="console" Jul 06 01:35:13.151883 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:13.151836 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe893ea8-ebec-4e52-95db-ef9285059853" containerName="console" Jul 06 01:35:13.152027 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:13.151889 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="58f5ee6f-e0e9-4b81-a6f3-d658bd11da0f" containerName="console" Jul 06 01:35:13.152027 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:13.151903 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe893ea8-ebec-4e52-95db-ef9285059853" containerName="console" Jul 06 01:35:13.154720 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:13.154697 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-656987b587-lfrb4" Jul 06 01:35:13.157580 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:13.157558 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 06 01:35:13.157580 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:13.157572 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 06 01:35:13.157758 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:13.157567 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 06 01:35:13.157877 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:13.157862 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-jrd6l\"" Jul 06 01:35:13.157943 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:13.157896 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 06 01:35:13.175211 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:13.175183 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-656987b587-lfrb4"] Jul 06 01:35:13.230023 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:13.229990 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6bff963e-5246-4800-b50e-84afb8612738-tmp\") pod \"opendatahub-operator-controller-manager-656987b587-lfrb4\" (UID: \"6bff963e-5246-4800-b50e-84afb8612738\") " pod="opendatahub/opendatahub-operator-controller-manager-656987b587-lfrb4" Jul 06 01:35:13.230476 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:13.230048 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6bff963e-5246-4800-b50e-84afb8612738-apiservice-cert\") pod \"opendatahub-operator-controller-manager-656987b587-lfrb4\" (UID: \"6bff963e-5246-4800-b50e-84afb8612738\") " pod="opendatahub/opendatahub-operator-controller-manager-656987b587-lfrb4" Jul 06 01:35:13.230476 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:13.230093 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjgm5\" (UniqueName: \"kubernetes.io/projected/6bff963e-5246-4800-b50e-84afb8612738-kube-api-access-sjgm5\") pod \"opendatahub-operator-controller-manager-656987b587-lfrb4\" (UID: \"6bff963e-5246-4800-b50e-84afb8612738\") " pod="opendatahub/opendatahub-operator-controller-manager-656987b587-lfrb4" Jul 06 01:35:13.230476 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:13.230118 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6bff963e-5246-4800-b50e-84afb8612738-webhook-cert\") pod \"opendatahub-operator-controller-manager-656987b587-lfrb4\" (UID: \"6bff963e-5246-4800-b50e-84afb8612738\") " pod="opendatahub/opendatahub-operator-controller-manager-656987b587-lfrb4" Jul 06 01:35:13.230476 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:13.230161 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6bff963e-5246-4800-b50e-84afb8612738-manifests\") pod \"opendatahub-operator-controller-manager-656987b587-lfrb4\" (UID: \"6bff963e-5246-4800-b50e-84afb8612738\") " pod="opendatahub/opendatahub-operator-controller-manager-656987b587-lfrb4" Jul 06 01:35:13.254608 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:13.254574 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:35:13.331010 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:13.330971 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6bff963e-5246-4800-b50e-84afb8612738-apiservice-cert\") pod \"opendatahub-operator-controller-manager-656987b587-lfrb4\" (UID: \"6bff963e-5246-4800-b50e-84afb8612738\") " pod="opendatahub/opendatahub-operator-controller-manager-656987b587-lfrb4" Jul 06 01:35:13.331208 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:13.331018 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjgm5\" (UniqueName: \"kubernetes.io/projected/6bff963e-5246-4800-b50e-84afb8612738-kube-api-access-sjgm5\") pod \"opendatahub-operator-controller-manager-656987b587-lfrb4\" (UID: \"6bff963e-5246-4800-b50e-84afb8612738\") " pod="opendatahub/opendatahub-operator-controller-manager-656987b587-lfrb4" Jul 06 01:35:13.331208 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:13.331047 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6bff963e-5246-4800-b50e-84afb8612738-webhook-cert\") pod \"opendatahub-operator-controller-manager-656987b587-lfrb4\" (UID: \"6bff963e-5246-4800-b50e-84afb8612738\") " pod="opendatahub/opendatahub-operator-controller-manager-656987b587-lfrb4" Jul 06 01:35:13.331208 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:13.331079 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6bff963e-5246-4800-b50e-84afb8612738-manifests\") pod \"opendatahub-operator-controller-manager-656987b587-lfrb4\" (UID: \"6bff963e-5246-4800-b50e-84afb8612738\") " pod="opendatahub/opendatahub-operator-controller-manager-656987b587-lfrb4" Jul 06 01:35:13.331208 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:13.331115 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6bff963e-5246-4800-b50e-84afb8612738-tmp\") pod \"opendatahub-operator-controller-manager-656987b587-lfrb4\" (UID: \"6bff963e-5246-4800-b50e-84afb8612738\") " pod="opendatahub/opendatahub-operator-controller-manager-656987b587-lfrb4" Jul 06 01:35:13.331564 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:13.331539 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6bff963e-5246-4800-b50e-84afb8612738-manifests\") pod \"opendatahub-operator-controller-manager-656987b587-lfrb4\" (UID: \"6bff963e-5246-4800-b50e-84afb8612738\") " pod="opendatahub/opendatahub-operator-controller-manager-656987b587-lfrb4" Jul 06 01:35:13.333491 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:13.333472 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6bff963e-5246-4800-b50e-84afb8612738-tmp\") pod \"opendatahub-operator-controller-manager-656987b587-lfrb4\" (UID: \"6bff963e-5246-4800-b50e-84afb8612738\") " pod="opendatahub/opendatahub-operator-controller-manager-656987b587-lfrb4" Jul 06 01:35:13.333718 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:13.333701 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6bff963e-5246-4800-b50e-84afb8612738-apiservice-cert\") pod \"opendatahub-operator-controller-manager-656987b587-lfrb4\" (UID: \"6bff963e-5246-4800-b50e-84afb8612738\") " pod="opendatahub/opendatahub-operator-controller-manager-656987b587-lfrb4" Jul 06 01:35:13.333779 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:13.333737 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6bff963e-5246-4800-b50e-84afb8612738-webhook-cert\") pod \"opendatahub-operator-controller-manager-656987b587-lfrb4\" (UID: \"6bff963e-5246-4800-b50e-84afb8612738\") " pod="opendatahub/opendatahub-operator-controller-manager-656987b587-lfrb4" Jul 06 01:35:13.339479 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:13.339425 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjgm5\" (UniqueName: \"kubernetes.io/projected/6bff963e-5246-4800-b50e-84afb8612738-kube-api-access-sjgm5\") pod \"opendatahub-operator-controller-manager-656987b587-lfrb4\" (UID: \"6bff963e-5246-4800-b50e-84afb8612738\") " pod="opendatahub/opendatahub-operator-controller-manager-656987b587-lfrb4" Jul 06 01:35:13.353793 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:13.353759 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:35:13.454845 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:13.454778 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:35:13.464748 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:13.464713 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-656987b587-lfrb4" Jul 06 01:35:13.557753 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:13.557703 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:35:13.605854 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:13.605829 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-656987b587-lfrb4"] Jul 06 01:35:13.609054 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:35:13.609024 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bff963e_5246_4800_b50e_84afb8612738.slice/crio-4d7dee3861082907f01f86b3f26c4ef72e6cd72c67112cbdb877e5bd268611a0 WatchSource:0}: Error finding container 4d7dee3861082907f01f86b3f26c4ef72e6cd72c67112cbdb877e5bd268611a0: Status 404 returned error can't find the container with id 4d7dee3861082907f01f86b3f26c4ef72e6cd72c67112cbdb877e5bd268611a0 Jul 06 01:35:13.660601 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:13.660556 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:35:14.231621 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:14.231587 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-656987b587-lfrb4" event={"ID":"6bff963e-5246-4800-b50e-84afb8612738","Type":"ContainerStarted","Data":"4d7dee3861082907f01f86b3f26c4ef72e6cd72c67112cbdb877e5bd268611a0"} Jul 06 01:35:17.247663 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:17.247620 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-656987b587-lfrb4" event={"ID":"6bff963e-5246-4800-b50e-84afb8612738","Type":"ContainerStarted","Data":"0dcc24d8ba45b8b1e1516378beaa15043f15bc7fd3a11f8e85089ebaaf301243"} Jul 06 01:35:18.226599 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:18.226561 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:35:18.252336 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:18.252301 2580 generic.go:356] "Generic (PLEG): container finished" podID="6bff963e-5246-4800-b50e-84afb8612738" containerID="0dcc24d8ba45b8b1e1516378beaa15043f15bc7fd3a11f8e85089ebaaf301243" exitCode=0 Jul 06 01:35:18.252740 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:18.252375 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-656987b587-lfrb4" event={"ID":"6bff963e-5246-4800-b50e-84afb8612738","Type":"ContainerDied","Data":"0dcc24d8ba45b8b1e1516378beaa15043f15bc7fd3a11f8e85089ebaaf301243"} Jul 06 01:35:18.345087 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:18.345048 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:35:19.257686 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:19.257651 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-656987b587-lfrb4" event={"ID":"6bff963e-5246-4800-b50e-84afb8612738","Type":"ContainerStarted","Data":"76857820de787ba42b26fc6685336a0819e1500a39e623b268d5bc07d2cf8309"} Jul 06 01:35:19.258070 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:19.257762 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-656987b587-lfrb4" Jul 06 01:35:19.284684 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:19.284614 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-656987b587-lfrb4" podStartSLOduration=1.998385646 podStartE2EDuration="6.28459241s" totalImagesPullingTime="3.254258051s" totalInitContainerRuntime="1.031948713s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:35:13 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:35:19.283127862 +0000 UTC m=+513.254379553" watchObservedRunningTime="2026-07-06 01:35:19.28459241 +0000 UTC m=+513.255844104" Jul 06 01:35:30.263553 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:30.263522 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-656987b587-lfrb4" Jul 06 01:35:32.302182 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:32.302133 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:35:32.434674 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:32.434642 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:35:32.702845 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:32.702806 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:35:32.769465 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:32.769414 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7f644fb7f6-brmqh"] Jul 06 01:35:32.782798 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:32.782759 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7f644fb7f6-brmqh" Jul 06 01:35:32.787009 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:32.786980 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 06 01:35:32.787177 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:32.787142 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-vxsmh\"" Jul 06 01:35:32.787294 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:32.787275 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 06 01:35:32.787705 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:32.787683 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 06 01:35:32.788263 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:32.788220 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 06 01:35:32.788492 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:32.788470 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7f644fb7f6-brmqh"] Jul 06 01:35:32.871528 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:32.871492 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:35:32.879577 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:32.879552 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9a9dd62f-b9cd-4c63-bcae-d2d40082c5de-tmp\") pod \"kube-auth-proxy-7f644fb7f6-brmqh\" (UID: \"9a9dd62f-b9cd-4c63-bcae-d2d40082c5de\") " pod="openshift-ingress/kube-auth-proxy-7f644fb7f6-brmqh" Jul 06 01:35:32.879731 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:32.879585 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzh8s\" (UniqueName: \"kubernetes.io/projected/9a9dd62f-b9cd-4c63-bcae-d2d40082c5de-kube-api-access-tzh8s\") pod \"kube-auth-proxy-7f644fb7f6-brmqh\" (UID: \"9a9dd62f-b9cd-4c63-bcae-d2d40082c5de\") " pod="openshift-ingress/kube-auth-proxy-7f644fb7f6-brmqh" Jul 06 01:35:32.879731 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:32.879626 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a9dd62f-b9cd-4c63-bcae-d2d40082c5de-tls-certs\") pod \"kube-auth-proxy-7f644fb7f6-brmqh\" (UID: \"9a9dd62f-b9cd-4c63-bcae-d2d40082c5de\") " pod="openshift-ingress/kube-auth-proxy-7f644fb7f6-brmqh" Jul 06 01:35:32.950316 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:32.950283 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-894jd"] Jul 06 01:35:32.953600 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:32.953544 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-894jd" Jul 06 01:35:32.956547 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:32.956523 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 06 01:35:32.956658 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:32.956579 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-fthxp\"" Jul 06 01:35:32.966095 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:32.966071 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-894jd"] Jul 06 01:35:32.980535 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:32.980507 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a9dd62f-b9cd-4c63-bcae-d2d40082c5de-tls-certs\") pod \"kube-auth-proxy-7f644fb7f6-brmqh\" (UID: \"9a9dd62f-b9cd-4c63-bcae-d2d40082c5de\") " pod="openshift-ingress/kube-auth-proxy-7f644fb7f6-brmqh" Jul 06 01:35:32.980702 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:32.980566 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9a9dd62f-b9cd-4c63-bcae-d2d40082c5de-tmp\") pod \"kube-auth-proxy-7f644fb7f6-brmqh\" (UID: \"9a9dd62f-b9cd-4c63-bcae-d2d40082c5de\") " pod="openshift-ingress/kube-auth-proxy-7f644fb7f6-brmqh" Jul 06 01:35:32.980702 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:32.980591 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzh8s\" (UniqueName: \"kubernetes.io/projected/9a9dd62f-b9cd-4c63-bcae-d2d40082c5de-kube-api-access-tzh8s\") pod \"kube-auth-proxy-7f644fb7f6-brmqh\" (UID: \"9a9dd62f-b9cd-4c63-bcae-d2d40082c5de\") " pod="openshift-ingress/kube-auth-proxy-7f644fb7f6-brmqh" Jul 06 01:35:32.985011 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:32.984972 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a9dd62f-b9cd-4c63-bcae-d2d40082c5de-tls-certs\") pod \"kube-auth-proxy-7f644fb7f6-brmqh\" (UID: \"9a9dd62f-b9cd-4c63-bcae-d2d40082c5de\") " pod="openshift-ingress/kube-auth-proxy-7f644fb7f6-brmqh" Jul 06 01:35:32.987178 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:32.987152 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9a9dd62f-b9cd-4c63-bcae-d2d40082c5de-tmp\") pod \"kube-auth-proxy-7f644fb7f6-brmqh\" (UID: \"9a9dd62f-b9cd-4c63-bcae-d2d40082c5de\") " pod="openshift-ingress/kube-auth-proxy-7f644fb7f6-brmqh" Jul 06 01:35:32.993090 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:32.993063 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzh8s\" (UniqueName: \"kubernetes.io/projected/9a9dd62f-b9cd-4c63-bcae-d2d40082c5de-kube-api-access-tzh8s\") pod \"kube-auth-proxy-7f644fb7f6-brmqh\" (UID: \"9a9dd62f-b9cd-4c63-bcae-d2d40082c5de\") " pod="openshift-ingress/kube-auth-proxy-7f644fb7f6-brmqh" Jul 06 01:35:32.993394 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:32.993364 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:35:33.076689 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:33.076652 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:35:33.081610 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:33.081585 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e9e51d11-b969-4794-ae2e-b34b00bd635b-cert\") pod \"odh-model-controller-7998c67dc4-894jd\" (UID: \"e9e51d11-b969-4794-ae2e-b34b00bd635b\") " pod="opendatahub/odh-model-controller-7998c67dc4-894jd" Jul 06 01:35:33.081724 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:33.081629 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sp59d\" (UniqueName: \"kubernetes.io/projected/e9e51d11-b969-4794-ae2e-b34b00bd635b-kube-api-access-sp59d\") pod \"odh-model-controller-7998c67dc4-894jd\" (UID: \"e9e51d11-b969-4794-ae2e-b34b00bd635b\") " pod="opendatahub/odh-model-controller-7998c67dc4-894jd" Jul 06 01:35:33.094556 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:33.094530 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7f644fb7f6-brmqh" Jul 06 01:35:33.182490 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:33.181988 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sp59d\" (UniqueName: \"kubernetes.io/projected/e9e51d11-b969-4794-ae2e-b34b00bd635b-kube-api-access-sp59d\") pod \"odh-model-controller-7998c67dc4-894jd\" (UID: \"e9e51d11-b969-4794-ae2e-b34b00bd635b\") " pod="opendatahub/odh-model-controller-7998c67dc4-894jd" Jul 06 01:35:33.182490 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:33.182096 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e9e51d11-b969-4794-ae2e-b34b00bd635b-cert\") pod \"odh-model-controller-7998c67dc4-894jd\" (UID: \"e9e51d11-b969-4794-ae2e-b34b00bd635b\") " pod="opendatahub/odh-model-controller-7998c67dc4-894jd" Jul 06 01:35:33.182490 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:35:33.182246 2580 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 06 01:35:33.182490 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:35:33.182310 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e9e51d11-b969-4794-ae2e-b34b00bd635b-cert podName:e9e51d11-b969-4794-ae2e-b34b00bd635b nodeName:}" failed. No retries permitted until 2026-07-06 01:35:33.682287738 +0000 UTC m=+527.653539428 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/e9e51d11-b969-4794-ae2e-b34b00bd635b-cert") pod "odh-model-controller-7998c67dc4-894jd" (UID: "e9e51d11-b969-4794-ae2e-b34b00bd635b") : secret "odh-model-controller-webhook-cert" not found Jul 06 01:35:33.193941 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:33.193911 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sp59d\" (UniqueName: \"kubernetes.io/projected/e9e51d11-b969-4794-ae2e-b34b00bd635b-kube-api-access-sp59d\") pod \"odh-model-controller-7998c67dc4-894jd\" (UID: \"e9e51d11-b969-4794-ae2e-b34b00bd635b\") " pod="opendatahub/odh-model-controller-7998c67dc4-894jd" Jul 06 01:35:33.245195 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:33.245109 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7f644fb7f6-brmqh"] Jul 06 01:35:33.249288 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:35:33.249259 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a9dd62f_b9cd_4c63_bcae_d2d40082c5de.slice/crio-c495ab246c1958712cb163b2c0b602b185fb2963ac0c69e49db8e63a54b965fb WatchSource:0}: Error finding container c495ab246c1958712cb163b2c0b602b185fb2963ac0c69e49db8e63a54b965fb: Status 404 returned error can't find the container with id c495ab246c1958712cb163b2c0b602b185fb2963ac0c69e49db8e63a54b965fb Jul 06 01:35:33.261377 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:33.261338 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:35:33.308513 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:33.308473 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7f644fb7f6-brmqh" event={"ID":"9a9dd62f-b9cd-4c63-bcae-d2d40082c5de","Type":"ContainerStarted","Data":"c495ab246c1958712cb163b2c0b602b185fb2963ac0c69e49db8e63a54b965fb"} Jul 06 01:35:33.483387 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:33.483349 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:35:33.545193 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:33.545110 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-cdlg2"] Jul 06 01:35:33.549755 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:33.549737 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-cdlg2" Jul 06 01:35:33.552384 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:33.552359 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 06 01:35:33.559832 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:33.559810 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-cdlg2"] Jul 06 01:35:33.685245 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:33.685212 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e9e51d11-b969-4794-ae2e-b34b00bd635b-cert\") pod \"odh-model-controller-7998c67dc4-894jd\" (UID: \"e9e51d11-b969-4794-ae2e-b34b00bd635b\") " pod="opendatahub/odh-model-controller-7998c67dc4-894jd" Jul 06 01:35:33.685469 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:33.685260 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22e8a402-6186-4a45-a5c7-20bb3975c6a3-tls-certs\") pod \"model-serving-api-54d89bd5fc-cdlg2\" (UID: \"22e8a402-6186-4a45-a5c7-20bb3975c6a3\") " pod="opendatahub/model-serving-api-54d89bd5fc-cdlg2" Jul 06 01:35:33.685469 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:33.685303 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7bbj\" (UniqueName: \"kubernetes.io/projected/22e8a402-6186-4a45-a5c7-20bb3975c6a3-kube-api-access-f7bbj\") pod \"model-serving-api-54d89bd5fc-cdlg2\" (UID: \"22e8a402-6186-4a45-a5c7-20bb3975c6a3\") " pod="opendatahub/model-serving-api-54d89bd5fc-cdlg2" Jul 06 01:35:33.688107 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:33.688078 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e9e51d11-b969-4794-ae2e-b34b00bd635b-cert\") pod \"odh-model-controller-7998c67dc4-894jd\" (UID: \"e9e51d11-b969-4794-ae2e-b34b00bd635b\") " pod="opendatahub/odh-model-controller-7998c67dc4-894jd" Jul 06 01:35:33.786183 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:33.786132 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7bbj\" (UniqueName: \"kubernetes.io/projected/22e8a402-6186-4a45-a5c7-20bb3975c6a3-kube-api-access-f7bbj\") pod \"model-serving-api-54d89bd5fc-cdlg2\" (UID: \"22e8a402-6186-4a45-a5c7-20bb3975c6a3\") " pod="opendatahub/model-serving-api-54d89bd5fc-cdlg2" Jul 06 01:35:33.786387 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:33.786201 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22e8a402-6186-4a45-a5c7-20bb3975c6a3-tls-certs\") pod \"model-serving-api-54d89bd5fc-cdlg2\" (UID: \"22e8a402-6186-4a45-a5c7-20bb3975c6a3\") " pod="opendatahub/model-serving-api-54d89bd5fc-cdlg2" Jul 06 01:35:33.788867 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:33.788838 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22e8a402-6186-4a45-a5c7-20bb3975c6a3-tls-certs\") pod \"model-serving-api-54d89bd5fc-cdlg2\" (UID: \"22e8a402-6186-4a45-a5c7-20bb3975c6a3\") " pod="opendatahub/model-serving-api-54d89bd5fc-cdlg2" Jul 06 01:35:33.797935 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:33.797845 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:35:33.799468 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:33.799414 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7bbj\" (UniqueName: \"kubernetes.io/projected/22e8a402-6186-4a45-a5c7-20bb3975c6a3-kube-api-access-f7bbj\") pod \"model-serving-api-54d89bd5fc-cdlg2\" (UID: \"22e8a402-6186-4a45-a5c7-20bb3975c6a3\") " pod="opendatahub/model-serving-api-54d89bd5fc-cdlg2" Jul 06 01:35:33.860618 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:33.860582 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-cdlg2" Jul 06 01:35:33.862689 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:33.862659 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-894jd" Jul 06 01:35:34.042137 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:34.041982 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-cdlg2"] Jul 06 01:35:34.045590 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:35:34.045554 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22e8a402_6186_4a45_a5c7_20bb3975c6a3.slice/crio-cd777596688010e2ea8fe5f11044a72538164bf93690af5bb1217bc2e40bc9e9 WatchSource:0}: Error finding container cd777596688010e2ea8fe5f11044a72538164bf93690af5bb1217bc2e40bc9e9: Status 404 returned error can't find the container with id cd777596688010e2ea8fe5f11044a72538164bf93690af5bb1217bc2e40bc9e9 Jul 06 01:35:34.047588 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:34.047559 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:35:34.055311 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:34.055288 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-t9sq7"] Jul 06 01:35:34.059710 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:34.059686 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-t9sq7" Jul 06 01:35:34.062143 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:34.062116 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 06 01:35:34.062315 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:34.062292 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-7nxcc\"" Jul 06 01:35:34.062539 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:34.062425 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 06 01:35:34.062607 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:34.062578 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 06 01:35:34.062666 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:34.062582 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 06 01:35:34.068585 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:34.068214 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-894jd"] Jul 06 01:35:34.072265 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:35:34.072235 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9e51d11_b969_4794_ae2e_b34b00bd635b.slice/crio-04f9af92e99539b147166d404f1f4eb0658ad81e3d64470c32ba757c466a868e WatchSource:0}: Error finding container 04f9af92e99539b147166d404f1f4eb0658ad81e3d64470c32ba757c466a868e: Status 404 returned error can't find the container with id 04f9af92e99539b147166d404f1f4eb0658ad81e3d64470c32ba757c466a868e Jul 06 01:35:34.075251 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:34.075185 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-t9sq7"] Jul 06 01:35:34.151288 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:34.151240 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:35:34.189736 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:34.189698 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/91c3a3db-b9c8-445d-82c9-3614834c3e8a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-t9sq7\" (UID: \"91c3a3db-b9c8-445d-82c9-3614834c3e8a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-t9sq7" Jul 06 01:35:34.189941 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:34.189755 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91c3a3db-b9c8-445d-82c9-3614834c3e8a-istio-token\") pod \"istiod-openshift-gateway-7574459656-t9sq7\" (UID: \"91c3a3db-b9c8-445d-82c9-3614834c3e8a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-t9sq7" Jul 06 01:35:34.189941 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:34.189782 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/91c3a3db-b9c8-445d-82c9-3614834c3e8a-cacerts\") pod \"istiod-openshift-gateway-7574459656-t9sq7\" (UID: \"91c3a3db-b9c8-445d-82c9-3614834c3e8a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-t9sq7" Jul 06 01:35:34.189941 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:34.189844 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/91c3a3db-b9c8-445d-82c9-3614834c3e8a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-t9sq7\" (UID: \"91c3a3db-b9c8-445d-82c9-3614834c3e8a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-t9sq7" Jul 06 01:35:34.189941 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:34.189896 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/91c3a3db-b9c8-445d-82c9-3614834c3e8a-local-certs\") pod \"istiod-openshift-gateway-7574459656-t9sq7\" (UID: \"91c3a3db-b9c8-445d-82c9-3614834c3e8a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-t9sq7" Jul 06 01:35:34.189941 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:34.189936 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvv44\" (UniqueName: \"kubernetes.io/projected/91c3a3db-b9c8-445d-82c9-3614834c3e8a-kube-api-access-qvv44\") pod \"istiod-openshift-gateway-7574459656-t9sq7\" (UID: \"91c3a3db-b9c8-445d-82c9-3614834c3e8a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-t9sq7" Jul 06 01:35:34.190180 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:34.189965 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/91c3a3db-b9c8-445d-82c9-3614834c3e8a-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-t9sq7\" (UID: \"91c3a3db-b9c8-445d-82c9-3614834c3e8a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-t9sq7" Jul 06 01:35:34.291246 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:34.291205 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91c3a3db-b9c8-445d-82c9-3614834c3e8a-istio-token\") pod \"istiod-openshift-gateway-7574459656-t9sq7\" (UID: \"91c3a3db-b9c8-445d-82c9-3614834c3e8a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-t9sq7" Jul 06 01:35:34.291246 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:34.291253 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/91c3a3db-b9c8-445d-82c9-3614834c3e8a-cacerts\") pod \"istiod-openshift-gateway-7574459656-t9sq7\" (UID: \"91c3a3db-b9c8-445d-82c9-3614834c3e8a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-t9sq7" Jul 06 01:35:34.291635 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:34.291276 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/91c3a3db-b9c8-445d-82c9-3614834c3e8a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-t9sq7\" (UID: \"91c3a3db-b9c8-445d-82c9-3614834c3e8a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-t9sq7" Jul 06 01:35:34.291635 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:34.291319 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/91c3a3db-b9c8-445d-82c9-3614834c3e8a-local-certs\") pod \"istiod-openshift-gateway-7574459656-t9sq7\" (UID: \"91c3a3db-b9c8-445d-82c9-3614834c3e8a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-t9sq7" Jul 06 01:35:34.291635 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:34.291359 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvv44\" (UniqueName: \"kubernetes.io/projected/91c3a3db-b9c8-445d-82c9-3614834c3e8a-kube-api-access-qvv44\") pod \"istiod-openshift-gateway-7574459656-t9sq7\" (UID: \"91c3a3db-b9c8-445d-82c9-3614834c3e8a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-t9sq7" Jul 06 01:35:34.291635 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:34.291384 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/91c3a3db-b9c8-445d-82c9-3614834c3e8a-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-t9sq7\" (UID: \"91c3a3db-b9c8-445d-82c9-3614834c3e8a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-t9sq7" Jul 06 01:35:34.291635 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:34.291467 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/91c3a3db-b9c8-445d-82c9-3614834c3e8a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-t9sq7\" (UID: \"91c3a3db-b9c8-445d-82c9-3614834c3e8a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-t9sq7" Jul 06 01:35:34.292125 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:34.292077 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/91c3a3db-b9c8-445d-82c9-3614834c3e8a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-t9sq7\" (UID: \"91c3a3db-b9c8-445d-82c9-3614834c3e8a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-t9sq7" Jul 06 01:35:34.294806 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:34.294776 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/91c3a3db-b9c8-445d-82c9-3614834c3e8a-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-t9sq7\" (UID: \"91c3a3db-b9c8-445d-82c9-3614834c3e8a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-t9sq7" Jul 06 01:35:34.294919 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:34.294778 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/91c3a3db-b9c8-445d-82c9-3614834c3e8a-local-certs\") pod \"istiod-openshift-gateway-7574459656-t9sq7\" (UID: \"91c3a3db-b9c8-445d-82c9-3614834c3e8a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-t9sq7" Jul 06 01:35:34.294919 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:34.294865 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/91c3a3db-b9c8-445d-82c9-3614834c3e8a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-t9sq7\" (UID: \"91c3a3db-b9c8-445d-82c9-3614834c3e8a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-t9sq7" Jul 06 01:35:34.295309 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:34.295286 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/91c3a3db-b9c8-445d-82c9-3614834c3e8a-cacerts\") pod \"istiod-openshift-gateway-7574459656-t9sq7\" (UID: \"91c3a3db-b9c8-445d-82c9-3614834c3e8a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-t9sq7" Jul 06 01:35:34.300628 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:34.300543 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvv44\" (UniqueName: \"kubernetes.io/projected/91c3a3db-b9c8-445d-82c9-3614834c3e8a-kube-api-access-qvv44\") pod \"istiod-openshift-gateway-7574459656-t9sq7\" (UID: \"91c3a3db-b9c8-445d-82c9-3614834c3e8a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-t9sq7" Jul 06 01:35:34.301326 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:34.301303 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91c3a3db-b9c8-445d-82c9-3614834c3e8a-istio-token\") pod \"istiod-openshift-gateway-7574459656-t9sq7\" (UID: \"91c3a3db-b9c8-445d-82c9-3614834c3e8a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-t9sq7" Jul 06 01:35:34.314588 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:34.314550 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-cdlg2" event={"ID":"22e8a402-6186-4a45-a5c7-20bb3975c6a3","Type":"ContainerStarted","Data":"cd777596688010e2ea8fe5f11044a72538164bf93690af5bb1217bc2e40bc9e9"} Jul 06 01:35:34.316004 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:34.315971 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-894jd" event={"ID":"e9e51d11-b969-4794-ae2e-b34b00bd635b","Type":"ContainerStarted","Data":"04f9af92e99539b147166d404f1f4eb0658ad81e3d64470c32ba757c466a868e"} Jul 06 01:35:34.371940 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:34.371905 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-t9sq7" Jul 06 01:35:34.525372 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:34.525342 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-t9sq7"] Jul 06 01:35:34.528136 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:35:34.528109 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91c3a3db_b9c8_445d_82c9_3614834c3e8a.slice/crio-02d7c45ef7d4d06978f07884a19b3b464519fcd8b9c6d3c92e014b5dd265ad2b WatchSource:0}: Error finding container 02d7c45ef7d4d06978f07884a19b3b464519fcd8b9c6d3c92e014b5dd265ad2b: Status 404 returned error can't find the container with id 02d7c45ef7d4d06978f07884a19b3b464519fcd8b9c6d3c92e014b5dd265ad2b Jul 06 01:35:35.018458 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:35.018323 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:35:35.328608 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:35.328519 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-t9sq7" event={"ID":"91c3a3db-b9c8-445d-82c9-3614834c3e8a","Type":"ContainerStarted","Data":"02d7c45ef7d4d06978f07884a19b3b464519fcd8b9c6d3c92e014b5dd265ad2b"} Jul 06 01:35:39.199577 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:39.199530 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 01:35:39.199908 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:39.199612 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 01:35:39.348641 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:39.348600 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-t9sq7" event={"ID":"91c3a3db-b9c8-445d-82c9-3614834c3e8a","Type":"ContainerStarted","Data":"9fc642bd36acf93f3baddaf4b7f96083567d717d59562dd24a98072b9277eebf"} Jul 06 01:35:39.348944 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:39.348743 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-t9sq7" Jul 06 01:35:39.350603 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:39.350572 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-894jd" event={"ID":"e9e51d11-b969-4794-ae2e-b34b00bd635b","Type":"ContainerStarted","Data":"bd5cccc06cc33fd8090b965f2853e4fbedf41feafafbbf849c6df81a22364784"} Jul 06 01:35:39.350743 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:39.350694 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-894jd" Jul 06 01:35:39.352172 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:39.352148 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7f644fb7f6-brmqh" event={"ID":"9a9dd62f-b9cd-4c63-bcae-d2d40082c5de","Type":"ContainerStarted","Data":"b08aedc86506e206f9a087b5bc653920c09d6e1493dde0af5d4ee7bf12bb12d6"} Jul 06 01:35:39.353515 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:39.353489 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-cdlg2" event={"ID":"22e8a402-6186-4a45-a5c7-20bb3975c6a3","Type":"ContainerStarted","Data":"fc9dc41f71c340077249f7d994e31977c9a8e84cb325421760cf942e65697e18"} Jul 06 01:35:39.353907 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:39.353889 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-cdlg2" Jul 06 01:35:39.369354 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:39.369310 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-t9sq7" podStartSLOduration=0.700112648 podStartE2EDuration="5.369296764s" totalImagesPullingTime="4.669184116s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:35:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:35:39.367582861 +0000 UTC m=+533.338834554" watchObservedRunningTime="2026-07-06 01:35:39.369296764 +0000 UTC m=+533.340548433" Jul 06 01:35:39.383006 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:39.382953 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-894jd" podStartSLOduration=2.250300276 podStartE2EDuration="7.382936811s" totalImagesPullingTime="5.132636535s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:35:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:35:39.381798943 +0000 UTC m=+533.353050635" watchObservedRunningTime="2026-07-06 01:35:39.382936811 +0000 UTC m=+533.354188502" Jul 06 01:35:39.398450 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:39.398372 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7f644fb7f6-brmqh" podStartSLOduration=1.456626596 podStartE2EDuration="7.398353838s" totalImagesPullingTime="5.941727242s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:35:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:35:39.396773172 +0000 UTC m=+533.368024864" watchObservedRunningTime="2026-07-06 01:35:39.398353838 +0000 UTC m=+533.369605528" Jul 06 01:35:39.414550 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:39.414421 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-cdlg2" podStartSLOduration=1.265735556 podStartE2EDuration="6.414397763s" totalImagesPullingTime="5.148662207s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:35:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:35:39.412947563 +0000 UTC m=+533.384199256" watchObservedRunningTime="2026-07-06 01:35:39.414397763 +0000 UTC m=+533.385649454" Jul 06 01:35:39.851461 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:39.851403 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:35:39.948352 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:39.948316 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:35:40.359292 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:40.359255 2580 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-t9sq7 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 06 01:35:40.359684 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:40.359310 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-t9sq7" podUID="91c3a3db-b9c8-445d-82c9-3614834c3e8a" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 01:35:40.588637 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:40.588603 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:35:40.748423 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:40.748386 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:35:41.113887 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:41.113777 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:35:42.699672 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:42.699639 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-577bbf7cf9-xqvfn"] Jul 06 01:35:42.703006 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:42.702986 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-577bbf7cf9-xqvfn" Jul 06 01:35:42.706425 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:42.706408 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 06 01:35:42.706544 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:42.706409 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 06 01:35:42.707105 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:42.707089 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 06 01:35:42.707520 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:42.707505 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 06 01:35:42.707569 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:42.707535 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-c52pm\"" Jul 06 01:35:42.707636 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:42.707621 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 06 01:35:42.716141 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:42.716116 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-577bbf7cf9-xqvfn"] Jul 06 01:35:42.865016 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:42.864976 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/cfbbbe93-c39e-4799-8aa6-bb2ca44d790d-metrics-cert\") pod \"lws-controller-manager-577bbf7cf9-xqvfn\" (UID: \"cfbbbe93-c39e-4799-8aa6-bb2ca44d790d\") " pod="openshift-lws-operator/lws-controller-manager-577bbf7cf9-xqvfn" Jul 06 01:35:42.865187 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:42.865034 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/cfbbbe93-c39e-4799-8aa6-bb2ca44d790d-manager-config\") pod \"lws-controller-manager-577bbf7cf9-xqvfn\" (UID: \"cfbbbe93-c39e-4799-8aa6-bb2ca44d790d\") " pod="openshift-lws-operator/lws-controller-manager-577bbf7cf9-xqvfn" Jul 06 01:35:42.865187 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:42.865099 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xptzb\" (UniqueName: \"kubernetes.io/projected/cfbbbe93-c39e-4799-8aa6-bb2ca44d790d-kube-api-access-xptzb\") pod \"lws-controller-manager-577bbf7cf9-xqvfn\" (UID: \"cfbbbe93-c39e-4799-8aa6-bb2ca44d790d\") " pod="openshift-lws-operator/lws-controller-manager-577bbf7cf9-xqvfn" Jul 06 01:35:42.865187 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:42.865155 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cfbbbe93-c39e-4799-8aa6-bb2ca44d790d-cert\") pod \"lws-controller-manager-577bbf7cf9-xqvfn\" (UID: \"cfbbbe93-c39e-4799-8aa6-bb2ca44d790d\") " pod="openshift-lws-operator/lws-controller-manager-577bbf7cf9-xqvfn" Jul 06 01:35:42.965593 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:42.965506 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/cfbbbe93-c39e-4799-8aa6-bb2ca44d790d-manager-config\") pod \"lws-controller-manager-577bbf7cf9-xqvfn\" (UID: \"cfbbbe93-c39e-4799-8aa6-bb2ca44d790d\") " pod="openshift-lws-operator/lws-controller-manager-577bbf7cf9-xqvfn" Jul 06 01:35:42.965593 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:42.965544 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xptzb\" (UniqueName: \"kubernetes.io/projected/cfbbbe93-c39e-4799-8aa6-bb2ca44d790d-kube-api-access-xptzb\") pod \"lws-controller-manager-577bbf7cf9-xqvfn\" (UID: \"cfbbbe93-c39e-4799-8aa6-bb2ca44d790d\") " pod="openshift-lws-operator/lws-controller-manager-577bbf7cf9-xqvfn" Jul 06 01:35:42.965826 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:42.965677 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cfbbbe93-c39e-4799-8aa6-bb2ca44d790d-cert\") pod \"lws-controller-manager-577bbf7cf9-xqvfn\" (UID: \"cfbbbe93-c39e-4799-8aa6-bb2ca44d790d\") " pod="openshift-lws-operator/lws-controller-manager-577bbf7cf9-xqvfn" Jul 06 01:35:42.965826 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:42.965737 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/cfbbbe93-c39e-4799-8aa6-bb2ca44d790d-metrics-cert\") pod \"lws-controller-manager-577bbf7cf9-xqvfn\" (UID: \"cfbbbe93-c39e-4799-8aa6-bb2ca44d790d\") " pod="openshift-lws-operator/lws-controller-manager-577bbf7cf9-xqvfn" Jul 06 01:35:42.966251 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:42.966223 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/cfbbbe93-c39e-4799-8aa6-bb2ca44d790d-manager-config\") pod \"lws-controller-manager-577bbf7cf9-xqvfn\" (UID: \"cfbbbe93-c39e-4799-8aa6-bb2ca44d790d\") " pod="openshift-lws-operator/lws-controller-manager-577bbf7cf9-xqvfn" Jul 06 01:35:42.968859 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:42.968835 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cfbbbe93-c39e-4799-8aa6-bb2ca44d790d-cert\") pod \"lws-controller-manager-577bbf7cf9-xqvfn\" (UID: \"cfbbbe93-c39e-4799-8aa6-bb2ca44d790d\") " pod="openshift-lws-operator/lws-controller-manager-577bbf7cf9-xqvfn" Jul 06 01:35:42.968967 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:42.968931 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/cfbbbe93-c39e-4799-8aa6-bb2ca44d790d-metrics-cert\") pod \"lws-controller-manager-577bbf7cf9-xqvfn\" (UID: \"cfbbbe93-c39e-4799-8aa6-bb2ca44d790d\") " pod="openshift-lws-operator/lws-controller-manager-577bbf7cf9-xqvfn" Jul 06 01:35:42.982280 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:42.982246 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xptzb\" (UniqueName: \"kubernetes.io/projected/cfbbbe93-c39e-4799-8aa6-bb2ca44d790d-kube-api-access-xptzb\") pod \"lws-controller-manager-577bbf7cf9-xqvfn\" (UID: \"cfbbbe93-c39e-4799-8aa6-bb2ca44d790d\") " pod="openshift-lws-operator/lws-controller-manager-577bbf7cf9-xqvfn" Jul 06 01:35:43.013944 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:43.013907 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-577bbf7cf9-xqvfn" Jul 06 01:35:43.149945 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:43.149921 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-577bbf7cf9-xqvfn"] Jul 06 01:35:43.152495 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:35:43.152470 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfbbbe93_c39e_4799_8aa6_bb2ca44d790d.slice/crio-c3df2401c64638ecb27ba83457c8bb0a54d08ed0e8e9d1cf03954066de70d351 WatchSource:0}: Error finding container c3df2401c64638ecb27ba83457c8bb0a54d08ed0e8e9d1cf03954066de70d351: Status 404 returned error can't find the container with id c3df2401c64638ecb27ba83457c8bb0a54d08ed0e8e9d1cf03954066de70d351 Jul 06 01:35:43.358594 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:43.358506 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-t9sq7" Jul 06 01:35:43.370271 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:43.370234 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-577bbf7cf9-xqvfn" event={"ID":"cfbbbe93-c39e-4799-8aa6-bb2ca44d790d","Type":"ContainerStarted","Data":"c3df2401c64638ecb27ba83457c8bb0a54d08ed0e8e9d1cf03954066de70d351"} Jul 06 01:35:44.080726 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:44.080685 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:35:44.165966 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:44.165913 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:35:44.251860 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:44.251801 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:35:44.764328 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:44.764295 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:35:45.374680 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:45.374640 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:35:45.527465 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:45.527393 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:35:46.389423 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:46.389380 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-577bbf7cf9-xqvfn" event={"ID":"cfbbbe93-c39e-4799-8aa6-bb2ca44d790d","Type":"ContainerStarted","Data":"9e10ae49433a2c9e888dafc78f3e2e425ae511cbeacb5bd08d3584c7eb14a7ed"} Jul 06 01:35:46.389947 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:46.389577 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-577bbf7cf9-xqvfn" Jul 06 01:35:46.420052 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:46.419988 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-577bbf7cf9-xqvfn" podStartSLOduration=1.463170468 podStartE2EDuration="4.419971553s" totalImagesPullingTime="2.956801085s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:35:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:35:46.419755164 +0000 UTC m=+540.391006855" watchObservedRunningTime="2026-07-06 01:35:46.419971553 +0000 UTC m=+540.391223244" Jul 06 01:35:48.476948 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:48.476907 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-t9sq7"] Jul 06 01:35:48.477405 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:48.477155 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-t9sq7" podUID="91c3a3db-b9c8-445d-82c9-3614834c3e8a" containerName="discovery" containerID="cri-o://9fc642bd36acf93f3baddaf4b7f96083567d717d59562dd24a98072b9277eebf" gracePeriod=30 Jul 06 01:35:48.723780 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:48.723753 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-t9sq7" Jul 06 01:35:48.812291 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:48.812197 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/91c3a3db-b9c8-445d-82c9-3614834c3e8a-local-certs\") pod \"91c3a3db-b9c8-445d-82c9-3614834c3e8a\" (UID: \"91c3a3db-b9c8-445d-82c9-3614834c3e8a\") " Jul 06 01:35:48.812291 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:48.812242 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91c3a3db-b9c8-445d-82c9-3614834c3e8a-istio-token\") pod \"91c3a3db-b9c8-445d-82c9-3614834c3e8a\" (UID: \"91c3a3db-b9c8-445d-82c9-3614834c3e8a\") " Jul 06 01:35:48.812291 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:48.812265 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/91c3a3db-b9c8-445d-82c9-3614834c3e8a-cacerts\") pod \"91c3a3db-b9c8-445d-82c9-3614834c3e8a\" (UID: \"91c3a3db-b9c8-445d-82c9-3614834c3e8a\") " Jul 06 01:35:48.812291 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:48.812288 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qvv44\" (UniqueName: \"kubernetes.io/projected/91c3a3db-b9c8-445d-82c9-3614834c3e8a-kube-api-access-qvv44\") pod \"91c3a3db-b9c8-445d-82c9-3614834c3e8a\" (UID: \"91c3a3db-b9c8-445d-82c9-3614834c3e8a\") " Jul 06 01:35:48.812647 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:48.812337 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/91c3a3db-b9c8-445d-82c9-3614834c3e8a-istio-csr-ca-configmap\") pod \"91c3a3db-b9c8-445d-82c9-3614834c3e8a\" (UID: \"91c3a3db-b9c8-445d-82c9-3614834c3e8a\") " Jul 06 01:35:48.812647 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:48.812509 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/91c3a3db-b9c8-445d-82c9-3614834c3e8a-istio-csr-dns-cert\") pod \"91c3a3db-b9c8-445d-82c9-3614834c3e8a\" (UID: \"91c3a3db-b9c8-445d-82c9-3614834c3e8a\") " Jul 06 01:35:48.812647 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:48.812548 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/91c3a3db-b9c8-445d-82c9-3614834c3e8a-istio-kubeconfig\") pod \"91c3a3db-b9c8-445d-82c9-3614834c3e8a\" (UID: \"91c3a3db-b9c8-445d-82c9-3614834c3e8a\") " Jul 06 01:35:48.812810 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:48.812784 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91c3a3db-b9c8-445d-82c9-3614834c3e8a-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "91c3a3db-b9c8-445d-82c9-3614834c3e8a" (UID: "91c3a3db-b9c8-445d-82c9-3614834c3e8a"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 01:35:48.814888 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:48.814857 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91c3a3db-b9c8-445d-82c9-3614834c3e8a-local-certs" (OuterVolumeSpecName: "local-certs") pod "91c3a3db-b9c8-445d-82c9-3614834c3e8a" (UID: "91c3a3db-b9c8-445d-82c9-3614834c3e8a"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:35:48.814888 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:48.814867 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91c3a3db-b9c8-445d-82c9-3614834c3e8a-kube-api-access-qvv44" (OuterVolumeSpecName: "kube-api-access-qvv44") pod "91c3a3db-b9c8-445d-82c9-3614834c3e8a" (UID: "91c3a3db-b9c8-445d-82c9-3614834c3e8a"). InnerVolumeSpecName "kube-api-access-qvv44". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 01:35:48.815181 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:48.815161 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91c3a3db-b9c8-445d-82c9-3614834c3e8a-cacerts" (OuterVolumeSpecName: "cacerts") pod "91c3a3db-b9c8-445d-82c9-3614834c3e8a" (UID: "91c3a3db-b9c8-445d-82c9-3614834c3e8a"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 01:35:48.815246 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:48.815214 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91c3a3db-b9c8-445d-82c9-3614834c3e8a-istio-token" (OuterVolumeSpecName: "istio-token") pod "91c3a3db-b9c8-445d-82c9-3614834c3e8a" (UID: "91c3a3db-b9c8-445d-82c9-3614834c3e8a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 01:35:48.815246 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:48.815211 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91c3a3db-b9c8-445d-82c9-3614834c3e8a-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "91c3a3db-b9c8-445d-82c9-3614834c3e8a" (UID: "91c3a3db-b9c8-445d-82c9-3614834c3e8a"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 01:35:48.815314 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:48.815270 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91c3a3db-b9c8-445d-82c9-3614834c3e8a-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "91c3a3db-b9c8-445d-82c9-3614834c3e8a" (UID: "91c3a3db-b9c8-445d-82c9-3614834c3e8a"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 01:35:48.913638 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:48.913598 2580 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/91c3a3db-b9c8-445d-82c9-3614834c3e8a-local-certs\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 01:35:48.913638 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:48.913630 2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91c3a3db-b9c8-445d-82c9-3614834c3e8a-istio-token\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 01:35:48.913638 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:48.913640 2580 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/91c3a3db-b9c8-445d-82c9-3614834c3e8a-cacerts\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 01:35:48.913866 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:48.913649 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qvv44\" (UniqueName: \"kubernetes.io/projected/91c3a3db-b9c8-445d-82c9-3614834c3e8a-kube-api-access-qvv44\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 01:35:48.913866 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:48.913660 2580 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/91c3a3db-b9c8-445d-82c9-3614834c3e8a-istio-csr-ca-configmap\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 01:35:48.913866 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:48.913668 2580 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/91c3a3db-b9c8-445d-82c9-3614834c3e8a-istio-csr-dns-cert\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 01:35:48.913866 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:48.913676 2580 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/91c3a3db-b9c8-445d-82c9-3614834c3e8a-istio-kubeconfig\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 01:35:49.402960 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:49.402928 2580 generic.go:356] "Generic (PLEG): container finished" podID="91c3a3db-b9c8-445d-82c9-3614834c3e8a" containerID="9fc642bd36acf93f3baddaf4b7f96083567d717d59562dd24a98072b9277eebf" exitCode=0 Jul 06 01:35:49.403139 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:49.402992 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-t9sq7" Jul 06 01:35:49.403139 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:49.402994 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-t9sq7" event={"ID":"91c3a3db-b9c8-445d-82c9-3614834c3e8a","Type":"ContainerDied","Data":"9fc642bd36acf93f3baddaf4b7f96083567d717d59562dd24a98072b9277eebf"} Jul 06 01:35:49.403139 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:49.403109 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-t9sq7" event={"ID":"91c3a3db-b9c8-445d-82c9-3614834c3e8a","Type":"ContainerDied","Data":"02d7c45ef7d4d06978f07884a19b3b464519fcd8b9c6d3c92e014b5dd265ad2b"} Jul 06 01:35:49.403139 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:49.403129 2580 scope.go:117] "RemoveContainer" containerID="9fc642bd36acf93f3baddaf4b7f96083567d717d59562dd24a98072b9277eebf" Jul 06 01:35:49.411625 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:49.411600 2580 scope.go:117] "RemoveContainer" containerID="9fc642bd36acf93f3baddaf4b7f96083567d717d59562dd24a98072b9277eebf" Jul 06 01:35:49.411950 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:35:49.411922 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9fc642bd36acf93f3baddaf4b7f96083567d717d59562dd24a98072b9277eebf\": container with ID starting with 9fc642bd36acf93f3baddaf4b7f96083567d717d59562dd24a98072b9277eebf not found: ID does not exist" containerID="9fc642bd36acf93f3baddaf4b7f96083567d717d59562dd24a98072b9277eebf" Jul 06 01:35:49.412020 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:49.411964 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fc642bd36acf93f3baddaf4b7f96083567d717d59562dd24a98072b9277eebf"} err="failed to get container status \"9fc642bd36acf93f3baddaf4b7f96083567d717d59562dd24a98072b9277eebf\": rpc error: code = NotFound desc = could not find container \"9fc642bd36acf93f3baddaf4b7f96083567d717d59562dd24a98072b9277eebf\": container with ID starting with 9fc642bd36acf93f3baddaf4b7f96083567d717d59562dd24a98072b9277eebf not found: ID does not exist" Jul 06 01:35:49.427056 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:49.427023 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-t9sq7"] Jul 06 01:35:49.430492 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:49.430422 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-t9sq7"] Jul 06 01:35:50.359516 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:50.359481 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-894jd" Jul 06 01:35:50.458534 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:50.458497 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91c3a3db-b9c8-445d-82c9-3614834c3e8a" path="/var/lib/kubelet/pods/91c3a3db-b9c8-445d-82c9-3614834c3e8a/volumes" Jul 06 01:35:51.367707 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:51.367676 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-cdlg2" Jul 06 01:35:55.398789 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:55.398756 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:35:57.395903 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:35:57.395867 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-577bbf7cf9-xqvfn" Jul 06 01:36:33.889967 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:36:33.889929 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:36:33.989577 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:36:33.989540 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:36:34.090786 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:36:34.090744 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:36:40.712769 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:36:40.712716 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:36:41.844114 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:36:41.844070 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:36:42.561623 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:36:42.561584 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:36:42.614523 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:36:42.614486 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:36:49.175103 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:36:49.175066 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:36:49.225589 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:36:49.225556 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:36:59.984411 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:36:59.984371 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:37:00.082688 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:00.082653 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:37:00.161984 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:00.161947 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:37:01.356455 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:01.356400 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:37:04.802730 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:04.802695 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:37:06.769146 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:06.769115 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:37:08.405844 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:08.405797 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:37:08.588223 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:08.588131 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:37:08.745921 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:08.745887 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:37:11.960065 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:11.960030 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:37:12.284552 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:12.284456 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:37:15.203845 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:15.203800 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:37:15.401324 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:15.401285 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-lmx6s"] Jul 06 01:37:15.401619 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:15.401606 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91c3a3db-b9c8-445d-82c9-3614834c3e8a" containerName="discovery" Jul 06 01:37:15.401668 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:15.401621 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="91c3a3db-b9c8-445d-82c9-3614834c3e8a" containerName="discovery" Jul 06 01:37:15.401707 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:15.401674 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="91c3a3db-b9c8-445d-82c9-3614834c3e8a" containerName="discovery" Jul 06 01:37:15.404581 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:15.404561 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-lmx6s" Jul 06 01:37:15.408396 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:15.408354 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 06 01:37:15.408396 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:15.408392 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 06 01:37:15.410584 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:15.410468 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 06 01:37:15.410732 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:15.410718 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 06 01:37:15.410805 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:15.410742 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-cgz9t\"" Jul 06 01:37:15.417674 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:15.417651 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-lmx6s"] Jul 06 01:37:15.423276 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:15.423245 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:37:15.458721 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:15.458632 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zb4jq\" (UniqueName: \"kubernetes.io/projected/2ec65f6a-2cde-4d3e-832c-7123ff9dcdc7-kube-api-access-zb4jq\") pod \"kuadrant-console-plugin-66b5774985-lmx6s\" (UID: \"2ec65f6a-2cde-4d3e-832c-7123ff9dcdc7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-lmx6s" Jul 06 01:37:15.458721 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:15.458680 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2ec65f6a-2cde-4d3e-832c-7123ff9dcdc7-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-lmx6s\" (UID: \"2ec65f6a-2cde-4d3e-832c-7123ff9dcdc7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-lmx6s" Jul 06 01:37:15.458721 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:15.458700 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/2ec65f6a-2cde-4d3e-832c-7123ff9dcdc7-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-lmx6s\" (UID: \"2ec65f6a-2cde-4d3e-832c-7123ff9dcdc7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-lmx6s" Jul 06 01:37:15.559077 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:15.559038 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/2ec65f6a-2cde-4d3e-832c-7123ff9dcdc7-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-lmx6s\" (UID: \"2ec65f6a-2cde-4d3e-832c-7123ff9dcdc7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-lmx6s" Jul 06 01:37:15.559303 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:15.559133 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zb4jq\" (UniqueName: \"kubernetes.io/projected/2ec65f6a-2cde-4d3e-832c-7123ff9dcdc7-kube-api-access-zb4jq\") pod \"kuadrant-console-plugin-66b5774985-lmx6s\" (UID: \"2ec65f6a-2cde-4d3e-832c-7123ff9dcdc7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-lmx6s" Jul 06 01:37:15.559303 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:15.559162 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2ec65f6a-2cde-4d3e-832c-7123ff9dcdc7-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-lmx6s\" (UID: \"2ec65f6a-2cde-4d3e-832c-7123ff9dcdc7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-lmx6s" Jul 06 01:37:15.559303 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:37:15.559180 2580 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 06 01:37:15.559303 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:37:15.559262 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ec65f6a-2cde-4d3e-832c-7123ff9dcdc7-plugin-serving-cert podName:2ec65f6a-2cde-4d3e-832c-7123ff9dcdc7 nodeName:}" failed. No retries permitted until 2026-07-06 01:37:16.059245358 +0000 UTC m=+630.030497027 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/2ec65f6a-2cde-4d3e-832c-7123ff9dcdc7-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-lmx6s" (UID: "2ec65f6a-2cde-4d3e-832c-7123ff9dcdc7") : secret "plugin-serving-cert" not found Jul 06 01:37:15.559850 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:15.559831 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2ec65f6a-2cde-4d3e-832c-7123ff9dcdc7-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-lmx6s\" (UID: \"2ec65f6a-2cde-4d3e-832c-7123ff9dcdc7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-lmx6s" Jul 06 01:37:15.568945 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:15.568917 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zb4jq\" (UniqueName: \"kubernetes.io/projected/2ec65f6a-2cde-4d3e-832c-7123ff9dcdc7-kube-api-access-zb4jq\") pod \"kuadrant-console-plugin-66b5774985-lmx6s\" (UID: \"2ec65f6a-2cde-4d3e-832c-7123ff9dcdc7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-lmx6s" Jul 06 01:37:15.669992 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:15.669954 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:37:16.062281 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:16.062227 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/2ec65f6a-2cde-4d3e-832c-7123ff9dcdc7-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-lmx6s\" (UID: \"2ec65f6a-2cde-4d3e-832c-7123ff9dcdc7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-lmx6s" Jul 06 01:37:16.064756 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:16.064737 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/2ec65f6a-2cde-4d3e-832c-7123ff9dcdc7-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-lmx6s\" (UID: \"2ec65f6a-2cde-4d3e-832c-7123ff9dcdc7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-lmx6s" Jul 06 01:37:16.314666 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:16.314576 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-lmx6s" Jul 06 01:37:16.417832 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:16.417803 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:37:16.447876 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:16.447846 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-lmx6s"] Jul 06 01:37:16.450243 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:37:16.450211 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2ec65f6a_2cde_4d3e_832c_7123ff9dcdc7.slice/crio-c54506cc852f1be17fccfcdec6cfaf5e92418da24aa9746418912015256ec33a WatchSource:0}: Error finding container c54506cc852f1be17fccfcdec6cfaf5e92418da24aa9746418912015256ec33a: Status 404 returned error can't find the container with id c54506cc852f1be17fccfcdec6cfaf5e92418da24aa9746418912015256ec33a Jul 06 01:37:16.451535 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:16.451520 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 01:37:16.517360 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:16.517320 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:37:16.724217 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:16.724176 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-lmx6s" event={"ID":"2ec65f6a-2cde-4d3e-832c-7123ff9dcdc7","Type":"ContainerStarted","Data":"c54506cc852f1be17fccfcdec6cfaf5e92418da24aa9746418912015256ec33a"} Jul 06 01:37:19.053372 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:19.053334 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:37:19.153902 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:19.153690 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:37:24.175323 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:24.175290 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:37:24.276666 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:24.276627 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:37:29.992189 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:29.992144 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:37:30.097309 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:30.097252 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:37:32.971577 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:32.971525 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:37:33.074701 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:33.074669 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:37:36.777004 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:36.776963 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:37:36.834845 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:36.834798 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:37:38.816052 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:38.816012 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-lmx6s" event={"ID":"2ec65f6a-2cde-4d3e-832c-7123ff9dcdc7","Type":"ContainerStarted","Data":"743812820fb1f7c8c7c2c0321c6a2aab271481593e6d52b6683425e4a4b05b12"} Jul 06 01:37:38.837468 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:38.837394 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-lmx6s" podStartSLOduration=1.651193561 podStartE2EDuration="23.837375529s" totalImagesPullingTime="22.186181968s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:37:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:37:38.836138526 +0000 UTC m=+652.807390208" watchObservedRunningTime="2026-07-06 01:37:38.837375529 +0000 UTC m=+652.808627220" Jul 06 01:37:43.169615 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:43.169574 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:37:43.265896 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:43.265863 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:37:44.626422 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:44.626387 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:37:44.676793 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:44.676736 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:37:49.744239 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:49.744185 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:37:49.833489 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:49.833427 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:37:52.731153 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:52.731121 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:37:52.835147 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:52.835096 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:37:54.766797 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:54.766763 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:37:54.881835 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:54.881796 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:37:57.233889 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:57.233844 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:37:57.347683 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:57.347639 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:37:57.450840 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:57.450810 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:37:57.761533 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:57.761498 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:37:57.879302 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:57.879269 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:37:58.112808 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:58.112728 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-gkpcj"] Jul 06 01:37:58.156061 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:58.156005 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:37:58.190328 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:58.190286 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-gkpcj"] Jul 06 01:37:58.190523 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:58.190391 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-gkpcj" Jul 06 01:37:58.194209 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:58.194190 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-bxqxj\"" Jul 06 01:37:58.260097 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:58.260069 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:37:58.332508 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:58.332466 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fb58f\" (UniqueName: \"kubernetes.io/projected/45c420e6-0082-43fd-aa9d-7f2422f37558-kube-api-access-fb58f\") pod \"authorino-58cd955675-gkpcj\" (UID: \"45c420e6-0082-43fd-aa9d-7f2422f37558\") " pod="kuadrant-system/authorino-58cd955675-gkpcj" Jul 06 01:37:58.356668 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:58.356633 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-9mwdh"] Jul 06 01:37:58.359768 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:58.359752 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-9mwdh" Jul 06 01:37:58.367771 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:58.367746 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-9mwdh"] Jul 06 01:37:58.433358 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:58.433312 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fb58f\" (UniqueName: \"kubernetes.io/projected/45c420e6-0082-43fd-aa9d-7f2422f37558-kube-api-access-fb58f\") pod \"authorino-58cd955675-gkpcj\" (UID: \"45c420e6-0082-43fd-aa9d-7f2422f37558\") " pod="kuadrant-system/authorino-58cd955675-gkpcj" Jul 06 01:37:58.442246 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:58.442218 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fb58f\" (UniqueName: \"kubernetes.io/projected/45c420e6-0082-43fd-aa9d-7f2422f37558-kube-api-access-fb58f\") pod \"authorino-58cd955675-gkpcj\" (UID: \"45c420e6-0082-43fd-aa9d-7f2422f37558\") " pod="kuadrant-system/authorino-58cd955675-gkpcj" Jul 06 01:37:58.498720 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:58.498685 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-gkpcj" Jul 06 01:37:58.534261 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:58.534226 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fszsc\" (UniqueName: \"kubernetes.io/projected/695fbb80-458a-4cee-b912-7a52e2e69d79-kube-api-access-fszsc\") pod \"authorino-75d556bd77-9mwdh\" (UID: \"695fbb80-458a-4cee-b912-7a52e2e69d79\") " pod="kuadrant-system/authorino-75d556bd77-9mwdh" Jul 06 01:37:58.627551 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:58.627526 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-gkpcj"] Jul 06 01:37:58.631025 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:37:58.630994 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod45c420e6_0082_43fd_aa9d_7f2422f37558.slice/crio-18a2f20b7baffa3a97e57de86b1cedf3afb7b21056354fe4ee65946f666c75d0 WatchSource:0}: Error finding container 18a2f20b7baffa3a97e57de86b1cedf3afb7b21056354fe4ee65946f666c75d0: Status 404 returned error can't find the container with id 18a2f20b7baffa3a97e57de86b1cedf3afb7b21056354fe4ee65946f666c75d0 Jul 06 01:37:58.635421 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:58.635366 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fszsc\" (UniqueName: \"kubernetes.io/projected/695fbb80-458a-4cee-b912-7a52e2e69d79-kube-api-access-fszsc\") pod \"authorino-75d556bd77-9mwdh\" (UID: \"695fbb80-458a-4cee-b912-7a52e2e69d79\") " pod="kuadrant-system/authorino-75d556bd77-9mwdh" Jul 06 01:37:58.643280 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:58.643256 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fszsc\" (UniqueName: \"kubernetes.io/projected/695fbb80-458a-4cee-b912-7a52e2e69d79-kube-api-access-fszsc\") pod \"authorino-75d556bd77-9mwdh\" (UID: \"695fbb80-458a-4cee-b912-7a52e2e69d79\") " pod="kuadrant-system/authorino-75d556bd77-9mwdh" Jul 06 01:37:58.669244 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:58.669208 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-9mwdh" Jul 06 01:37:58.801608 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:58.801410 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-9mwdh"] Jul 06 01:37:58.804221 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:37:58.804183 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod695fbb80_458a_4cee_b912_7a52e2e69d79.slice/crio-4b330f1201a4097e5705575a84925ee048f1bf467c08930cabac2f20d6b27d29 WatchSource:0}: Error finding container 4b330f1201a4097e5705575a84925ee048f1bf467c08930cabac2f20d6b27d29: Status 404 returned error can't find the container with id 4b330f1201a4097e5705575a84925ee048f1bf467c08930cabac2f20d6b27d29 Jul 06 01:37:58.893510 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:58.893473 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-gkpcj" event={"ID":"45c420e6-0082-43fd-aa9d-7f2422f37558","Type":"ContainerStarted","Data":"18a2f20b7baffa3a97e57de86b1cedf3afb7b21056354fe4ee65946f666c75d0"} Jul 06 01:37:58.894548 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:37:58.894519 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-9mwdh" event={"ID":"695fbb80-458a-4cee-b912-7a52e2e69d79","Type":"ContainerStarted","Data":"4b330f1201a4097e5705575a84925ee048f1bf467c08930cabac2f20d6b27d29"} Jul 06 01:38:02.915928 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:02.915885 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-gkpcj" event={"ID":"45c420e6-0082-43fd-aa9d-7f2422f37558","Type":"ContainerStarted","Data":"7a07aeb5f75d10b5fe7e935e17863265d24d1d5d3a2e9fc7c1c62ab782dad824"} Jul 06 01:38:02.917427 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:02.917400 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-9mwdh" event={"ID":"695fbb80-458a-4cee-b912-7a52e2e69d79","Type":"ContainerStarted","Data":"183d636c687cfd527845c22690290bcf09b43e696fda0b067dff30eeb072cf7d"} Jul 06 01:38:02.930123 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:02.930068 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-gkpcj" podStartSLOduration=1.22799886 podStartE2EDuration="4.93005255s" totalImagesPullingTime="3.70205369s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:37:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:38:02.929086373 +0000 UTC m=+676.900338064" watchObservedRunningTime="2026-07-06 01:38:02.93005255 +0000 UTC m=+676.901304240" Jul 06 01:38:02.943010 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:02.942953 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-9mwdh" podStartSLOduration=1.425716547 podStartE2EDuration="4.942936258s" totalImagesPullingTime="3.517219711s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:37:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:38:02.942066332 +0000 UTC m=+676.913318028" watchObservedRunningTime="2026-07-06 01:38:02.942936258 +0000 UTC m=+676.914187949" Jul 06 01:38:02.966214 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:02.966183 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-gkpcj"] Jul 06 01:38:04.925941 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:04.925905 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-gkpcj" podUID="45c420e6-0082-43fd-aa9d-7f2422f37558" containerName="authorino" containerID="cri-o://7a07aeb5f75d10b5fe7e935e17863265d24d1d5d3a2e9fc7c1c62ab782dad824" gracePeriod=30 Jul 06 01:38:05.167113 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:05.167088 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-gkpcj" Jul 06 01:38:05.194196 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:05.194102 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fb58f\" (UniqueName: \"kubernetes.io/projected/45c420e6-0082-43fd-aa9d-7f2422f37558-kube-api-access-fb58f\") pod \"45c420e6-0082-43fd-aa9d-7f2422f37558\" (UID: \"45c420e6-0082-43fd-aa9d-7f2422f37558\") " Jul 06 01:38:05.196399 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:05.196363 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45c420e6-0082-43fd-aa9d-7f2422f37558-kube-api-access-fb58f" (OuterVolumeSpecName: "kube-api-access-fb58f") pod "45c420e6-0082-43fd-aa9d-7f2422f37558" (UID: "45c420e6-0082-43fd-aa9d-7f2422f37558"). InnerVolumeSpecName "kube-api-access-fb58f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 01:38:05.294903 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:05.294866 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fb58f\" (UniqueName: \"kubernetes.io/projected/45c420e6-0082-43fd-aa9d-7f2422f37558-kube-api-access-fb58f\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 01:38:05.931621 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:05.931582 2580 generic.go:356] "Generic (PLEG): container finished" podID="45c420e6-0082-43fd-aa9d-7f2422f37558" containerID="7a07aeb5f75d10b5fe7e935e17863265d24d1d5d3a2e9fc7c1c62ab782dad824" exitCode=0 Jul 06 01:38:05.932035 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:05.931629 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-gkpcj" event={"ID":"45c420e6-0082-43fd-aa9d-7f2422f37558","Type":"ContainerDied","Data":"7a07aeb5f75d10b5fe7e935e17863265d24d1d5d3a2e9fc7c1c62ab782dad824"} Jul 06 01:38:05.932035 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:05.931644 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-gkpcj" Jul 06 01:38:05.932035 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:05.931661 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-gkpcj" event={"ID":"45c420e6-0082-43fd-aa9d-7f2422f37558","Type":"ContainerDied","Data":"18a2f20b7baffa3a97e57de86b1cedf3afb7b21056354fe4ee65946f666c75d0"} Jul 06 01:38:05.932035 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:05.931677 2580 scope.go:117] "RemoveContainer" containerID="7a07aeb5f75d10b5fe7e935e17863265d24d1d5d3a2e9fc7c1c62ab782dad824" Jul 06 01:38:05.939892 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:05.939873 2580 scope.go:117] "RemoveContainer" containerID="7a07aeb5f75d10b5fe7e935e17863265d24d1d5d3a2e9fc7c1c62ab782dad824" Jul 06 01:38:05.940141 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:38:05.940123 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a07aeb5f75d10b5fe7e935e17863265d24d1d5d3a2e9fc7c1c62ab782dad824\": container with ID starting with 7a07aeb5f75d10b5fe7e935e17863265d24d1d5d3a2e9fc7c1c62ab782dad824 not found: ID does not exist" containerID="7a07aeb5f75d10b5fe7e935e17863265d24d1d5d3a2e9fc7c1c62ab782dad824" Jul 06 01:38:05.940187 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:05.940148 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a07aeb5f75d10b5fe7e935e17863265d24d1d5d3a2e9fc7c1c62ab782dad824"} err="failed to get container status \"7a07aeb5f75d10b5fe7e935e17863265d24d1d5d3a2e9fc7c1c62ab782dad824\": rpc error: code = NotFound desc = could not find container \"7a07aeb5f75d10b5fe7e935e17863265d24d1d5d3a2e9fc7c1c62ab782dad824\": container with ID starting with 7a07aeb5f75d10b5fe7e935e17863265d24d1d5d3a2e9fc7c1c62ab782dad824 not found: ID does not exist" Jul 06 01:38:05.943793 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:05.943760 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:38:05.950962 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:05.950941 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-gkpcj"] Jul 06 01:38:05.955252 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:05.955223 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-gkpcj"] Jul 06 01:38:05.998183 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:05.998149 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:38:06.459160 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:06.459125 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45c420e6-0082-43fd-aa9d-7f2422f37558" path="/var/lib/kubelet/pods/45c420e6-0082-43fd-aa9d-7f2422f37558/volumes" Jul 06 01:38:11.943569 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:11.943537 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:38:12.005066 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:12.005025 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:38:12.544775 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:12.544740 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:38:12.656499 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:12.656464 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:38:16.140055 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:16.140022 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:38:16.240470 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:16.240421 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:38:18.980774 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:18.980721 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:38:30.759103 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:30.759064 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:38:30.858941 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:30.858911 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:38:31.007107 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:31.007070 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-rcmfd"] Jul 06 01:38:31.007373 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:31.007359 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45c420e6-0082-43fd-aa9d-7f2422f37558" containerName="authorino" Jul 06 01:38:31.007418 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:31.007378 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="45c420e6-0082-43fd-aa9d-7f2422f37558" containerName="authorino" Jul 06 01:38:31.007478 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:31.007454 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="45c420e6-0082-43fd-aa9d-7f2422f37558" containerName="authorino" Jul 06 01:38:31.053207 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:31.053118 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-rcmfd"] Jul 06 01:38:31.053345 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:31.053218 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-rcmfd" Jul 06 01:38:31.084646 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:31.084606 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkk7s\" (UniqueName: \"kubernetes.io/projected/e01f0d3c-df21-461d-9348-4688b9337826-kube-api-access-hkk7s\") pod \"authorino-5d4b67c446-rcmfd\" (UID: \"e01f0d3c-df21-461d-9348-4688b9337826\") " pod="kuadrant-system/authorino-5d4b67c446-rcmfd" Jul 06 01:38:31.185136 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:31.185093 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkk7s\" (UniqueName: \"kubernetes.io/projected/e01f0d3c-df21-461d-9348-4688b9337826-kube-api-access-hkk7s\") pod \"authorino-5d4b67c446-rcmfd\" (UID: \"e01f0d3c-df21-461d-9348-4688b9337826\") " pod="kuadrant-system/authorino-5d4b67c446-rcmfd" Jul 06 01:38:31.192877 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:31.192850 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkk7s\" (UniqueName: \"kubernetes.io/projected/e01f0d3c-df21-461d-9348-4688b9337826-kube-api-access-hkk7s\") pod \"authorino-5d4b67c446-rcmfd\" (UID: \"e01f0d3c-df21-461d-9348-4688b9337826\") " pod="kuadrant-system/authorino-5d4b67c446-rcmfd" Jul 06 01:38:31.271354 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:31.271317 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-rcmfd"] Jul 06 01:38:31.271570 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:31.271558 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-rcmfd" Jul 06 01:38:31.297375 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:31.297340 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7c64d9ffb8-zcp5c"] Jul 06 01:38:31.301476 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:31.301431 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c64d9ffb8-zcp5c" Jul 06 01:38:31.306986 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:31.306912 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7c64d9ffb8-zcp5c"] Jul 06 01:38:31.387126 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:31.387078 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vswgj\" (UniqueName: \"kubernetes.io/projected/1755ebe7-acfd-45bb-b7bb-b66a5d04dc39-kube-api-access-vswgj\") pod \"authorino-7c64d9ffb8-zcp5c\" (UID: \"1755ebe7-acfd-45bb-b7bb-b66a5d04dc39\") " pod="kuadrant-system/authorino-7c64d9ffb8-zcp5c" Jul 06 01:38:31.409283 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:31.409248 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-rcmfd"] Jul 06 01:38:31.412306 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:38:31.412277 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode01f0d3c_df21_461d_9348_4688b9337826.slice/crio-d2edbce2e47a88a17cd5a067fae3dfa83b9e2e01055aa61adbd43bf80b50ae9d WatchSource:0}: Error finding container d2edbce2e47a88a17cd5a067fae3dfa83b9e2e01055aa61adbd43bf80b50ae9d: Status 404 returned error can't find the container with id d2edbce2e47a88a17cd5a067fae3dfa83b9e2e01055aa61adbd43bf80b50ae9d Jul 06 01:38:31.464009 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:31.463974 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7c64d9ffb8-zcp5c"] Jul 06 01:38:31.464236 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:38:31.464206 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-vswgj], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-7c64d9ffb8-zcp5c" podUID="1755ebe7-acfd-45bb-b7bb-b66a5d04dc39" Jul 06 01:38:31.487932 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:31.487894 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vswgj\" (UniqueName: \"kubernetes.io/projected/1755ebe7-acfd-45bb-b7bb-b66a5d04dc39-kube-api-access-vswgj\") pod \"authorino-7c64d9ffb8-zcp5c\" (UID: \"1755ebe7-acfd-45bb-b7bb-b66a5d04dc39\") " pod="kuadrant-system/authorino-7c64d9ffb8-zcp5c" Jul 06 01:38:31.491948 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:31.491922 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-67b87b8cfb-grk6m"] Jul 06 01:38:31.495386 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:31.495365 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-67b87b8cfb-grk6m" Jul 06 01:38:31.497696 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:31.497673 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 06 01:38:31.499519 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:31.499496 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vswgj\" (UniqueName: \"kubernetes.io/projected/1755ebe7-acfd-45bb-b7bb-b66a5d04dc39-kube-api-access-vswgj\") pod \"authorino-7c64d9ffb8-zcp5c\" (UID: \"1755ebe7-acfd-45bb-b7bb-b66a5d04dc39\") " pod="kuadrant-system/authorino-7c64d9ffb8-zcp5c" Jul 06 01:38:31.504753 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:31.504731 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-67b87b8cfb-grk6m"] Jul 06 01:38:31.589032 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:31.588940 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/937ea1ee-e382-4254-93bd-3e8babadd77f-tls-cert\") pod \"authorino-67b87b8cfb-grk6m\" (UID: \"937ea1ee-e382-4254-93bd-3e8babadd77f\") " pod="kuadrant-system/authorino-67b87b8cfb-grk6m" Jul 06 01:38:31.589032 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:31.589014 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsjdq\" (UniqueName: \"kubernetes.io/projected/937ea1ee-e382-4254-93bd-3e8babadd77f-kube-api-access-rsjdq\") pod \"authorino-67b87b8cfb-grk6m\" (UID: \"937ea1ee-e382-4254-93bd-3e8babadd77f\") " pod="kuadrant-system/authorino-67b87b8cfb-grk6m" Jul 06 01:38:31.690325 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:31.690280 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsjdq\" (UniqueName: \"kubernetes.io/projected/937ea1ee-e382-4254-93bd-3e8babadd77f-kube-api-access-rsjdq\") pod \"authorino-67b87b8cfb-grk6m\" (UID: \"937ea1ee-e382-4254-93bd-3e8babadd77f\") " pod="kuadrant-system/authorino-67b87b8cfb-grk6m" Jul 06 01:38:31.690495 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:31.690347 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/937ea1ee-e382-4254-93bd-3e8babadd77f-tls-cert\") pod \"authorino-67b87b8cfb-grk6m\" (UID: \"937ea1ee-e382-4254-93bd-3e8babadd77f\") " pod="kuadrant-system/authorino-67b87b8cfb-grk6m" Jul 06 01:38:31.692947 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:31.692926 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/937ea1ee-e382-4254-93bd-3e8babadd77f-tls-cert\") pod \"authorino-67b87b8cfb-grk6m\" (UID: \"937ea1ee-e382-4254-93bd-3e8babadd77f\") " pod="kuadrant-system/authorino-67b87b8cfb-grk6m" Jul 06 01:38:31.698491 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:31.698426 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsjdq\" (UniqueName: \"kubernetes.io/projected/937ea1ee-e382-4254-93bd-3e8babadd77f-kube-api-access-rsjdq\") pod \"authorino-67b87b8cfb-grk6m\" (UID: \"937ea1ee-e382-4254-93bd-3e8babadd77f\") " pod="kuadrant-system/authorino-67b87b8cfb-grk6m" Jul 06 01:38:31.813321 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:31.813289 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-67b87b8cfb-grk6m" Jul 06 01:38:31.961769 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:31.961742 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-67b87b8cfb-grk6m"] Jul 06 01:38:31.964206 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:38:31.964171 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod937ea1ee_e382_4254_93bd_3e8babadd77f.slice/crio-a6bd181d5ead0129e65ad06d78e0809f8f28fb30a0a072ad16aa93273ca628a7 WatchSource:0}: Error finding container a6bd181d5ead0129e65ad06d78e0809f8f28fb30a0a072ad16aa93273ca628a7: Status 404 returned error can't find the container with id a6bd181d5ead0129e65ad06d78e0809f8f28fb30a0a072ad16aa93273ca628a7 Jul 06 01:38:32.035951 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:32.035906 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-67b87b8cfb-grk6m" event={"ID":"937ea1ee-e382-4254-93bd-3e8babadd77f","Type":"ContainerStarted","Data":"a6bd181d5ead0129e65ad06d78e0809f8f28fb30a0a072ad16aa93273ca628a7"} Jul 06 01:38:32.037607 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:32.037578 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-rcmfd" event={"ID":"e01f0d3c-df21-461d-9348-4688b9337826","Type":"ContainerStarted","Data":"32690d6c46c244461ace2f1a7371495c784c0f82c78494c8d63c37dec3cbd841"} Jul 06 01:38:32.037749 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:32.037613 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-rcmfd" event={"ID":"e01f0d3c-df21-461d-9348-4688b9337826","Type":"ContainerStarted","Data":"d2edbce2e47a88a17cd5a067fae3dfa83b9e2e01055aa61adbd43bf80b50ae9d"} Jul 06 01:38:32.037749 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:32.037610 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-rcmfd" podUID="e01f0d3c-df21-461d-9348-4688b9337826" containerName="authorino" containerID="cri-o://32690d6c46c244461ace2f1a7371495c784c0f82c78494c8d63c37dec3cbd841" gracePeriod=30 Jul 06 01:38:32.037749 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:32.037652 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c64d9ffb8-zcp5c" Jul 06 01:38:32.047943 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:32.047914 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c64d9ffb8-zcp5c" Jul 06 01:38:32.055201 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:32.055154 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-rcmfd" podStartSLOduration=1.699150646 podStartE2EDuration="2.055138149s" totalImagesPullingTime="355.987503ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:38:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:38:32.054522009 +0000 UTC m=+706.025773690" watchObservedRunningTime="2026-07-06 01:38:32.055138149 +0000 UTC m=+706.026389844" Jul 06 01:38:32.093169 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:32.093133 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vswgj\" (UniqueName: \"kubernetes.io/projected/1755ebe7-acfd-45bb-b7bb-b66a5d04dc39-kube-api-access-vswgj\") pod \"1755ebe7-acfd-45bb-b7bb-b66a5d04dc39\" (UID: \"1755ebe7-acfd-45bb-b7bb-b66a5d04dc39\") " Jul 06 01:38:32.095389 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:32.095364 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1755ebe7-acfd-45bb-b7bb-b66a5d04dc39-kube-api-access-vswgj" (OuterVolumeSpecName: "kube-api-access-vswgj") pod "1755ebe7-acfd-45bb-b7bb-b66a5d04dc39" (UID: "1755ebe7-acfd-45bb-b7bb-b66a5d04dc39"). InnerVolumeSpecName "kube-api-access-vswgj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 01:38:32.194814 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:32.194778 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vswgj\" (UniqueName: \"kubernetes.io/projected/1755ebe7-acfd-45bb-b7bb-b66a5d04dc39-kube-api-access-vswgj\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 01:38:32.308347 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:32.308311 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-rcmfd" Jul 06 01:38:32.395619 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:32.395581 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hkk7s\" (UniqueName: \"kubernetes.io/projected/e01f0d3c-df21-461d-9348-4688b9337826-kube-api-access-hkk7s\") pod \"e01f0d3c-df21-461d-9348-4688b9337826\" (UID: \"e01f0d3c-df21-461d-9348-4688b9337826\") " Jul 06 01:38:32.397707 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:32.397673 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e01f0d3c-df21-461d-9348-4688b9337826-kube-api-access-hkk7s" (OuterVolumeSpecName: "kube-api-access-hkk7s") pod "e01f0d3c-df21-461d-9348-4688b9337826" (UID: "e01f0d3c-df21-461d-9348-4688b9337826"). InnerVolumeSpecName "kube-api-access-hkk7s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 01:38:32.496625 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:32.496594 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hkk7s\" (UniqueName: \"kubernetes.io/projected/e01f0d3c-df21-461d-9348-4688b9337826-kube-api-access-hkk7s\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 01:38:33.043895 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:33.043856 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-67b87b8cfb-grk6m" event={"ID":"937ea1ee-e382-4254-93bd-3e8babadd77f","Type":"ContainerStarted","Data":"7df2d8f3f39fb128617c73115829dfb4a43feb3b6a026df4e91fbac960064e70"} Jul 06 01:38:33.045082 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:33.045057 2580 generic.go:356] "Generic (PLEG): container finished" podID="e01f0d3c-df21-461d-9348-4688b9337826" containerID="32690d6c46c244461ace2f1a7371495c784c0f82c78494c8d63c37dec3cbd841" exitCode=0 Jul 06 01:38:33.045203 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:33.045155 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c64d9ffb8-zcp5c" Jul 06 01:38:33.045203 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:33.045159 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-rcmfd" Jul 06 01:38:33.045203 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:33.045193 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-rcmfd" event={"ID":"e01f0d3c-df21-461d-9348-4688b9337826","Type":"ContainerDied","Data":"32690d6c46c244461ace2f1a7371495c784c0f82c78494c8d63c37dec3cbd841"} Jul 06 01:38:33.045337 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:33.045223 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-rcmfd" event={"ID":"e01f0d3c-df21-461d-9348-4688b9337826","Type":"ContainerDied","Data":"d2edbce2e47a88a17cd5a067fae3dfa83b9e2e01055aa61adbd43bf80b50ae9d"} Jul 06 01:38:33.045337 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:33.045240 2580 scope.go:117] "RemoveContainer" containerID="32690d6c46c244461ace2f1a7371495c784c0f82c78494c8d63c37dec3cbd841" Jul 06 01:38:33.053241 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:33.053214 2580 scope.go:117] "RemoveContainer" containerID="32690d6c46c244461ace2f1a7371495c784c0f82c78494c8d63c37dec3cbd841" Jul 06 01:38:33.053560 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:38:33.053533 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32690d6c46c244461ace2f1a7371495c784c0f82c78494c8d63c37dec3cbd841\": container with ID starting with 32690d6c46c244461ace2f1a7371495c784c0f82c78494c8d63c37dec3cbd841 not found: ID does not exist" containerID="32690d6c46c244461ace2f1a7371495c784c0f82c78494c8d63c37dec3cbd841" Jul 06 01:38:33.053647 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:33.053575 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32690d6c46c244461ace2f1a7371495c784c0f82c78494c8d63c37dec3cbd841"} err="failed to get container status \"32690d6c46c244461ace2f1a7371495c784c0f82c78494c8d63c37dec3cbd841\": rpc error: code = NotFound desc = could not find container \"32690d6c46c244461ace2f1a7371495c784c0f82c78494c8d63c37dec3cbd841\": container with ID starting with 32690d6c46c244461ace2f1a7371495c784c0f82c78494c8d63c37dec3cbd841 not found: ID does not exist" Jul 06 01:38:33.058646 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:33.058593 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-67b87b8cfb-grk6m" podStartSLOduration=1.542928686 podStartE2EDuration="2.058577255s" totalImagesPullingTime="515.648569ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:38:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:38:33.058266026 +0000 UTC m=+707.029517719" watchObservedRunningTime="2026-07-06 01:38:33.058577255 +0000 UTC m=+707.029829050" Jul 06 01:38:33.083116 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:33.083086 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-9mwdh"] Jul 06 01:38:33.083291 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:33.083271 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-9mwdh" podUID="695fbb80-458a-4cee-b912-7a52e2e69d79" containerName="authorino" containerID="cri-o://183d636c687cfd527845c22690290bcf09b43e696fda0b067dff30eeb072cf7d" gracePeriod=30 Jul 06 01:38:33.088371 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:33.088339 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7c64d9ffb8-zcp5c"] Jul 06 01:38:33.095262 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:33.095225 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7c64d9ffb8-zcp5c"] Jul 06 01:38:33.106291 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:33.106255 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-rcmfd"] Jul 06 01:38:33.111275 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:33.111249 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-rcmfd"] Jul 06 01:38:33.336397 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:33.336371 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-9mwdh" Jul 06 01:38:33.404708 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:33.404678 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fszsc\" (UniqueName: \"kubernetes.io/projected/695fbb80-458a-4cee-b912-7a52e2e69d79-kube-api-access-fszsc\") pod \"695fbb80-458a-4cee-b912-7a52e2e69d79\" (UID: \"695fbb80-458a-4cee-b912-7a52e2e69d79\") " Jul 06 01:38:33.407073 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:33.407049 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/695fbb80-458a-4cee-b912-7a52e2e69d79-kube-api-access-fszsc" (OuterVolumeSpecName: "kube-api-access-fszsc") pod "695fbb80-458a-4cee-b912-7a52e2e69d79" (UID: "695fbb80-458a-4cee-b912-7a52e2e69d79"). InnerVolumeSpecName "kube-api-access-fszsc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 01:38:33.505733 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:33.505687 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fszsc\" (UniqueName: \"kubernetes.io/projected/695fbb80-458a-4cee-b912-7a52e2e69d79-kube-api-access-fszsc\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 01:38:34.051123 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:34.051080 2580 generic.go:356] "Generic (PLEG): container finished" podID="695fbb80-458a-4cee-b912-7a52e2e69d79" containerID="183d636c687cfd527845c22690290bcf09b43e696fda0b067dff30eeb072cf7d" exitCode=0 Jul 06 01:38:34.051123 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:34.051124 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-9mwdh" Jul 06 01:38:34.051651 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:34.051142 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-9mwdh" event={"ID":"695fbb80-458a-4cee-b912-7a52e2e69d79","Type":"ContainerDied","Data":"183d636c687cfd527845c22690290bcf09b43e696fda0b067dff30eeb072cf7d"} Jul 06 01:38:34.051651 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:34.051188 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-9mwdh" event={"ID":"695fbb80-458a-4cee-b912-7a52e2e69d79","Type":"ContainerDied","Data":"4b330f1201a4097e5705575a84925ee048f1bf467c08930cabac2f20d6b27d29"} Jul 06 01:38:34.051651 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:34.051211 2580 scope.go:117] "RemoveContainer" containerID="183d636c687cfd527845c22690290bcf09b43e696fda0b067dff30eeb072cf7d" Jul 06 01:38:34.059731 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:34.059711 2580 scope.go:117] "RemoveContainer" containerID="183d636c687cfd527845c22690290bcf09b43e696fda0b067dff30eeb072cf7d" Jul 06 01:38:34.060031 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:38:34.060010 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"183d636c687cfd527845c22690290bcf09b43e696fda0b067dff30eeb072cf7d\": container with ID starting with 183d636c687cfd527845c22690290bcf09b43e696fda0b067dff30eeb072cf7d not found: ID does not exist" containerID="183d636c687cfd527845c22690290bcf09b43e696fda0b067dff30eeb072cf7d" Jul 06 01:38:34.060092 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:34.060038 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"183d636c687cfd527845c22690290bcf09b43e696fda0b067dff30eeb072cf7d"} err="failed to get container status \"183d636c687cfd527845c22690290bcf09b43e696fda0b067dff30eeb072cf7d\": rpc error: code = NotFound desc = could not find container \"183d636c687cfd527845c22690290bcf09b43e696fda0b067dff30eeb072cf7d\": container with ID starting with 183d636c687cfd527845c22690290bcf09b43e696fda0b067dff30eeb072cf7d not found: ID does not exist" Jul 06 01:38:34.071138 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:34.071080 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-9mwdh"] Jul 06 01:38:34.072578 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:34.072555 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-9mwdh"] Jul 06 01:38:34.458949 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:34.458915 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1755ebe7-acfd-45bb-b7bb-b66a5d04dc39" path="/var/lib/kubelet/pods/1755ebe7-acfd-45bb-b7bb-b66a5d04dc39/volumes" Jul 06 01:38:34.459141 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:34.459131 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="695fbb80-458a-4cee-b912-7a52e2e69d79" path="/var/lib/kubelet/pods/695fbb80-458a-4cee-b912-7a52e2e69d79/volumes" Jul 06 01:38:34.459401 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:34.459388 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e01f0d3c-df21-461d-9348-4688b9337826" path="/var/lib/kubelet/pods/e01f0d3c-df21-461d-9348-4688b9337826/volumes" Jul 06 01:38:38.157548 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:38.157455 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:38:38.255123 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:38.255090 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:38:38.345129 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:38.345098 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:38:38.533549 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:38.533457 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:38:41.053487 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:41.053420 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:38:41.140950 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:38:41.140918 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:39:17.511669 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:39:17.511636 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:39:18.011248 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:39:18.011215 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:39:18.322899 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:39:18.322802 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:39:18.558842 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:39:18.558813 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:39:18.661052 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:39:18.661017 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:39:28.719055 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:39:28.719012 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:39:28.813639 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:39:28.813602 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:39:40.864742 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:39:40.864707 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:39:48.032157 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:39:48.032121 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:39:50.155457 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:39:50.155398 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:39:53.184937 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:39:53.184898 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:39:53.246507 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:39:53.246466 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:39:57.715954 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:39:57.715916 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-nzfjx"] Jul 06 01:39:57.716494 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:39:57.716319 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="695fbb80-458a-4cee-b912-7a52e2e69d79" containerName="authorino" Jul 06 01:39:57.716494 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:39:57.716339 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="695fbb80-458a-4cee-b912-7a52e2e69d79" containerName="authorino" Jul 06 01:39:57.716494 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:39:57.716352 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e01f0d3c-df21-461d-9348-4688b9337826" containerName="authorino" Jul 06 01:39:57.716494 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:39:57.716360 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e01f0d3c-df21-461d-9348-4688b9337826" containerName="authorino" Jul 06 01:39:57.716494 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:39:57.716476 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e01f0d3c-df21-461d-9348-4688b9337826" containerName="authorino" Jul 06 01:39:57.716494 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:39:57.716491 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="695fbb80-458a-4cee-b912-7a52e2e69d79" containerName="authorino" Jul 06 01:39:57.719523 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:39:57.719501 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nzfjx" Jul 06 01:39:57.722249 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:39:57.722228 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 06 01:39:57.722356 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:39:57.722254 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 06 01:39:57.723282 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:39:57.723266 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-cxdt6\"" Jul 06 01:39:57.723333 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:39:57.723265 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 06 01:39:57.728785 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:39:57.728762 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-nzfjx"] Jul 06 01:39:57.768947 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:39:57.768907 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4d52f99a-240b-44c3-9ac5-852ed9b8f175-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nzfjx\" (UID: \"4d52f99a-240b-44c3-9ac5-852ed9b8f175\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nzfjx" Jul 06 01:39:57.769216 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:39:57.769182 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4d52f99a-240b-44c3-9ac5-852ed9b8f175-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nzfjx\" (UID: \"4d52f99a-240b-44c3-9ac5-852ed9b8f175\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nzfjx" Jul 06 01:39:57.769422 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:39:57.769401 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d52f99a-240b-44c3-9ac5-852ed9b8f175-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nzfjx\" (UID: \"4d52f99a-240b-44c3-9ac5-852ed9b8f175\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nzfjx" Jul 06 01:39:57.769661 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:39:57.769635 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d52f99a-240b-44c3-9ac5-852ed9b8f175-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nzfjx\" (UID: \"4d52f99a-240b-44c3-9ac5-852ed9b8f175\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nzfjx" Jul 06 01:39:57.769782 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:39:57.769673 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gknsn\" (UniqueName: \"kubernetes.io/projected/4d52f99a-240b-44c3-9ac5-852ed9b8f175-kube-api-access-gknsn\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nzfjx\" (UID: \"4d52f99a-240b-44c3-9ac5-852ed9b8f175\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nzfjx" Jul 06 01:39:57.769782 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:39:57.769738 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4d52f99a-240b-44c3-9ac5-852ed9b8f175-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nzfjx\" (UID: \"4d52f99a-240b-44c3-9ac5-852ed9b8f175\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nzfjx" Jul 06 01:39:57.809525 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:39:57.809493 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:39:57.870921 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:39:57.870885 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4d52f99a-240b-44c3-9ac5-852ed9b8f175-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nzfjx\" (UID: \"4d52f99a-240b-44c3-9ac5-852ed9b8f175\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nzfjx" Jul 06 01:39:57.871125 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:39:57.870942 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4d52f99a-240b-44c3-9ac5-852ed9b8f175-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nzfjx\" (UID: \"4d52f99a-240b-44c3-9ac5-852ed9b8f175\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nzfjx" Jul 06 01:39:57.871125 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:39:57.870958 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4d52f99a-240b-44c3-9ac5-852ed9b8f175-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nzfjx\" (UID: \"4d52f99a-240b-44c3-9ac5-852ed9b8f175\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nzfjx" Jul 06 01:39:57.871125 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:39:57.870980 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d52f99a-240b-44c3-9ac5-852ed9b8f175-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nzfjx\" (UID: \"4d52f99a-240b-44c3-9ac5-852ed9b8f175\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nzfjx" Jul 06 01:39:57.871125 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:39:57.871116 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d52f99a-240b-44c3-9ac5-852ed9b8f175-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nzfjx\" (UID: \"4d52f99a-240b-44c3-9ac5-852ed9b8f175\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nzfjx" Jul 06 01:39:57.871336 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:39:57.871153 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gknsn\" (UniqueName: \"kubernetes.io/projected/4d52f99a-240b-44c3-9ac5-852ed9b8f175-kube-api-access-gknsn\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nzfjx\" (UID: \"4d52f99a-240b-44c3-9ac5-852ed9b8f175\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nzfjx" Jul 06 01:39:57.871336 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:39:57.871326 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d52f99a-240b-44c3-9ac5-852ed9b8f175-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nzfjx\" (UID: \"4d52f99a-240b-44c3-9ac5-852ed9b8f175\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nzfjx" Jul 06 01:39:57.871474 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:39:57.871403 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4d52f99a-240b-44c3-9ac5-852ed9b8f175-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nzfjx\" (UID: \"4d52f99a-240b-44c3-9ac5-852ed9b8f175\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nzfjx" Jul 06 01:39:57.871474 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:39:57.871429 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4d52f99a-240b-44c3-9ac5-852ed9b8f175-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nzfjx\" (UID: \"4d52f99a-240b-44c3-9ac5-852ed9b8f175\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nzfjx" Jul 06 01:39:57.873322 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:39:57.873302 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4d52f99a-240b-44c3-9ac5-852ed9b8f175-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nzfjx\" (UID: \"4d52f99a-240b-44c3-9ac5-852ed9b8f175\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nzfjx" Jul 06 01:39:57.873591 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:39:57.873576 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d52f99a-240b-44c3-9ac5-852ed9b8f175-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nzfjx\" (UID: \"4d52f99a-240b-44c3-9ac5-852ed9b8f175\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nzfjx" Jul 06 01:39:57.878665 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:39:57.878642 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gknsn\" (UniqueName: \"kubernetes.io/projected/4d52f99a-240b-44c3-9ac5-852ed9b8f175-kube-api-access-gknsn\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nzfjx\" (UID: \"4d52f99a-240b-44c3-9ac5-852ed9b8f175\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nzfjx" Jul 06 01:39:58.029742 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:39:58.029658 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nzfjx" Jul 06 01:39:58.166482 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:39:58.166429 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-nzfjx"] Jul 06 01:39:58.169514 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:39:58.169485 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d52f99a_240b_44c3_9ac5_852ed9b8f175.slice/crio-ecb6b69fe72a8abd375b66e30c2acd882d974037bf3ea276184b3d930e37f121 WatchSource:0}: Error finding container ecb6b69fe72a8abd375b66e30c2acd882d974037bf3ea276184b3d930e37f121: Status 404 returned error can't find the container with id ecb6b69fe72a8abd375b66e30c2acd882d974037bf3ea276184b3d930e37f121 Jul 06 01:39:58.374338 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:39:58.374209 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nzfjx" event={"ID":"4d52f99a-240b-44c3-9ac5-852ed9b8f175","Type":"ContainerStarted","Data":"ecb6b69fe72a8abd375b66e30c2acd882d974037bf3ea276184b3d930e37f121"} Jul 06 01:39:59.381237 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:39:59.381200 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nzfjx" event={"ID":"4d52f99a-240b-44c3-9ac5-852ed9b8f175","Type":"ContainerStarted","Data":"41dc0c97f0c3eafcdac8014fe5cb5c067088e928470557e259ba4ead38db40ff"} Jul 06 01:39:59.397845 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:39:59.397794 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nzfjx" podStartSLOduration=1.371209466 podStartE2EDuration="2.397772328s" totalImagesPullingTime="1.026562862s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:39:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:39:59.397050378 +0000 UTC m=+793.368302069" watchObservedRunningTime="2026-07-06 01:39:59.397772328 +0000 UTC m=+793.369024019" Jul 06 01:40:04.516930 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:04.516896 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdk2pp"] Jul 06 01:40:04.520798 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:04.520772 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdk2pp" Jul 06 01:40:04.524506 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:04.524482 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 06 01:40:04.529050 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:04.529024 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdk2pp"] Jul 06 01:40:04.551730 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:04.551697 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:40:04.627006 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:04.626973 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/459003f0-b512-48db-9f0f-b7910dcded53-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdk2pp\" (UID: \"459003f0-b512-48db-9f0f-b7910dcded53\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdk2pp" Jul 06 01:40:04.627172 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:04.627012 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/459003f0-b512-48db-9f0f-b7910dcded53-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdk2pp\" (UID: \"459003f0-b512-48db-9f0f-b7910dcded53\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdk2pp" Jul 06 01:40:04.627172 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:04.627045 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ghff\" (UniqueName: \"kubernetes.io/projected/459003f0-b512-48db-9f0f-b7910dcded53-kube-api-access-4ghff\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdk2pp\" (UID: \"459003f0-b512-48db-9f0f-b7910dcded53\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdk2pp" Jul 06 01:40:04.627172 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:04.627131 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/459003f0-b512-48db-9f0f-b7910dcded53-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdk2pp\" (UID: \"459003f0-b512-48db-9f0f-b7910dcded53\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdk2pp" Jul 06 01:40:04.627299 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:04.627185 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/459003f0-b512-48db-9f0f-b7910dcded53-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdk2pp\" (UID: \"459003f0-b512-48db-9f0f-b7910dcded53\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdk2pp" Jul 06 01:40:04.627299 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:04.627205 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/459003f0-b512-48db-9f0f-b7910dcded53-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdk2pp\" (UID: \"459003f0-b512-48db-9f0f-b7910dcded53\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdk2pp" Jul 06 01:40:04.728598 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:04.728553 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/459003f0-b512-48db-9f0f-b7910dcded53-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdk2pp\" (UID: \"459003f0-b512-48db-9f0f-b7910dcded53\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdk2pp" Jul 06 01:40:04.728778 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:04.728607 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/459003f0-b512-48db-9f0f-b7910dcded53-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdk2pp\" (UID: \"459003f0-b512-48db-9f0f-b7910dcded53\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdk2pp" Jul 06 01:40:04.728778 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:04.728696 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/459003f0-b512-48db-9f0f-b7910dcded53-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdk2pp\" (UID: \"459003f0-b512-48db-9f0f-b7910dcded53\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdk2pp" Jul 06 01:40:04.728778 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:04.728772 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/459003f0-b512-48db-9f0f-b7910dcded53-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdk2pp\" (UID: \"459003f0-b512-48db-9f0f-b7910dcded53\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdk2pp" Jul 06 01:40:04.728926 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:04.728792 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/459003f0-b512-48db-9f0f-b7910dcded53-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdk2pp\" (UID: \"459003f0-b512-48db-9f0f-b7910dcded53\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdk2pp" Jul 06 01:40:04.728926 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:04.728816 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ghff\" (UniqueName: \"kubernetes.io/projected/459003f0-b512-48db-9f0f-b7910dcded53-kube-api-access-4ghff\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdk2pp\" (UID: \"459003f0-b512-48db-9f0f-b7910dcded53\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdk2pp" Jul 06 01:40:04.729058 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:04.729035 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/459003f0-b512-48db-9f0f-b7910dcded53-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdk2pp\" (UID: \"459003f0-b512-48db-9f0f-b7910dcded53\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdk2pp" Jul 06 01:40:04.729115 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:04.729056 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/459003f0-b512-48db-9f0f-b7910dcded53-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdk2pp\" (UID: \"459003f0-b512-48db-9f0f-b7910dcded53\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdk2pp" Jul 06 01:40:04.729198 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:04.729178 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/459003f0-b512-48db-9f0f-b7910dcded53-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdk2pp\" (UID: \"459003f0-b512-48db-9f0f-b7910dcded53\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdk2pp" Jul 06 01:40:04.731182 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:04.731155 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/459003f0-b512-48db-9f0f-b7910dcded53-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdk2pp\" (UID: \"459003f0-b512-48db-9f0f-b7910dcded53\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdk2pp" Jul 06 01:40:04.731298 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:04.731214 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/459003f0-b512-48db-9f0f-b7910dcded53-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdk2pp\" (UID: \"459003f0-b512-48db-9f0f-b7910dcded53\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdk2pp" Jul 06 01:40:04.737037 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:04.737013 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ghff\" (UniqueName: \"kubernetes.io/projected/459003f0-b512-48db-9f0f-b7910dcded53-kube-api-access-4ghff\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdk2pp\" (UID: \"459003f0-b512-48db-9f0f-b7910dcded53\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdk2pp" Jul 06 01:40:04.833427 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:04.833329 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdk2pp" Jul 06 01:40:04.968593 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:04.968402 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdk2pp"] Jul 06 01:40:04.971305 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:40:04.971277 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod459003f0_b512_48db_9f0f_b7910dcded53.slice/crio-62576bbf20ac83c73a517f39e8a980c88b6fe3bf3588c24b11d25df58e2b729c WatchSource:0}: Error finding container 62576bbf20ac83c73a517f39e8a980c88b6fe3bf3588c24b11d25df58e2b729c: Status 404 returned error can't find the container with id 62576bbf20ac83c73a517f39e8a980c88b6fe3bf3588c24b11d25df58e2b729c Jul 06 01:40:05.407311 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:05.407216 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdk2pp" event={"ID":"459003f0-b512-48db-9f0f-b7910dcded53","Type":"ContainerStarted","Data":"943f4444634b2961c5efef9833b27f169f9b68036ee0d85d1c536377026398e3"} Jul 06 01:40:05.407502 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:05.407320 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdk2pp" event={"ID":"459003f0-b512-48db-9f0f-b7910dcded53","Type":"ContainerStarted","Data":"62576bbf20ac83c73a517f39e8a980c88b6fe3bf3588c24b11d25df58e2b729c"} Jul 06 01:40:05.427517 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:05.427464 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdk2pp" podStartSLOduration=1.259972669 podStartE2EDuration="1.427428763s" totalImagesPullingTime="167.456094ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:40:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:40:05.426612717 +0000 UTC m=+799.397864408" watchObservedRunningTime="2026-07-06 01:40:05.427428763 +0000 UTC m=+799.398680454" Jul 06 01:40:08.030203 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:08.030105 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nzfjx" Jul 06 01:40:08.030203 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:08.030179 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nzfjx" Jul 06 01:40:08.042579 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:08.042548 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nzfjx" Jul 06 01:40:08.431552 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:08.431522 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nzfjx" Jul 06 01:40:10.623750 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:10.623706 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:40:14.834267 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:14.834211 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdk2pp" Jul 06 01:40:14.834723 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:14.834299 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdk2pp" Jul 06 01:40:14.846825 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:14.846799 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdk2pp" Jul 06 01:40:15.461834 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:15.461799 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdk2pp" Jul 06 01:40:16.713668 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:16.713631 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:40:24.517532 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:24.517498 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nhrl7"] Jul 06 01:40:24.520907 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:24.520886 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nhrl7" Jul 06 01:40:24.523798 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:24.523778 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 06 01:40:24.532641 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:24.532619 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nhrl7"] Jul 06 01:40:24.551308 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:24.551258 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:40:24.709526 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:24.709493 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6379a3b1-edbd-40bc-8ad6-0df732308f6a-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nhrl7\" (UID: \"6379a3b1-edbd-40bc-8ad6-0df732308f6a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nhrl7" Jul 06 01:40:24.709712 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:24.709555 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6379a3b1-edbd-40bc-8ad6-0df732308f6a-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nhrl7\" (UID: \"6379a3b1-edbd-40bc-8ad6-0df732308f6a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nhrl7" Jul 06 01:40:24.709712 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:24.709686 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6379a3b1-edbd-40bc-8ad6-0df732308f6a-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nhrl7\" (UID: \"6379a3b1-edbd-40bc-8ad6-0df732308f6a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nhrl7" Jul 06 01:40:24.709805 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:24.709729 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6379a3b1-edbd-40bc-8ad6-0df732308f6a-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nhrl7\" (UID: \"6379a3b1-edbd-40bc-8ad6-0df732308f6a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nhrl7" Jul 06 01:40:24.709805 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:24.709754 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6379a3b1-edbd-40bc-8ad6-0df732308f6a-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nhrl7\" (UID: \"6379a3b1-edbd-40bc-8ad6-0df732308f6a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nhrl7" Jul 06 01:40:24.709805 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:24.709793 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxj8n\" (UniqueName: \"kubernetes.io/projected/6379a3b1-edbd-40bc-8ad6-0df732308f6a-kube-api-access-rxj8n\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nhrl7\" (UID: \"6379a3b1-edbd-40bc-8ad6-0df732308f6a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nhrl7" Jul 06 01:40:24.810602 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:24.810514 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6379a3b1-edbd-40bc-8ad6-0df732308f6a-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nhrl7\" (UID: \"6379a3b1-edbd-40bc-8ad6-0df732308f6a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nhrl7" Jul 06 01:40:24.810602 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:24.810578 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6379a3b1-edbd-40bc-8ad6-0df732308f6a-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nhrl7\" (UID: \"6379a3b1-edbd-40bc-8ad6-0df732308f6a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nhrl7" Jul 06 01:40:24.810795 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:24.810624 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6379a3b1-edbd-40bc-8ad6-0df732308f6a-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nhrl7\" (UID: \"6379a3b1-edbd-40bc-8ad6-0df732308f6a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nhrl7" Jul 06 01:40:24.810795 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:24.810654 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6379a3b1-edbd-40bc-8ad6-0df732308f6a-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nhrl7\" (UID: \"6379a3b1-edbd-40bc-8ad6-0df732308f6a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nhrl7" Jul 06 01:40:24.810795 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:24.810682 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6379a3b1-edbd-40bc-8ad6-0df732308f6a-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nhrl7\" (UID: \"6379a3b1-edbd-40bc-8ad6-0df732308f6a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nhrl7" Jul 06 01:40:24.810795 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:24.810729 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxj8n\" (UniqueName: \"kubernetes.io/projected/6379a3b1-edbd-40bc-8ad6-0df732308f6a-kube-api-access-rxj8n\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nhrl7\" (UID: \"6379a3b1-edbd-40bc-8ad6-0df732308f6a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nhrl7" Jul 06 01:40:24.811029 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:24.811005 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6379a3b1-edbd-40bc-8ad6-0df732308f6a-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nhrl7\" (UID: \"6379a3b1-edbd-40bc-8ad6-0df732308f6a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nhrl7" Jul 06 01:40:24.811112 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:24.811057 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6379a3b1-edbd-40bc-8ad6-0df732308f6a-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nhrl7\" (UID: \"6379a3b1-edbd-40bc-8ad6-0df732308f6a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nhrl7" Jul 06 01:40:24.811191 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:24.811173 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6379a3b1-edbd-40bc-8ad6-0df732308f6a-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nhrl7\" (UID: \"6379a3b1-edbd-40bc-8ad6-0df732308f6a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nhrl7" Jul 06 01:40:24.813060 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:24.813040 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6379a3b1-edbd-40bc-8ad6-0df732308f6a-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nhrl7\" (UID: \"6379a3b1-edbd-40bc-8ad6-0df732308f6a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nhrl7" Jul 06 01:40:24.813431 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:24.813407 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6379a3b1-edbd-40bc-8ad6-0df732308f6a-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nhrl7\" (UID: \"6379a3b1-edbd-40bc-8ad6-0df732308f6a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nhrl7" Jul 06 01:40:24.821200 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:24.821171 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxj8n\" (UniqueName: \"kubernetes.io/projected/6379a3b1-edbd-40bc-8ad6-0df732308f6a-kube-api-access-rxj8n\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nhrl7\" (UID: \"6379a3b1-edbd-40bc-8ad6-0df732308f6a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nhrl7" Jul 06 01:40:24.830126 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:24.830104 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nhrl7" Jul 06 01:40:24.967012 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:24.966976 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nhrl7"] Jul 06 01:40:24.969709 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:40:24.969681 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6379a3b1_edbd_40bc_8ad6_0df732308f6a.slice/crio-b4d49348f9ee879d521ee6a580ad631d1d5e926db30900ca19ee5eef949a2bdc WatchSource:0}: Error finding container b4d49348f9ee879d521ee6a580ad631d1d5e926db30900ca19ee5eef949a2bdc: Status 404 returned error can't find the container with id b4d49348f9ee879d521ee6a580ad631d1d5e926db30900ca19ee5eef949a2bdc Jul 06 01:40:25.497756 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:25.497717 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nhrl7" event={"ID":"6379a3b1-edbd-40bc-8ad6-0df732308f6a","Type":"ContainerStarted","Data":"ea64cdfce20259e275f3842f56d388cf7b3c1bd025e89ecb48b32d37b62923d7"} Jul 06 01:40:25.497756 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:25.497759 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nhrl7" event={"ID":"6379a3b1-edbd-40bc-8ad6-0df732308f6a","Type":"ContainerStarted","Data":"b4d49348f9ee879d521ee6a580ad631d1d5e926db30900ca19ee5eef949a2bdc"} Jul 06 01:40:25.514268 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:25.514207 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nhrl7" podStartSLOduration=1.3507811379999999 podStartE2EDuration="1.514190466s" totalImagesPullingTime="163.409328ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:40:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:40:25.513184863 +0000 UTC m=+819.484436555" watchObservedRunningTime="2026-07-06 01:40:25.514190466 +0000 UTC m=+819.485442156" Jul 06 01:40:34.831103 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:34.831062 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nhrl7" Jul 06 01:40:34.831568 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:34.831138 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nhrl7" Jul 06 01:40:34.843802 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:34.843774 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nhrl7" Jul 06 01:40:35.552711 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:35.552680 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nhrl7" Jul 06 01:40:38.395832 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:40:38.395793 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:41:29.382386 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:41:29.382347 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d6bbcbc89-bl28q"] Jul 06 01:41:29.385527 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:41:29.385508 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d6bbcbc89-bl28q" Jul 06 01:41:29.394585 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:41:29.394545 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d6bbcbc89-bl28q"] Jul 06 01:41:29.447950 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:41:29.447903 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tnx9\" (UniqueName: \"kubernetes.io/projected/7beebb74-3118-44cb-8bac-6dbe85d41396-kube-api-access-8tnx9\") pod \"authorino-6d6bbcbc89-bl28q\" (UID: \"7beebb74-3118-44cb-8bac-6dbe85d41396\") " pod="kuadrant-system/authorino-6d6bbcbc89-bl28q" Jul 06 01:41:29.448142 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:41:29.447993 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7beebb74-3118-44cb-8bac-6dbe85d41396-tls-cert\") pod \"authorino-6d6bbcbc89-bl28q\" (UID: \"7beebb74-3118-44cb-8bac-6dbe85d41396\") " pod="kuadrant-system/authorino-6d6bbcbc89-bl28q" Jul 06 01:41:29.548951 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:41:29.548906 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tnx9\" (UniqueName: \"kubernetes.io/projected/7beebb74-3118-44cb-8bac-6dbe85d41396-kube-api-access-8tnx9\") pod \"authorino-6d6bbcbc89-bl28q\" (UID: \"7beebb74-3118-44cb-8bac-6dbe85d41396\") " pod="kuadrant-system/authorino-6d6bbcbc89-bl28q" Jul 06 01:41:29.549152 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:41:29.548986 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7beebb74-3118-44cb-8bac-6dbe85d41396-tls-cert\") pod \"authorino-6d6bbcbc89-bl28q\" (UID: \"7beebb74-3118-44cb-8bac-6dbe85d41396\") " pod="kuadrant-system/authorino-6d6bbcbc89-bl28q" Jul 06 01:41:29.551597 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:41:29.551570 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7beebb74-3118-44cb-8bac-6dbe85d41396-tls-cert\") pod \"authorino-6d6bbcbc89-bl28q\" (UID: \"7beebb74-3118-44cb-8bac-6dbe85d41396\") " pod="kuadrant-system/authorino-6d6bbcbc89-bl28q" Jul 06 01:41:29.556788 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:41:29.556758 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tnx9\" (UniqueName: \"kubernetes.io/projected/7beebb74-3118-44cb-8bac-6dbe85d41396-kube-api-access-8tnx9\") pod \"authorino-6d6bbcbc89-bl28q\" (UID: \"7beebb74-3118-44cb-8bac-6dbe85d41396\") " pod="kuadrant-system/authorino-6d6bbcbc89-bl28q" Jul 06 01:41:29.694463 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:41:29.694390 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d6bbcbc89-bl28q" Jul 06 01:41:29.826986 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:41:29.826951 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d6bbcbc89-bl28q"] Jul 06 01:41:29.830172 ip-10-0-139-44 kubenswrapper[2580]: W0706 01:41:29.830146 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7beebb74_3118_44cb_8bac_6dbe85d41396.slice/crio-e0a9192d1ba0ac34773ad13debfd537dab82a2079a933bebe2be03600c087d4c WatchSource:0}: Error finding container e0a9192d1ba0ac34773ad13debfd537dab82a2079a933bebe2be03600c087d4c: Status 404 returned error can't find the container with id e0a9192d1ba0ac34773ad13debfd537dab82a2079a933bebe2be03600c087d4c Jul 06 01:41:30.770326 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:41:30.770284 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6d6bbcbc89-bl28q" event={"ID":"7beebb74-3118-44cb-8bac-6dbe85d41396","Type":"ContainerStarted","Data":"2547def0cd9575020b4fb7f46cd94e8f6faa21ef65c6f147571a51e1a0ab8036"} Jul 06 01:41:30.770326 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:41:30.770328 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6d6bbcbc89-bl28q" event={"ID":"7beebb74-3118-44cb-8bac-6dbe85d41396","Type":"ContainerStarted","Data":"e0a9192d1ba0ac34773ad13debfd537dab82a2079a933bebe2be03600c087d4c"} Jul 06 01:41:30.789958 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:41:30.789879 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6d6bbcbc89-bl28q" podStartSLOduration=1.415377237 podStartE2EDuration="1.789856197s" totalImagesPullingTime="374.47896ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:41:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:41:30.788002315 +0000 UTC m=+884.759254007" watchObservedRunningTime="2026-07-06 01:41:30.789856197 +0000 UTC m=+884.761107888" Jul 06 01:41:30.814987 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:41:30.814952 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-67b87b8cfb-grk6m"] Jul 06 01:41:30.815215 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:41:30.815189 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-67b87b8cfb-grk6m" podUID="937ea1ee-e382-4254-93bd-3e8babadd77f" containerName="authorino" containerID="cri-o://7df2d8f3f39fb128617c73115829dfb4a43feb3b6a026df4e91fbac960064e70" gracePeriod=30 Jul 06 01:41:31.073185 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:41:31.073155 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-67b87b8cfb-grk6m" Jul 06 01:41:31.164841 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:41:31.164801 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/937ea1ee-e382-4254-93bd-3e8babadd77f-tls-cert\") pod \"937ea1ee-e382-4254-93bd-3e8babadd77f\" (UID: \"937ea1ee-e382-4254-93bd-3e8babadd77f\") " Jul 06 01:41:31.165018 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:41:31.164899 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rsjdq\" (UniqueName: \"kubernetes.io/projected/937ea1ee-e382-4254-93bd-3e8babadd77f-kube-api-access-rsjdq\") pod \"937ea1ee-e382-4254-93bd-3e8babadd77f\" (UID: \"937ea1ee-e382-4254-93bd-3e8babadd77f\") " Jul 06 01:41:31.167106 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:41:31.167069 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/937ea1ee-e382-4254-93bd-3e8babadd77f-kube-api-access-rsjdq" (OuterVolumeSpecName: "kube-api-access-rsjdq") pod "937ea1ee-e382-4254-93bd-3e8babadd77f" (UID: "937ea1ee-e382-4254-93bd-3e8babadd77f"). InnerVolumeSpecName "kube-api-access-rsjdq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 01:41:31.176711 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:41:31.176674 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/937ea1ee-e382-4254-93bd-3e8babadd77f-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "937ea1ee-e382-4254-93bd-3e8babadd77f" (UID: "937ea1ee-e382-4254-93bd-3e8babadd77f"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 01:41:31.265432 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:41:31.265390 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rsjdq\" (UniqueName: \"kubernetes.io/projected/937ea1ee-e382-4254-93bd-3e8babadd77f-kube-api-access-rsjdq\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 01:41:31.265432 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:41:31.265424 2580 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/937ea1ee-e382-4254-93bd-3e8babadd77f-tls-cert\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 01:41:31.348877 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:41:31.348786 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:41:31.453005 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:41:31.452970 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:41:31.777230 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:41:31.777189 2580 generic.go:356] "Generic (PLEG): container finished" podID="937ea1ee-e382-4254-93bd-3e8babadd77f" containerID="7df2d8f3f39fb128617c73115829dfb4a43feb3b6a026df4e91fbac960064e70" exitCode=0 Jul 06 01:41:31.777230 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:41:31.777230 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-67b87b8cfb-grk6m" Jul 06 01:41:31.777793 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:41:31.777274 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-67b87b8cfb-grk6m" event={"ID":"937ea1ee-e382-4254-93bd-3e8babadd77f","Type":"ContainerDied","Data":"7df2d8f3f39fb128617c73115829dfb4a43feb3b6a026df4e91fbac960064e70"} Jul 06 01:41:31.777793 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:41:31.777321 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-67b87b8cfb-grk6m" event={"ID":"937ea1ee-e382-4254-93bd-3e8babadd77f","Type":"ContainerDied","Data":"a6bd181d5ead0129e65ad06d78e0809f8f28fb30a0a072ad16aa93273ca628a7"} Jul 06 01:41:31.777793 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:41:31.777348 2580 scope.go:117] "RemoveContainer" containerID="7df2d8f3f39fb128617c73115829dfb4a43feb3b6a026df4e91fbac960064e70" Jul 06 01:41:31.786778 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:41:31.786756 2580 scope.go:117] "RemoveContainer" containerID="7df2d8f3f39fb128617c73115829dfb4a43feb3b6a026df4e91fbac960064e70" Jul 06 01:41:31.787117 ip-10-0-139-44 kubenswrapper[2580]: E0706 01:41:31.787095 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7df2d8f3f39fb128617c73115829dfb4a43feb3b6a026df4e91fbac960064e70\": container with ID starting with 7df2d8f3f39fb128617c73115829dfb4a43feb3b6a026df4e91fbac960064e70 not found: ID does not exist" containerID="7df2d8f3f39fb128617c73115829dfb4a43feb3b6a026df4e91fbac960064e70" Jul 06 01:41:31.787213 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:41:31.787124 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7df2d8f3f39fb128617c73115829dfb4a43feb3b6a026df4e91fbac960064e70"} err="failed to get container status \"7df2d8f3f39fb128617c73115829dfb4a43feb3b6a026df4e91fbac960064e70\": rpc error: code = NotFound desc = could not find container \"7df2d8f3f39fb128617c73115829dfb4a43feb3b6a026df4e91fbac960064e70\": container with ID starting with 7df2d8f3f39fb128617c73115829dfb4a43feb3b6a026df4e91fbac960064e70 not found: ID does not exist" Jul 06 01:41:31.798261 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:41:31.798235 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-67b87b8cfb-grk6m"] Jul 06 01:41:31.808865 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:41:31.808839 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-67b87b8cfb-grk6m"] Jul 06 01:41:32.459883 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:41:32.459848 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="937ea1ee-e382-4254-93bd-3e8babadd77f" path="/var/lib/kubelet/pods/937ea1ee-e382-4254-93bd-3e8babadd77f/volumes" Jul 06 01:41:32.592855 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:41:32.592819 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:41:32.689789 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:41:32.689757 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:41:52.295014 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:41:52.294974 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:42:21.360935 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:42:21.360894 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:42:21.460650 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:42:21.460617 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:42:42.905037 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:42:42.904999 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:42:44.577511 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:42:44.577473 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:43:25.236650 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:43:25.236606 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:43:25.332799 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:43:25.332763 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:43:25.467750 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:43:25.467717 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:43:31.312667 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:43:31.312618 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:43:31.420312 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:43:31.420281 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:43:31.514505 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:43:31.514471 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:43:31.744741 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:43:31.744712 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:43:37.630376 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:43:37.630330 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:43:37.702548 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:43:37.702510 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:46:43.222779 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:46:43.222746 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:46:46.460676 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:46:46.460644 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:51:31.163296 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:51:31.163259 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:53:36.911538 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:53:36.911453 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:54:20.810523 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:54:20.810486 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:54:26.189356 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:54:26.189305 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:54:34.450761 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:54:34.450726 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:54:39.591252 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:54:39.591217 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:54:42.694538 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:54:42.694500 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:54:42.784289 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:54:42.784251 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:54:43.156521 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:54:43.156484 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:54:48.953519 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:54:48.953479 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:54:49.056698 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:54:49.056642 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:54:49.177992 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:54:49.177958 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:54:49.253593 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:54:49.253484 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:54:50.461157 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:54:50.461119 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:54:50.557843 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:54:50.557805 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:54:50.627494 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:54:50.627459 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:54:50.813996 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:54:50.813913 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:54:51.052266 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:54:51.052233 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:54:51.154342 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:54:51.154303 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:54:52.946808 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:54:52.946725 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:54:55.592983 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:54:55.592943 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:54:55.764762 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:54:55.764714 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:54:55.869542 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:54:55.869455 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:54:56.005732 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:54:56.005699 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:54:59.340827 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:54:59.340786 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:01.833486 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:01.833428 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:01.895960 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:01.895924 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:01.989372 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:01.989339 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:02.107785 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:02.107692 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:02.357930 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:02.357842 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:02.549452 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:02.549394 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:03.638349 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:03.638305 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:03.756558 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:03.756523 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:04.225924 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:04.225891 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:04.350904 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:04.350865 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:04.453009 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:04.452978 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:05.723351 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:05.723316 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:08.236145 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:08.236061 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:08.394813 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:08.394777 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:08.502236 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:08.502138 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:08.629639 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:08.629594 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:10.539146 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:10.539106 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:14.382031 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:14.381988 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:14.482431 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:14.482391 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:14.616400 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:14.616345 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:14.711972 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:14.711931 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:15.820210 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:15.820181 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:15.902364 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:15.902334 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:15.969707 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:15.969674 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:16.101052 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:16.100963 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:16.485768 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:16.485725 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:16.586342 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:16.586303 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:18.002962 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:18.002927 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:21.167708 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:21.167645 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:21.573550 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:21.573449 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:27.472795 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:27.472752 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:27.571865 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:27.571828 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:33.442711 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:33.442671 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:33.502136 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:33.502097 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:33.677943 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:33.677898 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:34.605932 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:34.605892 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:34.665451 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:34.665413 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:34.990315 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:34.990279 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:35.085028 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:35.084996 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:35.187412 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:35.187376 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:35.393795 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:35.393759 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:35.808353 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:35.808251 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:35.960708 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:35.960674 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:37.913033 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:37.912983 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:39.385152 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:39.385115 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:39.557234 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:39.557187 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:39.672432 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:39.672392 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:39.917538 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:39.917488 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:41.143616 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:41.143582 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:45.603824 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:45.603784 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:45.702707 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:45.702674 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:46.088757 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:46.088713 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:46.182824 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:46.182781 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:46.285777 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:46.285735 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:47.697217 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:47.697181 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:53.343240 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:53.343202 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:53.448964 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:53.448909 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:53.543960 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:53.543923 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:53.644645 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:53.644607 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:55:54.079784 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:55:54.079690 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:56:20.051503 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:56:20.051464 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:56:25.901340 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:56:25.901295 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:56:26.003997 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:56:26.003953 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:56:26.101684 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:56:26.101635 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:56:42.022735 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:56:42.022652 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:56:48.432195 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:56:48.432159 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:56:48.499155 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:56:48.499114 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:56:48.576679 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:56:48.576630 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:56:59.152016 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:56:59.151972 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:56:59.967594 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:56:59.967555 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:57:05.840150 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:57:05.840099 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:57:05.953179 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:57:05.953140 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:57:06.040279 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:57:06.040241 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:57:06.139770 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:57:06.139717 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:57:06.938574 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:57:06.938544 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:57:07.037811 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:57:07.037776 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:57:07.137459 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:57:07.137408 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:57:08.810697 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:57:08.810663 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:57:14.593823 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:57:14.593786 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:57:14.672229 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:57:14.672198 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:57:15.279635 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:57:15.279595 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:57:21.318905 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:57:21.318862 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:57:21.420120 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:57:21.420073 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:57:21.622537 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:57:21.622431 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:57:22.009942 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:57:22.009907 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:57:22.270664 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:57:22.270579 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:57:24.992002 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:57:24.991940 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:57:25.098012 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:57:25.097963 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:57:25.192494 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:57:25.192451 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:57:25.323481 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:57:25.323378 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:57:25.822347 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:57:25.822309 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:57:25.923998 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:57:25.923965 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:57:29.538363 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:57:29.538331 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:57:30.257763 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:57:30.257723 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:57:30.354281 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:57:30.354238 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:57:30.683307 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:57:30.683262 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:57:36.622156 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:57:36.622101 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:57:36.692910 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:57:36.692872 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:57:37.062494 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:57:37.062457 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:57:37.285350 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:57:37.285303 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:57:37.452054 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:57:37.451996 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:57:37.522423 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:57:37.522392 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:57:43.320099 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:57:43.320061 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:57:43.385924 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:57:43.385889 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:57:43.608819 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:57:43.608706 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:57:43.835953 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:57:43.835901 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:57:43.979151 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:57:43.979117 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:57:44.075546 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:57:44.075510 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:57:44.181803 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:57:44.181771 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:57:49.864667 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:57:49.864635 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:57:49.976572 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:57:49.976538 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:57:50.065334 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:57:50.065291 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:57:50.278550 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:57:50.278507 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:57:56.669596 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:57:56.669558 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:57:56.838278 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:57:56.838240 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:58:03.720621 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:58:03.720579 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:58:04.013337 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:58:04.013232 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:58:04.109619 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:58:04.109581 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:58:07.167405 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:58:07.167313 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:58:07.270989 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:58:07.270943 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:58:07.644456 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:58:07.644399 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:58:07.946061 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:58:07.945980 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:58:08.343123 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:58:08.343024 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:58:08.450822 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:58:08.450788 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 01:58:11.683253 ip-10-0-139-44 kubenswrapper[2580]: I0706 01:58:11.683217 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:00:45.939500 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:00:45.939429 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:00:46.051748 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:00:46.051707 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:01:16.653047 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:01:16.652951 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:01:16.989050 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:01:16.988965 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:01:17.086937 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:01:17.086894 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:01:22.716564 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:01:22.716525 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:01:22.817505 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:01:22.817466 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:01:22.982243 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:01:22.982141 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:01:23.213948 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:01:23.213913 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:01:24.323193 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:01:24.323147 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:01:24.422488 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:01:24.422418 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:01:24.522997 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:01:24.522959 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:01:25.325336 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:01:25.325301 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:01:25.478961 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:01:25.478931 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:01:25.580563 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:01:25.580424 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:01:26.927295 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:01:26.927262 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:01:32.576770 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:01:32.576734 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:01:32.675030 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:01:32.674998 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:01:33.250346 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:01:33.250309 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:01:38.971906 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:01:38.971869 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:01:39.035683 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:01:39.035653 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:01:39.122473 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:01:39.122418 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:01:39.252876 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:01:39.252779 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:01:49.714091 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:01:49.714056 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:01:50.980466 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:01:50.980418 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:02:11.066379 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:02:11.066344 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:02:16.850293 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:02:16.850240 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:02:16.910827 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:02:16.910787 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:02:17.215309 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:02:17.215270 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:02:17.523488 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:02:17.523367 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:02:17.846412 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:02:17.846327 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:02:18.147783 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:02:18.147751 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:02:20.652861 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:02:20.652824 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:02:20.747119 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:02:20.747086 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:02:20.820739 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:02:20.820703 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:02:21.137763 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:02:21.137723 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:02:21.302148 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:02:21.302113 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:02:21.402952 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:02:21.402856 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:02:25.162486 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:02:25.162423 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:02:25.506509 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:02:25.506401 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:02:25.825605 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:02:25.825517 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:02:25.937208 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:02:25.937177 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:02:28.697416 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:02:28.697379 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:02:28.782333 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:02:28.782302 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:02:28.853850 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:02:28.853817 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:02:28.957650 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:02:28.957543 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:02:29.250823 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:02:29.250722 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:02:29.351746 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:02:29.351712 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:02:33.205399 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:02:33.205360 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:02:42.156992 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:02:42.156890 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:02:42.258654 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:02:42.258614 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:02:49.987364 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:02:49.987326 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:02:50.087523 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:02:50.087484 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:03:02.325412 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:03:02.325374 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-f285a0/test-model-f285a0-kserve-7d94f9b7c7-fknjn"] Jul 06 02:03:02.326154 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:03:02.325743 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="937ea1ee-e382-4254-93bd-3e8babadd77f" containerName="authorino" Jul 06 02:03:02.326154 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:03:02.325757 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="937ea1ee-e382-4254-93bd-3e8babadd77f" containerName="authorino" Jul 06 02:03:02.326154 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:03:02.325832 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="937ea1ee-e382-4254-93bd-3e8babadd77f" containerName="authorino" Jul 06 02:03:02.328719 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:03:02.328695 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-f285a0/test-model-f285a0-kserve-7d94f9b7c7-fknjn" Jul 06 02:03:02.331271 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:03:02.331251 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-f285a0\"/\"kube-root-ca.crt\"" Jul 06 02:03:02.331401 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:03:02.331301 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-f285a0\"/\"test-model-f285a0-kserve-self-signed-certs\"" Jul 06 02:03:02.332418 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:03:02.332391 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-f285a0\"/\"openshift-service-ca.crt\"" Jul 06 02:03:02.332567 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:03:02.332456 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-f285a0\"/\"default-dockercfg-9vnkk\"" Jul 06 02:03:02.337922 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:03:02.337895 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-f285a0/test-model-f285a0-kserve-7d94f9b7c7-fknjn"] Jul 06 02:03:02.358848 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:03:02.358812 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:03:02.388962 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:03:02.388919 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be93d58f-9e08-4468-b5b5-af396c4dab60-tmp-dir\") pod \"test-model-f285a0-kserve-7d94f9b7c7-fknjn\" (UID: \"be93d58f-9e08-4468-b5b5-af396c4dab60\") " pod="ai-tenant-e2e-inf-a-f285a0/test-model-f285a0-kserve-7d94f9b7c7-fknjn" Jul 06 02:03:02.389126 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:03:02.388987 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be93d58f-9e08-4468-b5b5-af396c4dab60-model-cache\") pod \"test-model-f285a0-kserve-7d94f9b7c7-fknjn\" (UID: \"be93d58f-9e08-4468-b5b5-af396c4dab60\") " pod="ai-tenant-e2e-inf-a-f285a0/test-model-f285a0-kserve-7d94f9b7c7-fknjn" Jul 06 02:03:02.389126 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:03:02.389046 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhwj5\" (UniqueName: \"kubernetes.io/projected/be93d58f-9e08-4468-b5b5-af396c4dab60-kube-api-access-qhwj5\") pod \"test-model-f285a0-kserve-7d94f9b7c7-fknjn\" (UID: \"be93d58f-9e08-4468-b5b5-af396c4dab60\") " pod="ai-tenant-e2e-inf-a-f285a0/test-model-f285a0-kserve-7d94f9b7c7-fknjn" Jul 06 02:03:02.389126 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:03:02.389102 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be93d58f-9e08-4468-b5b5-af396c4dab60-tls-certs\") pod \"test-model-f285a0-kserve-7d94f9b7c7-fknjn\" (UID: \"be93d58f-9e08-4468-b5b5-af396c4dab60\") " pod="ai-tenant-e2e-inf-a-f285a0/test-model-f285a0-kserve-7d94f9b7c7-fknjn" Jul 06 02:03:02.389227 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:03:02.389138 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/be93d58f-9e08-4468-b5b5-af396c4dab60-dshm\") pod \"test-model-f285a0-kserve-7d94f9b7c7-fknjn\" (UID: \"be93d58f-9e08-4468-b5b5-af396c4dab60\") " pod="ai-tenant-e2e-inf-a-f285a0/test-model-f285a0-kserve-7d94f9b7c7-fknjn" Jul 06 02:03:02.389227 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:03:02.389166 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/be93d58f-9e08-4468-b5b5-af396c4dab60-home\") pod \"test-model-f285a0-kserve-7d94f9b7c7-fknjn\" (UID: \"be93d58f-9e08-4468-b5b5-af396c4dab60\") " pod="ai-tenant-e2e-inf-a-f285a0/test-model-f285a0-kserve-7d94f9b7c7-fknjn" Jul 06 02:03:02.490609 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:03:02.490567 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be93d58f-9e08-4468-b5b5-af396c4dab60-tls-certs\") pod \"test-model-f285a0-kserve-7d94f9b7c7-fknjn\" (UID: \"be93d58f-9e08-4468-b5b5-af396c4dab60\") " pod="ai-tenant-e2e-inf-a-f285a0/test-model-f285a0-kserve-7d94f9b7c7-fknjn" Jul 06 02:03:02.490609 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:03:02.490612 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/be93d58f-9e08-4468-b5b5-af396c4dab60-dshm\") pod \"test-model-f285a0-kserve-7d94f9b7c7-fknjn\" (UID: \"be93d58f-9e08-4468-b5b5-af396c4dab60\") " pod="ai-tenant-e2e-inf-a-f285a0/test-model-f285a0-kserve-7d94f9b7c7-fknjn" Jul 06 02:03:02.490877 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:03:02.490633 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/be93d58f-9e08-4468-b5b5-af396c4dab60-home\") pod \"test-model-f285a0-kserve-7d94f9b7c7-fknjn\" (UID: \"be93d58f-9e08-4468-b5b5-af396c4dab60\") " pod="ai-tenant-e2e-inf-a-f285a0/test-model-f285a0-kserve-7d94f9b7c7-fknjn" Jul 06 02:03:02.490877 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:03:02.490669 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be93d58f-9e08-4468-b5b5-af396c4dab60-tmp-dir\") pod \"test-model-f285a0-kserve-7d94f9b7c7-fknjn\" (UID: \"be93d58f-9e08-4468-b5b5-af396c4dab60\") " pod="ai-tenant-e2e-inf-a-f285a0/test-model-f285a0-kserve-7d94f9b7c7-fknjn" Jul 06 02:03:02.490877 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:03:02.490728 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be93d58f-9e08-4468-b5b5-af396c4dab60-model-cache\") pod \"test-model-f285a0-kserve-7d94f9b7c7-fknjn\" (UID: \"be93d58f-9e08-4468-b5b5-af396c4dab60\") " pod="ai-tenant-e2e-inf-a-f285a0/test-model-f285a0-kserve-7d94f9b7c7-fknjn" Jul 06 02:03:02.490877 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:03:02.490759 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhwj5\" (UniqueName: \"kubernetes.io/projected/be93d58f-9e08-4468-b5b5-af396c4dab60-kube-api-access-qhwj5\") pod \"test-model-f285a0-kserve-7d94f9b7c7-fknjn\" (UID: \"be93d58f-9e08-4468-b5b5-af396c4dab60\") " pod="ai-tenant-e2e-inf-a-f285a0/test-model-f285a0-kserve-7d94f9b7c7-fknjn" Jul 06 02:03:02.491212 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:03:02.491175 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be93d58f-9e08-4468-b5b5-af396c4dab60-tmp-dir\") pod \"test-model-f285a0-kserve-7d94f9b7c7-fknjn\" (UID: \"be93d58f-9e08-4468-b5b5-af396c4dab60\") " pod="ai-tenant-e2e-inf-a-f285a0/test-model-f285a0-kserve-7d94f9b7c7-fknjn" Jul 06 02:03:02.491212 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:03:02.491192 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/be93d58f-9e08-4468-b5b5-af396c4dab60-home\") pod \"test-model-f285a0-kserve-7d94f9b7c7-fknjn\" (UID: \"be93d58f-9e08-4468-b5b5-af396c4dab60\") " pod="ai-tenant-e2e-inf-a-f285a0/test-model-f285a0-kserve-7d94f9b7c7-fknjn" Jul 06 02:03:02.491595 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:03:02.491233 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be93d58f-9e08-4468-b5b5-af396c4dab60-model-cache\") pod \"test-model-f285a0-kserve-7d94f9b7c7-fknjn\" (UID: \"be93d58f-9e08-4468-b5b5-af396c4dab60\") " pod="ai-tenant-e2e-inf-a-f285a0/test-model-f285a0-kserve-7d94f9b7c7-fknjn" Jul 06 02:03:02.493157 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:03:02.493137 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/be93d58f-9e08-4468-b5b5-af396c4dab60-dshm\") pod \"test-model-f285a0-kserve-7d94f9b7c7-fknjn\" (UID: \"be93d58f-9e08-4468-b5b5-af396c4dab60\") " pod="ai-tenant-e2e-inf-a-f285a0/test-model-f285a0-kserve-7d94f9b7c7-fknjn" Jul 06 02:03:02.493407 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:03:02.493391 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be93d58f-9e08-4468-b5b5-af396c4dab60-tls-certs\") pod \"test-model-f285a0-kserve-7d94f9b7c7-fknjn\" (UID: \"be93d58f-9e08-4468-b5b5-af396c4dab60\") " pod="ai-tenant-e2e-inf-a-f285a0/test-model-f285a0-kserve-7d94f9b7c7-fknjn" Jul 06 02:03:02.499219 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:03:02.499197 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhwj5\" (UniqueName: \"kubernetes.io/projected/be93d58f-9e08-4468-b5b5-af396c4dab60-kube-api-access-qhwj5\") pod \"test-model-f285a0-kserve-7d94f9b7c7-fknjn\" (UID: \"be93d58f-9e08-4468-b5b5-af396c4dab60\") " pod="ai-tenant-e2e-inf-a-f285a0/test-model-f285a0-kserve-7d94f9b7c7-fknjn" Jul 06 02:03:02.639389 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:03:02.639344 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-f285a0/test-model-f285a0-kserve-7d94f9b7c7-fknjn" Jul 06 02:03:02.777473 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:03:02.777406 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-f285a0/test-model-f285a0-kserve-7d94f9b7c7-fknjn"] Jul 06 02:03:02.781179 ip-10-0-139-44 kubenswrapper[2580]: W0706 02:03:02.781152 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbe93d58f_9e08_4468_b5b5_af396c4dab60.slice/crio-384ba99aba4362452820ff31701d7d4324b0384e6960e6fb1f17f40a46f17f5a WatchSource:0}: Error finding container 384ba99aba4362452820ff31701d7d4324b0384e6960e6fb1f17f40a46f17f5a: Status 404 returned error can't find the container with id 384ba99aba4362452820ff31701d7d4324b0384e6960e6fb1f17f40a46f17f5a Jul 06 02:03:03.141256 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:03:03.141213 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-f285a0/test-model-f285a0-kserve-7d94f9b7c7-fknjn" event={"ID":"be93d58f-9e08-4468-b5b5-af396c4dab60","Type":"ContainerStarted","Data":"433df6d675702fd144ce35d46970e60eec8b98d5640d8fca1289be6fdabf4a55"} Jul 06 02:03:03.141256 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:03:03.141256 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-f285a0/test-model-f285a0-kserve-7d94f9b7c7-fknjn" event={"ID":"be93d58f-9e08-4468-b5b5-af396c4dab60","Type":"ContainerStarted","Data":"384ba99aba4362452820ff31701d7d4324b0384e6960e6fb1f17f40a46f17f5a"} Jul 06 02:03:03.162746 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:03:03.162695 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-f285a0/test-model-f285a0-kserve-7d94f9b7c7-fknjn" podStartSLOduration=1.162679259 podStartE2EDuration="1.162679259s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 02:03:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 02:03:03.162102214 +0000 UTC m=+2177.133353907" watchObservedRunningTime="2026-07-06 02:03:03.162679259 +0000 UTC m=+2177.133930948" Jul 06 02:03:07.985697 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:03:07.985664 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:03:08.086745 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:03:08.086709 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:03:11.584517 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:03:11.584471 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:03:11.658200 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:03:11.658157 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:03:12.639833 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:03:12.639797 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-f285a0/test-model-f285a0-kserve-7d94f9b7c7-fknjn" Jul 06 02:03:12.640241 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:03:12.639854 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-f285a0/test-model-f285a0-kserve-7d94f9b7c7-fknjn" Jul 06 02:03:12.652415 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:03:12.652384 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-f285a0/test-model-f285a0-kserve-7d94f9b7c7-fknjn" Jul 06 02:03:13.194579 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:03:13.194551 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-f285a0/test-model-f285a0-kserve-7d94f9b7c7-fknjn" Jul 06 02:03:26.262249 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:03:26.262211 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:03:47.116951 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:03:47.116913 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-f285a0/test-model-f285a0-kserve-7d94f9b7c7-fknjn"] Jul 06 02:03:47.143079 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:03:47.143025 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:03:48.044283 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:03:48.044245 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:03:48.237379 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:03:48.237349 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:03:48.609802 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:03:48.609764 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:03:48.712834 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:03:48.712796 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:03:48.810164 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:03:48.810130 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:03:48.945868 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:03:48.945838 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:03:53.683636 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:03:53.683593 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:04:02.140293 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:02.140246 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-f285a0/test-model-f285a0-kserve-7d94f9b7c7-fknjn" podUID="be93d58f-9e08-4468-b5b5-af396c4dab60" containerName="main" containerID="cri-o://433df6d675702fd144ce35d46970e60eec8b98d5640d8fca1289be6fdabf4a55" gracePeriod=45 Jul 06 02:04:02.385156 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:02.385127 2580 generic.go:356] "Generic (PLEG): container finished" podID="be93d58f-9e08-4468-b5b5-af396c4dab60" containerID="433df6d675702fd144ce35d46970e60eec8b98d5640d8fca1289be6fdabf4a55" exitCode=0 Jul 06 02:04:02.385293 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:02.385206 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-f285a0/test-model-f285a0-kserve-7d94f9b7c7-fknjn" event={"ID":"be93d58f-9e08-4468-b5b5-af396c4dab60","Type":"ContainerDied","Data":"433df6d675702fd144ce35d46970e60eec8b98d5640d8fca1289be6fdabf4a55"} Jul 06 02:04:02.385293 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:02.385242 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-f285a0/test-model-f285a0-kserve-7d94f9b7c7-fknjn" event={"ID":"be93d58f-9e08-4468-b5b5-af396c4dab60","Type":"ContainerDied","Data":"384ba99aba4362452820ff31701d7d4324b0384e6960e6fb1f17f40a46f17f5a"} Jul 06 02:04:02.385293 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:02.385254 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="384ba99aba4362452820ff31701d7d4324b0384e6960e6fb1f17f40a46f17f5a" Jul 06 02:04:02.391609 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:02.391557 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-f285a0/test-model-f285a0-kserve-7d94f9b7c7-fknjn" Jul 06 02:04:02.416192 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:02.416167 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be93d58f-9e08-4468-b5b5-af396c4dab60-tmp-dir\") pod \"be93d58f-9e08-4468-b5b5-af396c4dab60\" (UID: \"be93d58f-9e08-4468-b5b5-af396c4dab60\") " Jul 06 02:04:02.416333 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:02.416207 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/be93d58f-9e08-4468-b5b5-af396c4dab60-dshm\") pod \"be93d58f-9e08-4468-b5b5-af396c4dab60\" (UID: \"be93d58f-9e08-4468-b5b5-af396c4dab60\") " Jul 06 02:04:02.416333 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:02.416242 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be93d58f-9e08-4468-b5b5-af396c4dab60-tls-certs\") pod \"be93d58f-9e08-4468-b5b5-af396c4dab60\" (UID: \"be93d58f-9e08-4468-b5b5-af396c4dab60\") " Jul 06 02:04:02.416333 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:02.416257 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/be93d58f-9e08-4468-b5b5-af396c4dab60-home\") pod \"be93d58f-9e08-4468-b5b5-af396c4dab60\" (UID: \"be93d58f-9e08-4468-b5b5-af396c4dab60\") " Jul 06 02:04:02.416333 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:02.416305 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be93d58f-9e08-4468-b5b5-af396c4dab60-model-cache\") pod \"be93d58f-9e08-4468-b5b5-af396c4dab60\" (UID: \"be93d58f-9e08-4468-b5b5-af396c4dab60\") " Jul 06 02:04:02.416572 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:02.416348 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qhwj5\" (UniqueName: \"kubernetes.io/projected/be93d58f-9e08-4468-b5b5-af396c4dab60-kube-api-access-qhwj5\") pod \"be93d58f-9e08-4468-b5b5-af396c4dab60\" (UID: \"be93d58f-9e08-4468-b5b5-af396c4dab60\") " Jul 06 02:04:02.416572 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:02.416487 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be93d58f-9e08-4468-b5b5-af396c4dab60-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "be93d58f-9e08-4468-b5b5-af396c4dab60" (UID: "be93d58f-9e08-4468-b5b5-af396c4dab60"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:04:02.416687 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:02.416584 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be93d58f-9e08-4468-b5b5-af396c4dab60-home" (OuterVolumeSpecName: "home") pod "be93d58f-9e08-4468-b5b5-af396c4dab60" (UID: "be93d58f-9e08-4468-b5b5-af396c4dab60"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:04:02.416687 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:02.416621 2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be93d58f-9e08-4468-b5b5-af396c4dab60-tmp-dir\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 02:04:02.416687 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:02.416623 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be93d58f-9e08-4468-b5b5-af396c4dab60-model-cache" (OuterVolumeSpecName: "model-cache") pod "be93d58f-9e08-4468-b5b5-af396c4dab60" (UID: "be93d58f-9e08-4468-b5b5-af396c4dab60"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:04:02.418598 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:02.418570 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be93d58f-9e08-4468-b5b5-af396c4dab60-dshm" (OuterVolumeSpecName: "dshm") pod "be93d58f-9e08-4468-b5b5-af396c4dab60" (UID: "be93d58f-9e08-4468-b5b5-af396c4dab60"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:04:02.418678 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:02.418597 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be93d58f-9e08-4468-b5b5-af396c4dab60-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "be93d58f-9e08-4468-b5b5-af396c4dab60" (UID: "be93d58f-9e08-4468-b5b5-af396c4dab60"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 02:04:02.418678 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:02.418658 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be93d58f-9e08-4468-b5b5-af396c4dab60-kube-api-access-qhwj5" (OuterVolumeSpecName: "kube-api-access-qhwj5") pod "be93d58f-9e08-4468-b5b5-af396c4dab60" (UID: "be93d58f-9e08-4468-b5b5-af396c4dab60"). InnerVolumeSpecName "kube-api-access-qhwj5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 02:04:02.517527 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:02.517489 2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be93d58f-9e08-4468-b5b5-af396c4dab60-model-cache\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 02:04:02.517527 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:02.517519 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qhwj5\" (UniqueName: \"kubernetes.io/projected/be93d58f-9e08-4468-b5b5-af396c4dab60-kube-api-access-qhwj5\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 02:04:02.517527 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:02.517528 2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/be93d58f-9e08-4468-b5b5-af396c4dab60-dshm\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 02:04:02.517527 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:02.517536 2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be93d58f-9e08-4468-b5b5-af396c4dab60-tls-certs\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 02:04:02.517802 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:02.517545 2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/be93d58f-9e08-4468-b5b5-af396c4dab60-home\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 02:04:03.389167 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:03.389130 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-f285a0/test-model-f285a0-kserve-7d94f9b7c7-fknjn" Jul 06 02:04:03.410634 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:03.410603 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-f285a0/test-model-f285a0-kserve-7d94f9b7c7-fknjn"] Jul 06 02:04:03.413005 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:03.412976 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-f285a0/test-model-f285a0-kserve-7d94f9b7c7-fknjn"] Jul 06 02:04:04.459122 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:04.459082 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be93d58f-9e08-4468-b5b5-af396c4dab60" path="/var/lib/kubelet/pods/be93d58f-9e08-4468-b5b5-af396c4dab60/volumes" Jul 06 02:04:30.371883 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:30.371848 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:04:30.701310 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:30.701271 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:04:30.902188 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:30.902157 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:04:31.073811 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:31.073714 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:04:31.478054 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:31.478018 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:04:31.582399 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:31.582351 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:04:31.682536 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:31.682506 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:04:31.780512 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:31.780405 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:04:37.437833 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:37.437786 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:04:37.538186 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:37.538159 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:04:37.729312 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:37.729212 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:04:37.941903 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:37.941866 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:04:38.291237 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:38.291201 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:04:38.628148 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:38.628061 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:04:39.042707 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:39.042674 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:04:41.570338 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:41.570298 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:04:41.648282 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:41.648247 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:04:41.720313 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:41.720277 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:04:41.899786 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:41.899750 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:04:42.270588 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:42.270493 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:04:42.368674 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:42.368635 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:04:46.064629 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:46.064595 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:04:46.521370 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:46.521337 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:04:46.845327 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:46.845231 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:04:47.410792 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:47.410754 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:04:49.598925 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:49.598887 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:04:49.681825 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:49.681787 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:04:49.755946 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:49.755901 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:04:50.053347 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:50.053308 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:04:50.496837 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:50.496798 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:04:50.594454 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:50.594410 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:04:54.118409 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:04:54.118371 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:05:02.293895 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:02.293856 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:05:02.730651 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:02.730613 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:05:02.864267 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:02.864237 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:05:02.934234 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:02.934196 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:05:03.135481 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:03.135370 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:05:08.581524 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:08.581491 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:05:08.680956 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:08.680921 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:05:08.847291 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:08.847198 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:05:09.069056 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:09.069015 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:05:09.259456 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:09.259399 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:05:09.624490 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:09.624376 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:05:09.723351 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:09.723315 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:05:09.824638 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:09.824605 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:05:09.997417 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:09.997378 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:05:15.820763 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:15.820728 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:05:15.923961 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:15.923916 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:05:16.094371 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:16.094281 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:05:16.313606 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:16.313573 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:05:16.649849 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:16.649805 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:05:16.980783 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:16.980684 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:05:17.451127 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:17.451092 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:05:19.788156 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:19.788121 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:05:19.861843 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:19.861811 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:05:19.931933 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:19.931896 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:05:20.475308 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:20.475271 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:05:20.612446 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:20.612395 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:05:20.715998 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:20.715956 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:05:24.294005 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:24.293959 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:05:24.634526 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:24.634421 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:05:24.945630 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:24.945596 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:05:25.230969 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:25.230873 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:05:27.877701 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:27.877669 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:05:27.968642 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:27.968607 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:05:28.033646 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:28.033613 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:05:28.268855 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:28.268816 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:05:28.820274 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:28.820240 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:05:28.922124 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:28.922086 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:05:32.363808 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:32.363715 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:05:41.432973 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:41.432879 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:05:41.533394 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:41.533360 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:05:42.018827 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:42.018798 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-388824/auth-test-model-388824-kserve-597d77d9c5-8px62"] Jul 06 02:05:42.019093 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:42.019080 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be93d58f-9e08-4468-b5b5-af396c4dab60" containerName="main" Jul 06 02:05:42.019093 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:42.019093 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="be93d58f-9e08-4468-b5b5-af396c4dab60" containerName="main" Jul 06 02:05:42.019184 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:42.019146 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="be93d58f-9e08-4468-b5b5-af396c4dab60" containerName="main" Jul 06 02:05:42.022078 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:42.022062 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-388824/auth-test-model-388824-kserve-597d77d9c5-8px62" Jul 06 02:05:42.024867 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:42.024844 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-388824\"/\"kube-root-ca.crt\"" Jul 06 02:05:42.026089 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:42.026065 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-388824\"/\"default-dockercfg-d8pxv\"" Jul 06 02:05:42.026244 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:42.026065 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-388824\"/\"openshift-service-ca.crt\"" Jul 06 02:05:42.026244 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:42.026070 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-388824\"/\"auth-test-model-388824-kserve-self-signed-certs\"" Jul 06 02:05:42.030804 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:42.030777 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-388824/auth-test-model-388824-kserve-597d77d9c5-8px62"] Jul 06 02:05:42.050413 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:42.050379 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:05:42.080771 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:42.080730 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c7a12cc4-5a6e-49e5-9906-7ec6df3b2032-model-cache\") pod \"auth-test-model-388824-kserve-597d77d9c5-8px62\" (UID: \"c7a12cc4-5a6e-49e5-9906-7ec6df3b2032\") " pod="ai-tenant-e2e-shared-a-388824/auth-test-model-388824-kserve-597d77d9c5-8px62" Jul 06 02:05:42.080960 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:42.080789 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9hxl\" (UniqueName: \"kubernetes.io/projected/c7a12cc4-5a6e-49e5-9906-7ec6df3b2032-kube-api-access-c9hxl\") pod \"auth-test-model-388824-kserve-597d77d9c5-8px62\" (UID: \"c7a12cc4-5a6e-49e5-9906-7ec6df3b2032\") " pod="ai-tenant-e2e-shared-a-388824/auth-test-model-388824-kserve-597d77d9c5-8px62" Jul 06 02:05:42.080960 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:42.080821 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c7a12cc4-5a6e-49e5-9906-7ec6df3b2032-tmp-dir\") pod \"auth-test-model-388824-kserve-597d77d9c5-8px62\" (UID: \"c7a12cc4-5a6e-49e5-9906-7ec6df3b2032\") " pod="ai-tenant-e2e-shared-a-388824/auth-test-model-388824-kserve-597d77d9c5-8px62" Jul 06 02:05:42.080960 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:42.080870 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7a12cc4-5a6e-49e5-9906-7ec6df3b2032-tls-certs\") pod \"auth-test-model-388824-kserve-597d77d9c5-8px62\" (UID: \"c7a12cc4-5a6e-49e5-9906-7ec6df3b2032\") " pod="ai-tenant-e2e-shared-a-388824/auth-test-model-388824-kserve-597d77d9c5-8px62" Jul 06 02:05:42.080960 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:42.080932 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c7a12cc4-5a6e-49e5-9906-7ec6df3b2032-dshm\") pod \"auth-test-model-388824-kserve-597d77d9c5-8px62\" (UID: \"c7a12cc4-5a6e-49e5-9906-7ec6df3b2032\") " pod="ai-tenant-e2e-shared-a-388824/auth-test-model-388824-kserve-597d77d9c5-8px62" Jul 06 02:05:42.081140 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:42.080993 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c7a12cc4-5a6e-49e5-9906-7ec6df3b2032-home\") pod \"auth-test-model-388824-kserve-597d77d9c5-8px62\" (UID: \"c7a12cc4-5a6e-49e5-9906-7ec6df3b2032\") " pod="ai-tenant-e2e-shared-a-388824/auth-test-model-388824-kserve-597d77d9c5-8px62" Jul 06 02:05:42.181456 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:42.181379 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c7a12cc4-5a6e-49e5-9906-7ec6df3b2032-dshm\") pod \"auth-test-model-388824-kserve-597d77d9c5-8px62\" (UID: \"c7a12cc4-5a6e-49e5-9906-7ec6df3b2032\") " pod="ai-tenant-e2e-shared-a-388824/auth-test-model-388824-kserve-597d77d9c5-8px62" Jul 06 02:05:42.181456 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:42.181462 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c7a12cc4-5a6e-49e5-9906-7ec6df3b2032-home\") pod \"auth-test-model-388824-kserve-597d77d9c5-8px62\" (UID: \"c7a12cc4-5a6e-49e5-9906-7ec6df3b2032\") " pod="ai-tenant-e2e-shared-a-388824/auth-test-model-388824-kserve-597d77d9c5-8px62" Jul 06 02:05:42.181682 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:42.181497 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c7a12cc4-5a6e-49e5-9906-7ec6df3b2032-model-cache\") pod \"auth-test-model-388824-kserve-597d77d9c5-8px62\" (UID: \"c7a12cc4-5a6e-49e5-9906-7ec6df3b2032\") " pod="ai-tenant-e2e-shared-a-388824/auth-test-model-388824-kserve-597d77d9c5-8px62" Jul 06 02:05:42.181682 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:42.181538 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9hxl\" (UniqueName: \"kubernetes.io/projected/c7a12cc4-5a6e-49e5-9906-7ec6df3b2032-kube-api-access-c9hxl\") pod \"auth-test-model-388824-kserve-597d77d9c5-8px62\" (UID: \"c7a12cc4-5a6e-49e5-9906-7ec6df3b2032\") " pod="ai-tenant-e2e-shared-a-388824/auth-test-model-388824-kserve-597d77d9c5-8px62" Jul 06 02:05:42.181756 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:42.181675 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c7a12cc4-5a6e-49e5-9906-7ec6df3b2032-tmp-dir\") pod \"auth-test-model-388824-kserve-597d77d9c5-8px62\" (UID: \"c7a12cc4-5a6e-49e5-9906-7ec6df3b2032\") " pod="ai-tenant-e2e-shared-a-388824/auth-test-model-388824-kserve-597d77d9c5-8px62" Jul 06 02:05:42.181802 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:42.181765 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7a12cc4-5a6e-49e5-9906-7ec6df3b2032-tls-certs\") pod \"auth-test-model-388824-kserve-597d77d9c5-8px62\" (UID: \"c7a12cc4-5a6e-49e5-9906-7ec6df3b2032\") " pod="ai-tenant-e2e-shared-a-388824/auth-test-model-388824-kserve-597d77d9c5-8px62" Jul 06 02:05:42.181943 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:42.181921 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c7a12cc4-5a6e-49e5-9906-7ec6df3b2032-model-cache\") pod \"auth-test-model-388824-kserve-597d77d9c5-8px62\" (UID: \"c7a12cc4-5a6e-49e5-9906-7ec6df3b2032\") " pod="ai-tenant-e2e-shared-a-388824/auth-test-model-388824-kserve-597d77d9c5-8px62" Jul 06 02:05:42.182003 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:42.181933 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c7a12cc4-5a6e-49e5-9906-7ec6df3b2032-home\") pod \"auth-test-model-388824-kserve-597d77d9c5-8px62\" (UID: \"c7a12cc4-5a6e-49e5-9906-7ec6df3b2032\") " pod="ai-tenant-e2e-shared-a-388824/auth-test-model-388824-kserve-597d77d9c5-8px62" Jul 06 02:05:42.182003 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:42.181983 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c7a12cc4-5a6e-49e5-9906-7ec6df3b2032-tmp-dir\") pod \"auth-test-model-388824-kserve-597d77d9c5-8px62\" (UID: \"c7a12cc4-5a6e-49e5-9906-7ec6df3b2032\") " pod="ai-tenant-e2e-shared-a-388824/auth-test-model-388824-kserve-597d77d9c5-8px62" Jul 06 02:05:42.183883 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:42.183849 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c7a12cc4-5a6e-49e5-9906-7ec6df3b2032-dshm\") pod \"auth-test-model-388824-kserve-597d77d9c5-8px62\" (UID: \"c7a12cc4-5a6e-49e5-9906-7ec6df3b2032\") " pod="ai-tenant-e2e-shared-a-388824/auth-test-model-388824-kserve-597d77d9c5-8px62" Jul 06 02:05:42.184190 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:42.184173 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7a12cc4-5a6e-49e5-9906-7ec6df3b2032-tls-certs\") pod \"auth-test-model-388824-kserve-597d77d9c5-8px62\" (UID: \"c7a12cc4-5a6e-49e5-9906-7ec6df3b2032\") " pod="ai-tenant-e2e-shared-a-388824/auth-test-model-388824-kserve-597d77d9c5-8px62" Jul 06 02:05:42.189793 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:42.189770 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9hxl\" (UniqueName: \"kubernetes.io/projected/c7a12cc4-5a6e-49e5-9906-7ec6df3b2032-kube-api-access-c9hxl\") pod \"auth-test-model-388824-kserve-597d77d9c5-8px62\" (UID: \"c7a12cc4-5a6e-49e5-9906-7ec6df3b2032\") " pod="ai-tenant-e2e-shared-a-388824/auth-test-model-388824-kserve-597d77d9c5-8px62" Jul 06 02:05:42.332574 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:42.332477 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-388824/auth-test-model-388824-kserve-597d77d9c5-8px62" Jul 06 02:05:42.467399 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:42.467346 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-388824/auth-test-model-388824-kserve-597d77d9c5-8px62"] Jul 06 02:05:42.470068 ip-10-0-139-44 kubenswrapper[2580]: W0706 02:05:42.470033 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc7a12cc4_5a6e_49e5_9906_7ec6df3b2032.slice/crio-f66539d865ab1a8ebd775c382dfbef9ab3d776f244a59c280dc49ff7588dbaed WatchSource:0}: Error finding container f66539d865ab1a8ebd775c382dfbef9ab3d776f244a59c280dc49ff7588dbaed: Status 404 returned error can't find the container with id f66539d865ab1a8ebd775c382dfbef9ab3d776f244a59c280dc49ff7588dbaed Jul 06 02:05:42.802147 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:42.802109 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-388824/auth-test-model-388824-kserve-597d77d9c5-8px62" event={"ID":"c7a12cc4-5a6e-49e5-9906-7ec6df3b2032","Type":"ContainerStarted","Data":"d66564a97f4b888c0890c0d36889facda5793dae6b527116e82b352912a12fc4"} Jul 06 02:05:42.802147 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:42.802152 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-388824/auth-test-model-388824-kserve-597d77d9c5-8px62" event={"ID":"c7a12cc4-5a6e-49e5-9906-7ec6df3b2032","Type":"ContainerStarted","Data":"f66539d865ab1a8ebd775c382dfbef9ab3d776f244a59c280dc49ff7588dbaed"} Jul 06 02:05:42.819357 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:42.819303 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-388824/auth-test-model-388824-kserve-597d77d9c5-8px62" podStartSLOduration=1.819286943 podStartE2EDuration="1.819286943s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 02:05:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 02:05:42.817612133 +0000 UTC m=+2336.788863828" watchObservedRunningTime="2026-07-06 02:05:42.819286943 +0000 UTC m=+2336.790538668" Jul 06 02:05:50.961096 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:50.961055 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:05:51.058543 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:51.058501 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:05:52.333409 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:52.333370 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-388824/auth-test-model-388824-kserve-597d77d9c5-8px62" Jul 06 02:05:52.333409 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:52.333413 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-388824/auth-test-model-388824-kserve-597d77d9c5-8px62" Jul 06 02:05:52.345790 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:52.345760 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-388824/auth-test-model-388824-kserve-597d77d9c5-8px62" Jul 06 02:05:52.854855 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:05:52.854817 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-388824/auth-test-model-388824-kserve-597d77d9c5-8px62" Jul 06 02:06:06.218048 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:06:06.218010 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:06:24.723152 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:06:24.723122 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-3d280d/auth-test-model-3d280d-kserve-94c8fd55-55cnx"] Jul 06 02:06:24.726296 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:06:24.726279 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3d280d/auth-test-model-3d280d-kserve-94c8fd55-55cnx" Jul 06 02:06:24.729908 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:06:24.729877 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-3d280d\"/\"default-dockercfg-f62w6\"" Jul 06 02:06:24.730047 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:06:24.729911 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-3d280d\"/\"auth-test-model-3d280d-kserve-self-signed-certs\"" Jul 06 02:06:24.730047 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:06:24.729880 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-3d280d\"/\"kube-root-ca.crt\"" Jul 06 02:06:24.730047 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:06:24.729940 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-3d280d\"/\"openshift-service-ca.crt\"" Jul 06 02:06:24.735983 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:06:24.735948 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-3d280d/auth-test-model-3d280d-kserve-94c8fd55-55cnx"] Jul 06 02:06:24.746697 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:06:24.746666 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:06:24.759865 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:06:24.759834 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5d503655-a046-48ac-bf03-d08d923af672-dshm\") pod \"auth-test-model-3d280d-kserve-94c8fd55-55cnx\" (UID: \"5d503655-a046-48ac-bf03-d08d923af672\") " pod="ai-tenant-e2e-shared-b-3d280d/auth-test-model-3d280d-kserve-94c8fd55-55cnx" Jul 06 02:06:24.760010 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:06:24.759893 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d503655-a046-48ac-bf03-d08d923af672-tls-certs\") pod \"auth-test-model-3d280d-kserve-94c8fd55-55cnx\" (UID: \"5d503655-a046-48ac-bf03-d08d923af672\") " pod="ai-tenant-e2e-shared-b-3d280d/auth-test-model-3d280d-kserve-94c8fd55-55cnx" Jul 06 02:06:24.760010 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:06:24.759947 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5d503655-a046-48ac-bf03-d08d923af672-model-cache\") pod \"auth-test-model-3d280d-kserve-94c8fd55-55cnx\" (UID: \"5d503655-a046-48ac-bf03-d08d923af672\") " pod="ai-tenant-e2e-shared-b-3d280d/auth-test-model-3d280d-kserve-94c8fd55-55cnx" Jul 06 02:06:24.760096 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:06:24.760035 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5d503655-a046-48ac-bf03-d08d923af672-home\") pod \"auth-test-model-3d280d-kserve-94c8fd55-55cnx\" (UID: \"5d503655-a046-48ac-bf03-d08d923af672\") " pod="ai-tenant-e2e-shared-b-3d280d/auth-test-model-3d280d-kserve-94c8fd55-55cnx" Jul 06 02:06:24.760096 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:06:24.760073 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d503655-a046-48ac-bf03-d08d923af672-tmp-dir\") pod \"auth-test-model-3d280d-kserve-94c8fd55-55cnx\" (UID: \"5d503655-a046-48ac-bf03-d08d923af672\") " pod="ai-tenant-e2e-shared-b-3d280d/auth-test-model-3d280d-kserve-94c8fd55-55cnx" Jul 06 02:06:24.760096 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:06:24.760091 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcqvw\" (UniqueName: \"kubernetes.io/projected/5d503655-a046-48ac-bf03-d08d923af672-kube-api-access-bcqvw\") pod \"auth-test-model-3d280d-kserve-94c8fd55-55cnx\" (UID: \"5d503655-a046-48ac-bf03-d08d923af672\") " pod="ai-tenant-e2e-shared-b-3d280d/auth-test-model-3d280d-kserve-94c8fd55-55cnx" Jul 06 02:06:24.860747 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:06:24.860702 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5d503655-a046-48ac-bf03-d08d923af672-home\") pod \"auth-test-model-3d280d-kserve-94c8fd55-55cnx\" (UID: \"5d503655-a046-48ac-bf03-d08d923af672\") " pod="ai-tenant-e2e-shared-b-3d280d/auth-test-model-3d280d-kserve-94c8fd55-55cnx" Jul 06 02:06:24.860747 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:06:24.860749 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d503655-a046-48ac-bf03-d08d923af672-tmp-dir\") pod \"auth-test-model-3d280d-kserve-94c8fd55-55cnx\" (UID: \"5d503655-a046-48ac-bf03-d08d923af672\") " pod="ai-tenant-e2e-shared-b-3d280d/auth-test-model-3d280d-kserve-94c8fd55-55cnx" Jul 06 02:06:24.861023 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:06:24.860775 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcqvw\" (UniqueName: \"kubernetes.io/projected/5d503655-a046-48ac-bf03-d08d923af672-kube-api-access-bcqvw\") pod \"auth-test-model-3d280d-kserve-94c8fd55-55cnx\" (UID: \"5d503655-a046-48ac-bf03-d08d923af672\") " pod="ai-tenant-e2e-shared-b-3d280d/auth-test-model-3d280d-kserve-94c8fd55-55cnx" Jul 06 02:06:24.861023 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:06:24.860823 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5d503655-a046-48ac-bf03-d08d923af672-dshm\") pod \"auth-test-model-3d280d-kserve-94c8fd55-55cnx\" (UID: \"5d503655-a046-48ac-bf03-d08d923af672\") " pod="ai-tenant-e2e-shared-b-3d280d/auth-test-model-3d280d-kserve-94c8fd55-55cnx" Jul 06 02:06:24.861023 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:06:24.860867 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d503655-a046-48ac-bf03-d08d923af672-tls-certs\") pod \"auth-test-model-3d280d-kserve-94c8fd55-55cnx\" (UID: \"5d503655-a046-48ac-bf03-d08d923af672\") " pod="ai-tenant-e2e-shared-b-3d280d/auth-test-model-3d280d-kserve-94c8fd55-55cnx" Jul 06 02:06:24.861023 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:06:24.860901 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5d503655-a046-48ac-bf03-d08d923af672-model-cache\") pod \"auth-test-model-3d280d-kserve-94c8fd55-55cnx\" (UID: \"5d503655-a046-48ac-bf03-d08d923af672\") " pod="ai-tenant-e2e-shared-b-3d280d/auth-test-model-3d280d-kserve-94c8fd55-55cnx" Jul 06 02:06:24.861236 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:06:24.861209 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d503655-a046-48ac-bf03-d08d923af672-tmp-dir\") pod \"auth-test-model-3d280d-kserve-94c8fd55-55cnx\" (UID: \"5d503655-a046-48ac-bf03-d08d923af672\") " pod="ai-tenant-e2e-shared-b-3d280d/auth-test-model-3d280d-kserve-94c8fd55-55cnx" Jul 06 02:06:24.861280 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:06:24.861228 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5d503655-a046-48ac-bf03-d08d923af672-home\") pod \"auth-test-model-3d280d-kserve-94c8fd55-55cnx\" (UID: \"5d503655-a046-48ac-bf03-d08d923af672\") " pod="ai-tenant-e2e-shared-b-3d280d/auth-test-model-3d280d-kserve-94c8fd55-55cnx" Jul 06 02:06:24.861315 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:06:24.861282 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5d503655-a046-48ac-bf03-d08d923af672-model-cache\") pod \"auth-test-model-3d280d-kserve-94c8fd55-55cnx\" (UID: \"5d503655-a046-48ac-bf03-d08d923af672\") " pod="ai-tenant-e2e-shared-b-3d280d/auth-test-model-3d280d-kserve-94c8fd55-55cnx" Jul 06 02:06:24.863429 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:06:24.863404 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5d503655-a046-48ac-bf03-d08d923af672-dshm\") pod \"auth-test-model-3d280d-kserve-94c8fd55-55cnx\" (UID: \"5d503655-a046-48ac-bf03-d08d923af672\") " pod="ai-tenant-e2e-shared-b-3d280d/auth-test-model-3d280d-kserve-94c8fd55-55cnx" Jul 06 02:06:24.863689 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:06:24.863669 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d503655-a046-48ac-bf03-d08d923af672-tls-certs\") pod \"auth-test-model-3d280d-kserve-94c8fd55-55cnx\" (UID: \"5d503655-a046-48ac-bf03-d08d923af672\") " pod="ai-tenant-e2e-shared-b-3d280d/auth-test-model-3d280d-kserve-94c8fd55-55cnx" Jul 06 02:06:24.869050 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:06:24.869018 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcqvw\" (UniqueName: \"kubernetes.io/projected/5d503655-a046-48ac-bf03-d08d923af672-kube-api-access-bcqvw\") pod \"auth-test-model-3d280d-kserve-94c8fd55-55cnx\" (UID: \"5d503655-a046-48ac-bf03-d08d923af672\") " pod="ai-tenant-e2e-shared-b-3d280d/auth-test-model-3d280d-kserve-94c8fd55-55cnx" Jul 06 02:06:25.037775 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:06:25.037679 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3d280d/auth-test-model-3d280d-kserve-94c8fd55-55cnx" Jul 06 02:06:25.171160 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:06:25.171114 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-3d280d/auth-test-model-3d280d-kserve-94c8fd55-55cnx"] Jul 06 02:06:25.174216 ip-10-0-139-44 kubenswrapper[2580]: W0706 02:06:25.174178 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5d503655_a046_48ac_bf03_d08d923af672.slice/crio-6ca180fad29d218cd395f2496aff77bc0cecfc7a0a41dc175869adcf41ef1266 WatchSource:0}: Error finding container 6ca180fad29d218cd395f2496aff77bc0cecfc7a0a41dc175869adcf41ef1266: Status 404 returned error can't find the container with id 6ca180fad29d218cd395f2496aff77bc0cecfc7a0a41dc175869adcf41ef1266 Jul 06 02:06:25.981936 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:06:25.981902 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3d280d/auth-test-model-3d280d-kserve-94c8fd55-55cnx" event={"ID":"5d503655-a046-48ac-bf03-d08d923af672","Type":"ContainerStarted","Data":"dfbaf97e35cacb59380fe7405ff197bb95d179885aec225af31ad2e96b251ab8"} Jul 06 02:06:25.981936 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:06:25.981945 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3d280d/auth-test-model-3d280d-kserve-94c8fd55-55cnx" event={"ID":"5d503655-a046-48ac-bf03-d08d923af672","Type":"ContainerStarted","Data":"6ca180fad29d218cd395f2496aff77bc0cecfc7a0a41dc175869adcf41ef1266"} Jul 06 02:06:25.999364 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:06:25.999309 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-3d280d/auth-test-model-3d280d-kserve-94c8fd55-55cnx" podStartSLOduration=1.999292236 podStartE2EDuration="1.999292236s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 02:06:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 02:06:25.998345704 +0000 UTC m=+2379.969597396" watchObservedRunningTime="2026-07-06 02:06:25.999292236 +0000 UTC m=+2379.970543928" Jul 06 02:06:35.038598 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:06:35.038562 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-3d280d/auth-test-model-3d280d-kserve-94c8fd55-55cnx" Jul 06 02:06:35.038999 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:06:35.038846 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-3d280d/auth-test-model-3d280d-kserve-94c8fd55-55cnx" Jul 06 02:06:35.051171 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:06:35.051141 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-3d280d/auth-test-model-3d280d-kserve-94c8fd55-55cnx" Jul 06 02:06:36.036252 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:06:36.036223 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-3d280d/auth-test-model-3d280d-kserve-94c8fd55-55cnx" Jul 06 02:08:08.943991 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:08.943951 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:08:10.119109 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:10.119074 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:08:23.531782 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:23.531745 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-388824/auth-test-model-388824-kserve-597d77d9c5-8px62"] Jul 06 02:08:23.551473 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:23.551420 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:08:24.493987 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:24.493946 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-3d280d/auth-test-model-3d280d-kserve-94c8fd55-55cnx"] Jul 06 02:08:24.516286 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:24.516255 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:08:38.552947 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:38.552854 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-388824/auth-test-model-388824-kserve-597d77d9c5-8px62" podUID="c7a12cc4-5a6e-49e5-9906-7ec6df3b2032" containerName="main" containerID="cri-o://d66564a97f4b888c0890c0d36889facda5793dae6b527116e82b352912a12fc4" gracePeriod=45 Jul 06 02:08:38.805125 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:38.805055 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-388824/auth-test-model-388824-kserve-597d77d9c5-8px62" Jul 06 02:08:38.854988 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:38.854947 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:08:38.947430 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:38.947387 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c9hxl\" (UniqueName: \"kubernetes.io/projected/c7a12cc4-5a6e-49e5-9906-7ec6df3b2032-kube-api-access-c9hxl\") pod \"c7a12cc4-5a6e-49e5-9906-7ec6df3b2032\" (UID: \"c7a12cc4-5a6e-49e5-9906-7ec6df3b2032\") " Jul 06 02:08:38.947688 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:38.947470 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c7a12cc4-5a6e-49e5-9906-7ec6df3b2032-home\") pod \"c7a12cc4-5a6e-49e5-9906-7ec6df3b2032\" (UID: \"c7a12cc4-5a6e-49e5-9906-7ec6df3b2032\") " Jul 06 02:08:38.947688 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:38.947519 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7a12cc4-5a6e-49e5-9906-7ec6df3b2032-tls-certs\") pod \"c7a12cc4-5a6e-49e5-9906-7ec6df3b2032\" (UID: \"c7a12cc4-5a6e-49e5-9906-7ec6df3b2032\") " Jul 06 02:08:38.947688 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:38.947537 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c7a12cc4-5a6e-49e5-9906-7ec6df3b2032-model-cache\") pod \"c7a12cc4-5a6e-49e5-9906-7ec6df3b2032\" (UID: \"c7a12cc4-5a6e-49e5-9906-7ec6df3b2032\") " Jul 06 02:08:38.947688 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:38.947585 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c7a12cc4-5a6e-49e5-9906-7ec6df3b2032-dshm\") pod \"c7a12cc4-5a6e-49e5-9906-7ec6df3b2032\" (UID: \"c7a12cc4-5a6e-49e5-9906-7ec6df3b2032\") " Jul 06 02:08:38.947688 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:38.947613 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c7a12cc4-5a6e-49e5-9906-7ec6df3b2032-tmp-dir\") pod \"c7a12cc4-5a6e-49e5-9906-7ec6df3b2032\" (UID: \"c7a12cc4-5a6e-49e5-9906-7ec6df3b2032\") " Jul 06 02:08:38.947959 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:38.947735 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7a12cc4-5a6e-49e5-9906-7ec6df3b2032-home" (OuterVolumeSpecName: "home") pod "c7a12cc4-5a6e-49e5-9906-7ec6df3b2032" (UID: "c7a12cc4-5a6e-49e5-9906-7ec6df3b2032"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:08:38.947959 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:38.947849 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7a12cc4-5a6e-49e5-9906-7ec6df3b2032-model-cache" (OuterVolumeSpecName: "model-cache") pod "c7a12cc4-5a6e-49e5-9906-7ec6df3b2032" (UID: "c7a12cc4-5a6e-49e5-9906-7ec6df3b2032"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:08:38.947959 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:38.947890 2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c7a12cc4-5a6e-49e5-9906-7ec6df3b2032-home\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 02:08:38.948066 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:38.947981 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7a12cc4-5a6e-49e5-9906-7ec6df3b2032-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c7a12cc4-5a6e-49e5-9906-7ec6df3b2032" (UID: "c7a12cc4-5a6e-49e5-9906-7ec6df3b2032"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:08:38.949967 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:38.949944 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7a12cc4-5a6e-49e5-9906-7ec6df3b2032-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c7a12cc4-5a6e-49e5-9906-7ec6df3b2032" (UID: "c7a12cc4-5a6e-49e5-9906-7ec6df3b2032"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 02:08:38.949967 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:38.949948 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7a12cc4-5a6e-49e5-9906-7ec6df3b2032-dshm" (OuterVolumeSpecName: "dshm") pod "c7a12cc4-5a6e-49e5-9906-7ec6df3b2032" (UID: "c7a12cc4-5a6e-49e5-9906-7ec6df3b2032"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:08:38.950147 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:38.950063 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7a12cc4-5a6e-49e5-9906-7ec6df3b2032-kube-api-access-c9hxl" (OuterVolumeSpecName: "kube-api-access-c9hxl") pod "c7a12cc4-5a6e-49e5-9906-7ec6df3b2032" (UID: "c7a12cc4-5a6e-49e5-9906-7ec6df3b2032"). InnerVolumeSpecName "kube-api-access-c9hxl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 02:08:39.049102 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:39.049060 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c9hxl\" (UniqueName: \"kubernetes.io/projected/c7a12cc4-5a6e-49e5-9906-7ec6df3b2032-kube-api-access-c9hxl\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 02:08:39.049102 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:39.049094 2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7a12cc4-5a6e-49e5-9906-7ec6df3b2032-tls-certs\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 02:08:39.049102 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:39.049105 2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c7a12cc4-5a6e-49e5-9906-7ec6df3b2032-model-cache\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 02:08:39.049102 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:39.049113 2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c7a12cc4-5a6e-49e5-9906-7ec6df3b2032-dshm\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 02:08:39.049370 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:39.049122 2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c7a12cc4-5a6e-49e5-9906-7ec6df3b2032-tmp-dir\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 02:08:39.515526 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:39.515479 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-3d280d/auth-test-model-3d280d-kserve-94c8fd55-55cnx" podUID="5d503655-a046-48ac-bf03-d08d923af672" containerName="main" containerID="cri-o://dfbaf97e35cacb59380fe7405ff197bb95d179885aec225af31ad2e96b251ab8" gracePeriod=45 Jul 06 02:08:39.552063 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:39.552024 2580 generic.go:356] "Generic (PLEG): container finished" podID="c7a12cc4-5a6e-49e5-9906-7ec6df3b2032" containerID="d66564a97f4b888c0890c0d36889facda5793dae6b527116e82b352912a12fc4" exitCode=0 Jul 06 02:08:39.552262 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:39.552074 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-388824/auth-test-model-388824-kserve-597d77d9c5-8px62" Jul 06 02:08:39.552262 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:39.552104 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-388824/auth-test-model-388824-kserve-597d77d9c5-8px62" event={"ID":"c7a12cc4-5a6e-49e5-9906-7ec6df3b2032","Type":"ContainerDied","Data":"d66564a97f4b888c0890c0d36889facda5793dae6b527116e82b352912a12fc4"} Jul 06 02:08:39.552262 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:39.552161 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-388824/auth-test-model-388824-kserve-597d77d9c5-8px62" event={"ID":"c7a12cc4-5a6e-49e5-9906-7ec6df3b2032","Type":"ContainerDied","Data":"f66539d865ab1a8ebd775c382dfbef9ab3d776f244a59c280dc49ff7588dbaed"} Jul 06 02:08:39.552262 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:39.552185 2580 scope.go:117] "RemoveContainer" containerID="d66564a97f4b888c0890c0d36889facda5793dae6b527116e82b352912a12fc4" Jul 06 02:08:39.594165 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:39.594116 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-388824/auth-test-model-388824-kserve-597d77d9c5-8px62"] Jul 06 02:08:39.599732 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:39.599699 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-388824/auth-test-model-388824-kserve-597d77d9c5-8px62"] Jul 06 02:08:39.626409 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:39.626276 2580 scope.go:117] "RemoveContainer" containerID="d66564a97f4b888c0890c0d36889facda5793dae6b527116e82b352912a12fc4" Jul 06 02:08:39.626666 ip-10-0-139-44 kubenswrapper[2580]: E0706 02:08:39.626645 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d66564a97f4b888c0890c0d36889facda5793dae6b527116e82b352912a12fc4\": container with ID starting with d66564a97f4b888c0890c0d36889facda5793dae6b527116e82b352912a12fc4 not found: ID does not exist" containerID="d66564a97f4b888c0890c0d36889facda5793dae6b527116e82b352912a12fc4" Jul 06 02:08:39.626718 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:39.626679 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d66564a97f4b888c0890c0d36889facda5793dae6b527116e82b352912a12fc4"} err="failed to get container status \"d66564a97f4b888c0890c0d36889facda5793dae6b527116e82b352912a12fc4\": rpc error: code = NotFound desc = could not find container \"d66564a97f4b888c0890c0d36889facda5793dae6b527116e82b352912a12fc4\": container with ID starting with d66564a97f4b888c0890c0d36889facda5793dae6b527116e82b352912a12fc4 not found: ID does not exist" Jul 06 02:08:39.763832 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:39.763795 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3d280d/auth-test-model-3d280d-kserve-94c8fd55-55cnx" Jul 06 02:08:39.856286 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:39.856193 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5d503655-a046-48ac-bf03-d08d923af672-model-cache\") pod \"5d503655-a046-48ac-bf03-d08d923af672\" (UID: \"5d503655-a046-48ac-bf03-d08d923af672\") " Jul 06 02:08:39.856286 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:39.856239 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5d503655-a046-48ac-bf03-d08d923af672-dshm\") pod \"5d503655-a046-48ac-bf03-d08d923af672\" (UID: \"5d503655-a046-48ac-bf03-d08d923af672\") " Jul 06 02:08:39.856286 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:39.856283 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d503655-a046-48ac-bf03-d08d923af672-tmp-dir\") pod \"5d503655-a046-48ac-bf03-d08d923af672\" (UID: \"5d503655-a046-48ac-bf03-d08d923af672\") " Jul 06 02:08:39.856615 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:39.856316 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bcqvw\" (UniqueName: \"kubernetes.io/projected/5d503655-a046-48ac-bf03-d08d923af672-kube-api-access-bcqvw\") pod \"5d503655-a046-48ac-bf03-d08d923af672\" (UID: \"5d503655-a046-48ac-bf03-d08d923af672\") " Jul 06 02:08:39.856615 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:39.856372 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d503655-a046-48ac-bf03-d08d923af672-tls-certs\") pod \"5d503655-a046-48ac-bf03-d08d923af672\" (UID: \"5d503655-a046-48ac-bf03-d08d923af672\") " Jul 06 02:08:39.856615 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:39.856483 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5d503655-a046-48ac-bf03-d08d923af672-home\") pod \"5d503655-a046-48ac-bf03-d08d923af672\" (UID: \"5d503655-a046-48ac-bf03-d08d923af672\") " Jul 06 02:08:39.856615 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:39.856565 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d503655-a046-48ac-bf03-d08d923af672-model-cache" (OuterVolumeSpecName: "model-cache") pod "5d503655-a046-48ac-bf03-d08d923af672" (UID: "5d503655-a046-48ac-bf03-d08d923af672"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:08:39.856811 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:39.856627 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d503655-a046-48ac-bf03-d08d923af672-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5d503655-a046-48ac-bf03-d08d923af672" (UID: "5d503655-a046-48ac-bf03-d08d923af672"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:08:39.856811 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:39.856760 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d503655-a046-48ac-bf03-d08d923af672-home" (OuterVolumeSpecName: "home") pod "5d503655-a046-48ac-bf03-d08d923af672" (UID: "5d503655-a046-48ac-bf03-d08d923af672"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:08:39.856811 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:39.856801 2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5d503655-a046-48ac-bf03-d08d923af672-model-cache\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 02:08:39.856917 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:39.856815 2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d503655-a046-48ac-bf03-d08d923af672-tmp-dir\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 02:08:39.858843 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:39.858813 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d503655-a046-48ac-bf03-d08d923af672-dshm" (OuterVolumeSpecName: "dshm") pod "5d503655-a046-48ac-bf03-d08d923af672" (UID: "5d503655-a046-48ac-bf03-d08d923af672"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:08:39.858843 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:39.858839 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d503655-a046-48ac-bf03-d08d923af672-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5d503655-a046-48ac-bf03-d08d923af672" (UID: "5d503655-a046-48ac-bf03-d08d923af672"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 02:08:39.858992 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:39.858897 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d503655-a046-48ac-bf03-d08d923af672-kube-api-access-bcqvw" (OuterVolumeSpecName: "kube-api-access-bcqvw") pod "5d503655-a046-48ac-bf03-d08d923af672" (UID: "5d503655-a046-48ac-bf03-d08d923af672"). InnerVolumeSpecName "kube-api-access-bcqvw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 02:08:39.957542 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:39.957503 2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d503655-a046-48ac-bf03-d08d923af672-tls-certs\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 02:08:39.957542 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:39.957536 2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5d503655-a046-48ac-bf03-d08d923af672-home\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 02:08:39.957542 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:39.957545 2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5d503655-a046-48ac-bf03-d08d923af672-dshm\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 02:08:39.957796 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:39.957557 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bcqvw\" (UniqueName: \"kubernetes.io/projected/5d503655-a046-48ac-bf03-d08d923af672-kube-api-access-bcqvw\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 02:08:40.459526 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:40.459490 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7a12cc4-5a6e-49e5-9906-7ec6df3b2032" path="/var/lib/kubelet/pods/c7a12cc4-5a6e-49e5-9906-7ec6df3b2032/volumes" Jul 06 02:08:40.558349 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:40.558308 2580 generic.go:356] "Generic (PLEG): container finished" podID="5d503655-a046-48ac-bf03-d08d923af672" containerID="dfbaf97e35cacb59380fe7405ff197bb95d179885aec225af31ad2e96b251ab8" exitCode=0 Jul 06 02:08:40.558569 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:40.558356 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3d280d/auth-test-model-3d280d-kserve-94c8fd55-55cnx" Jul 06 02:08:40.558569 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:40.558358 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3d280d/auth-test-model-3d280d-kserve-94c8fd55-55cnx" event={"ID":"5d503655-a046-48ac-bf03-d08d923af672","Type":"ContainerDied","Data":"dfbaf97e35cacb59380fe7405ff197bb95d179885aec225af31ad2e96b251ab8"} Jul 06 02:08:40.558569 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:40.558482 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3d280d/auth-test-model-3d280d-kserve-94c8fd55-55cnx" event={"ID":"5d503655-a046-48ac-bf03-d08d923af672","Type":"ContainerDied","Data":"6ca180fad29d218cd395f2496aff77bc0cecfc7a0a41dc175869adcf41ef1266"} Jul 06 02:08:40.558569 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:40.558509 2580 scope.go:117] "RemoveContainer" containerID="dfbaf97e35cacb59380fe7405ff197bb95d179885aec225af31ad2e96b251ab8" Jul 06 02:08:40.566766 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:40.566745 2580 scope.go:117] "RemoveContainer" containerID="dfbaf97e35cacb59380fe7405ff197bb95d179885aec225af31ad2e96b251ab8" Jul 06 02:08:40.567088 ip-10-0-139-44 kubenswrapper[2580]: E0706 02:08:40.567068 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dfbaf97e35cacb59380fe7405ff197bb95d179885aec225af31ad2e96b251ab8\": container with ID starting with dfbaf97e35cacb59380fe7405ff197bb95d179885aec225af31ad2e96b251ab8 not found: ID does not exist" containerID="dfbaf97e35cacb59380fe7405ff197bb95d179885aec225af31ad2e96b251ab8" Jul 06 02:08:40.567141 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:40.567097 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfbaf97e35cacb59380fe7405ff197bb95d179885aec225af31ad2e96b251ab8"} err="failed to get container status \"dfbaf97e35cacb59380fe7405ff197bb95d179885aec225af31ad2e96b251ab8\": rpc error: code = NotFound desc = could not find container \"dfbaf97e35cacb59380fe7405ff197bb95d179885aec225af31ad2e96b251ab8\": container with ID starting with dfbaf97e35cacb59380fe7405ff197bb95d179885aec225af31ad2e96b251ab8 not found: ID does not exist" Jul 06 02:08:40.575508 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:40.575480 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-3d280d/auth-test-model-3d280d-kserve-94c8fd55-55cnx"] Jul 06 02:08:40.579664 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:40.579637 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-3d280d/auth-test-model-3d280d-kserve-94c8fd55-55cnx"] Jul 06 02:08:42.459586 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:08:42.459552 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d503655-a046-48ac-bf03-d08d923af672" path="/var/lib/kubelet/pods/5d503655-a046-48ac-bf03-d08d923af672/volumes" Jul 06 02:09:10.822137 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:10.822102 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-3d280d/sub-test-model-3d280d-kserve-7f5cb9f68f-tbzkh"] Jul 06 02:09:10.822620 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:10.822414 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d503655-a046-48ac-bf03-d08d923af672" containerName="main" Jul 06 02:09:10.822620 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:10.822428 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d503655-a046-48ac-bf03-d08d923af672" containerName="main" Jul 06 02:09:10.822620 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:10.822457 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7a12cc4-5a6e-49e5-9906-7ec6df3b2032" containerName="main" Jul 06 02:09:10.822620 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:10.822463 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7a12cc4-5a6e-49e5-9906-7ec6df3b2032" containerName="main" Jul 06 02:09:10.822620 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:10.822510 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7a12cc4-5a6e-49e5-9906-7ec6df3b2032" containerName="main" Jul 06 02:09:10.822620 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:10.822517 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d503655-a046-48ac-bf03-d08d923af672" containerName="main" Jul 06 02:09:10.825053 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:10.825035 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3d280d/sub-test-model-3d280d-kserve-7f5cb9f68f-tbzkh" Jul 06 02:09:10.827748 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:10.827720 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-3d280d\"/\"openshift-service-ca.crt\"" Jul 06 02:09:10.828958 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:10.828937 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-3d280d\"/\"kube-root-ca.crt\"" Jul 06 02:09:10.829076 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:10.828936 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-3d280d\"/\"sub-test-model-3d280d-kserve-self-signed-certs\"" Jul 06 02:09:10.829076 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:10.828975 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-3d280d\"/\"default-dockercfg-f62w6\"" Jul 06 02:09:10.834859 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:10.834833 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-3d280d/sub-test-model-3d280d-kserve-7f5cb9f68f-tbzkh"] Jul 06 02:09:10.852979 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:10.852947 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:09:10.919105 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:10.919064 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d-model-cache\") pod \"sub-test-model-3d280d-kserve-7f5cb9f68f-tbzkh\" (UID: \"d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d\") " pod="ai-tenant-e2e-shared-b-3d280d/sub-test-model-3d280d-kserve-7f5cb9f68f-tbzkh" Jul 06 02:09:10.919105 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:10.919110 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d-tmp-dir\") pod \"sub-test-model-3d280d-kserve-7f5cb9f68f-tbzkh\" (UID: \"d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d\") " pod="ai-tenant-e2e-shared-b-3d280d/sub-test-model-3d280d-kserve-7f5cb9f68f-tbzkh" Jul 06 02:09:10.919323 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:10.919134 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d-tls-certs\") pod \"sub-test-model-3d280d-kserve-7f5cb9f68f-tbzkh\" (UID: \"d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d\") " pod="ai-tenant-e2e-shared-b-3d280d/sub-test-model-3d280d-kserve-7f5cb9f68f-tbzkh" Jul 06 02:09:10.919323 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:10.919235 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgwps\" (UniqueName: \"kubernetes.io/projected/d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d-kube-api-access-rgwps\") pod \"sub-test-model-3d280d-kserve-7f5cb9f68f-tbzkh\" (UID: \"d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d\") " pod="ai-tenant-e2e-shared-b-3d280d/sub-test-model-3d280d-kserve-7f5cb9f68f-tbzkh" Jul 06 02:09:10.919323 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:10.919266 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d-dshm\") pod \"sub-test-model-3d280d-kserve-7f5cb9f68f-tbzkh\" (UID: \"d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d\") " pod="ai-tenant-e2e-shared-b-3d280d/sub-test-model-3d280d-kserve-7f5cb9f68f-tbzkh" Jul 06 02:09:10.919323 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:10.919290 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d-home\") pod \"sub-test-model-3d280d-kserve-7f5cb9f68f-tbzkh\" (UID: \"d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d\") " pod="ai-tenant-e2e-shared-b-3d280d/sub-test-model-3d280d-kserve-7f5cb9f68f-tbzkh" Jul 06 02:09:11.020621 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:11.020580 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgwps\" (UniqueName: \"kubernetes.io/projected/d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d-kube-api-access-rgwps\") pod \"sub-test-model-3d280d-kserve-7f5cb9f68f-tbzkh\" (UID: \"d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d\") " pod="ai-tenant-e2e-shared-b-3d280d/sub-test-model-3d280d-kserve-7f5cb9f68f-tbzkh" Jul 06 02:09:11.020621 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:11.020618 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d-dshm\") pod \"sub-test-model-3d280d-kserve-7f5cb9f68f-tbzkh\" (UID: \"d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d\") " pod="ai-tenant-e2e-shared-b-3d280d/sub-test-model-3d280d-kserve-7f5cb9f68f-tbzkh" Jul 06 02:09:11.020888 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:11.020640 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d-home\") pod \"sub-test-model-3d280d-kserve-7f5cb9f68f-tbzkh\" (UID: \"d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d\") " pod="ai-tenant-e2e-shared-b-3d280d/sub-test-model-3d280d-kserve-7f5cb9f68f-tbzkh" Jul 06 02:09:11.020888 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:11.020695 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d-model-cache\") pod \"sub-test-model-3d280d-kserve-7f5cb9f68f-tbzkh\" (UID: \"d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d\") " pod="ai-tenant-e2e-shared-b-3d280d/sub-test-model-3d280d-kserve-7f5cb9f68f-tbzkh" Jul 06 02:09:11.020888 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:11.020728 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d-tmp-dir\") pod \"sub-test-model-3d280d-kserve-7f5cb9f68f-tbzkh\" (UID: \"d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d\") " pod="ai-tenant-e2e-shared-b-3d280d/sub-test-model-3d280d-kserve-7f5cb9f68f-tbzkh" Jul 06 02:09:11.020888 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:11.020746 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d-tls-certs\") pod \"sub-test-model-3d280d-kserve-7f5cb9f68f-tbzkh\" (UID: \"d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d\") " pod="ai-tenant-e2e-shared-b-3d280d/sub-test-model-3d280d-kserve-7f5cb9f68f-tbzkh" Jul 06 02:09:11.021175 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:11.021141 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d-home\") pod \"sub-test-model-3d280d-kserve-7f5cb9f68f-tbzkh\" (UID: \"d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d\") " pod="ai-tenant-e2e-shared-b-3d280d/sub-test-model-3d280d-kserve-7f5cb9f68f-tbzkh" Jul 06 02:09:11.021175 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:11.021160 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d-tmp-dir\") pod \"sub-test-model-3d280d-kserve-7f5cb9f68f-tbzkh\" (UID: \"d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d\") " pod="ai-tenant-e2e-shared-b-3d280d/sub-test-model-3d280d-kserve-7f5cb9f68f-tbzkh" Jul 06 02:09:11.021319 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:11.021213 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d-model-cache\") pod \"sub-test-model-3d280d-kserve-7f5cb9f68f-tbzkh\" (UID: \"d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d\") " pod="ai-tenant-e2e-shared-b-3d280d/sub-test-model-3d280d-kserve-7f5cb9f68f-tbzkh" Jul 06 02:09:11.023117 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:11.023097 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d-dshm\") pod \"sub-test-model-3d280d-kserve-7f5cb9f68f-tbzkh\" (UID: \"d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d\") " pod="ai-tenant-e2e-shared-b-3d280d/sub-test-model-3d280d-kserve-7f5cb9f68f-tbzkh" Jul 06 02:09:11.023385 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:11.023368 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d-tls-certs\") pod \"sub-test-model-3d280d-kserve-7f5cb9f68f-tbzkh\" (UID: \"d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d\") " pod="ai-tenant-e2e-shared-b-3d280d/sub-test-model-3d280d-kserve-7f5cb9f68f-tbzkh" Jul 06 02:09:11.028836 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:11.028814 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgwps\" (UniqueName: \"kubernetes.io/projected/d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d-kube-api-access-rgwps\") pod \"sub-test-model-3d280d-kserve-7f5cb9f68f-tbzkh\" (UID: \"d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d\") " pod="ai-tenant-e2e-shared-b-3d280d/sub-test-model-3d280d-kserve-7f5cb9f68f-tbzkh" Jul 06 02:09:11.135093 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:11.135007 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3d280d/sub-test-model-3d280d-kserve-7f5cb9f68f-tbzkh" Jul 06 02:09:11.277056 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:11.277019 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-3d280d/sub-test-model-3d280d-kserve-7f5cb9f68f-tbzkh"] Jul 06 02:09:11.281265 ip-10-0-139-44 kubenswrapper[2580]: W0706 02:09:11.281235 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd32654a1_ac2a_4d2a_af35_bfe8c1a20c9d.slice/crio-7d5be9784b1df6a344b046f6fbfbceaae8f6142240ad200d2692c8f18726d7c4 WatchSource:0}: Error finding container 7d5be9784b1df6a344b046f6fbfbceaae8f6142240ad200d2692c8f18726d7c4: Status 404 returned error can't find the container with id 7d5be9784b1df6a344b046f6fbfbceaae8f6142240ad200d2692c8f18726d7c4 Jul 06 02:09:11.691802 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:11.691766 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3d280d/sub-test-model-3d280d-kserve-7f5cb9f68f-tbzkh" event={"ID":"d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d","Type":"ContainerStarted","Data":"49fc1c00278b8008c8ede617613fab61b8d678cea4047a87d50ead995ce7e0dd"} Jul 06 02:09:11.691802 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:11.691807 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3d280d/sub-test-model-3d280d-kserve-7f5cb9f68f-tbzkh" event={"ID":"d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d","Type":"ContainerStarted","Data":"7d5be9784b1df6a344b046f6fbfbceaae8f6142240ad200d2692c8f18726d7c4"} Jul 06 02:09:11.709744 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:11.709683 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-3d280d/sub-test-model-3d280d-kserve-7f5cb9f68f-tbzkh" podStartSLOduration=1.709665449 podStartE2EDuration="1.709665449s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 02:09:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 02:09:11.707382026 +0000 UTC m=+2545.678633715" watchObservedRunningTime="2026-07-06 02:09:11.709665449 +0000 UTC m=+2545.680917139" Jul 06 02:09:21.136090 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:21.136048 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-3d280d/sub-test-model-3d280d-kserve-7f5cb9f68f-tbzkh" Jul 06 02:09:21.136644 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:21.136120 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-3d280d/sub-test-model-3d280d-kserve-7f5cb9f68f-tbzkh" Jul 06 02:09:21.148836 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:21.148808 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-3d280d/sub-test-model-3d280d-kserve-7f5cb9f68f-tbzkh" Jul 06 02:09:21.744608 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:21.744578 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-3d280d/sub-test-model-3d280d-kserve-7f5cb9f68f-tbzkh" Jul 06 02:09:34.149825 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:34.149784 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:09:43.438563 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:43.438527 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:09:44.504758 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:44.504710 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-3d280d/sub-test-model-3d280d-kserve-7f5cb9f68f-tbzkh"] Jul 06 02:09:44.528415 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:44.528375 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:09:44.901031 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:44.900990 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:09:45.001224 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:45.001183 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:09:46.615515 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:46.615488 2580 scope.go:117] "RemoveContainer" containerID="433df6d675702fd144ce35d46970e60eec8b98d5640d8fca1289be6fdabf4a55" Jul 06 02:09:48.323769 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:48.323721 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:09:53.722780 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:53.722744 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-388824/rate-test-model-388824-kserve-f456bd85f-4xzk7"] Jul 06 02:09:53.726008 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:53.725991 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-388824/rate-test-model-388824-kserve-f456bd85f-4xzk7" Jul 06 02:09:53.730106 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:53.730072 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-388824\"/\"openshift-service-ca.crt\"" Jul 06 02:09:53.730285 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:53.730243 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-388824\"/\"default-dockercfg-d8pxv\"" Jul 06 02:09:53.730285 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:53.730251 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-388824\"/\"rate-test-model-388824-kserve-self-signed-certs\"" Jul 06 02:09:53.730285 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:53.730243 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-388824\"/\"kube-root-ca.crt\"" Jul 06 02:09:53.735501 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:53.735475 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-388824/rate-test-model-388824-kserve-f456bd85f-4xzk7"] Jul 06 02:09:53.749221 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:53.749165 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:09:53.786080 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:53.786039 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/426b2504-0e69-40a4-9f4f-2c1fa76230db-tmp-dir\") pod \"rate-test-model-388824-kserve-f456bd85f-4xzk7\" (UID: \"426b2504-0e69-40a4-9f4f-2c1fa76230db\") " pod="ai-tenant-e2e-shared-a-388824/rate-test-model-388824-kserve-f456bd85f-4xzk7" Jul 06 02:09:53.786277 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:53.786091 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/426b2504-0e69-40a4-9f4f-2c1fa76230db-dshm\") pod \"rate-test-model-388824-kserve-f456bd85f-4xzk7\" (UID: \"426b2504-0e69-40a4-9f4f-2c1fa76230db\") " pod="ai-tenant-e2e-shared-a-388824/rate-test-model-388824-kserve-f456bd85f-4xzk7" Jul 06 02:09:53.786277 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:53.786170 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/426b2504-0e69-40a4-9f4f-2c1fa76230db-home\") pod \"rate-test-model-388824-kserve-f456bd85f-4xzk7\" (UID: \"426b2504-0e69-40a4-9f4f-2c1fa76230db\") " pod="ai-tenant-e2e-shared-a-388824/rate-test-model-388824-kserve-f456bd85f-4xzk7" Jul 06 02:09:53.786277 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:53.786221 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/426b2504-0e69-40a4-9f4f-2c1fa76230db-tls-certs\") pod \"rate-test-model-388824-kserve-f456bd85f-4xzk7\" (UID: \"426b2504-0e69-40a4-9f4f-2c1fa76230db\") " pod="ai-tenant-e2e-shared-a-388824/rate-test-model-388824-kserve-f456bd85f-4xzk7" Jul 06 02:09:53.786277 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:53.786262 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/426b2504-0e69-40a4-9f4f-2c1fa76230db-model-cache\") pod \"rate-test-model-388824-kserve-f456bd85f-4xzk7\" (UID: \"426b2504-0e69-40a4-9f4f-2c1fa76230db\") " pod="ai-tenant-e2e-shared-a-388824/rate-test-model-388824-kserve-f456bd85f-4xzk7" Jul 06 02:09:53.786486 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:53.786293 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skzkz\" (UniqueName: \"kubernetes.io/projected/426b2504-0e69-40a4-9f4f-2c1fa76230db-kube-api-access-skzkz\") pod \"rate-test-model-388824-kserve-f456bd85f-4xzk7\" (UID: \"426b2504-0e69-40a4-9f4f-2c1fa76230db\") " pod="ai-tenant-e2e-shared-a-388824/rate-test-model-388824-kserve-f456bd85f-4xzk7" Jul 06 02:09:53.886999 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:53.886958 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/426b2504-0e69-40a4-9f4f-2c1fa76230db-tmp-dir\") pod \"rate-test-model-388824-kserve-f456bd85f-4xzk7\" (UID: \"426b2504-0e69-40a4-9f4f-2c1fa76230db\") " pod="ai-tenant-e2e-shared-a-388824/rate-test-model-388824-kserve-f456bd85f-4xzk7" Jul 06 02:09:53.886999 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:53.887010 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/426b2504-0e69-40a4-9f4f-2c1fa76230db-dshm\") pod \"rate-test-model-388824-kserve-f456bd85f-4xzk7\" (UID: \"426b2504-0e69-40a4-9f4f-2c1fa76230db\") " pod="ai-tenant-e2e-shared-a-388824/rate-test-model-388824-kserve-f456bd85f-4xzk7" Jul 06 02:09:53.887255 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:53.887151 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/426b2504-0e69-40a4-9f4f-2c1fa76230db-home\") pod \"rate-test-model-388824-kserve-f456bd85f-4xzk7\" (UID: \"426b2504-0e69-40a4-9f4f-2c1fa76230db\") " pod="ai-tenant-e2e-shared-a-388824/rate-test-model-388824-kserve-f456bd85f-4xzk7" Jul 06 02:09:53.887255 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:53.887198 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/426b2504-0e69-40a4-9f4f-2c1fa76230db-tls-certs\") pod \"rate-test-model-388824-kserve-f456bd85f-4xzk7\" (UID: \"426b2504-0e69-40a4-9f4f-2c1fa76230db\") " pod="ai-tenant-e2e-shared-a-388824/rate-test-model-388824-kserve-f456bd85f-4xzk7" Jul 06 02:09:53.887255 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:53.887239 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/426b2504-0e69-40a4-9f4f-2c1fa76230db-model-cache\") pod \"rate-test-model-388824-kserve-f456bd85f-4xzk7\" (UID: \"426b2504-0e69-40a4-9f4f-2c1fa76230db\") " pod="ai-tenant-e2e-shared-a-388824/rate-test-model-388824-kserve-f456bd85f-4xzk7" Jul 06 02:09:53.887421 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:53.887266 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skzkz\" (UniqueName: \"kubernetes.io/projected/426b2504-0e69-40a4-9f4f-2c1fa76230db-kube-api-access-skzkz\") pod \"rate-test-model-388824-kserve-f456bd85f-4xzk7\" (UID: \"426b2504-0e69-40a4-9f4f-2c1fa76230db\") " pod="ai-tenant-e2e-shared-a-388824/rate-test-model-388824-kserve-f456bd85f-4xzk7" Jul 06 02:09:53.887558 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:53.887532 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/426b2504-0e69-40a4-9f4f-2c1fa76230db-home\") pod \"rate-test-model-388824-kserve-f456bd85f-4xzk7\" (UID: \"426b2504-0e69-40a4-9f4f-2c1fa76230db\") " pod="ai-tenant-e2e-shared-a-388824/rate-test-model-388824-kserve-f456bd85f-4xzk7" Jul 06 02:09:53.887684 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:53.887533 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/426b2504-0e69-40a4-9f4f-2c1fa76230db-tmp-dir\") pod \"rate-test-model-388824-kserve-f456bd85f-4xzk7\" (UID: \"426b2504-0e69-40a4-9f4f-2c1fa76230db\") " pod="ai-tenant-e2e-shared-a-388824/rate-test-model-388824-kserve-f456bd85f-4xzk7" Jul 06 02:09:53.887684 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:53.887629 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/426b2504-0e69-40a4-9f4f-2c1fa76230db-model-cache\") pod \"rate-test-model-388824-kserve-f456bd85f-4xzk7\" (UID: \"426b2504-0e69-40a4-9f4f-2c1fa76230db\") " pod="ai-tenant-e2e-shared-a-388824/rate-test-model-388824-kserve-f456bd85f-4xzk7" Jul 06 02:09:53.889588 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:53.889566 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/426b2504-0e69-40a4-9f4f-2c1fa76230db-dshm\") pod \"rate-test-model-388824-kserve-f456bd85f-4xzk7\" (UID: \"426b2504-0e69-40a4-9f4f-2c1fa76230db\") " pod="ai-tenant-e2e-shared-a-388824/rate-test-model-388824-kserve-f456bd85f-4xzk7" Jul 06 02:09:53.889770 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:53.889745 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/426b2504-0e69-40a4-9f4f-2c1fa76230db-tls-certs\") pod \"rate-test-model-388824-kserve-f456bd85f-4xzk7\" (UID: \"426b2504-0e69-40a4-9f4f-2c1fa76230db\") " pod="ai-tenant-e2e-shared-a-388824/rate-test-model-388824-kserve-f456bd85f-4xzk7" Jul 06 02:09:53.895507 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:53.895480 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-skzkz\" (UniqueName: \"kubernetes.io/projected/426b2504-0e69-40a4-9f4f-2c1fa76230db-kube-api-access-skzkz\") pod \"rate-test-model-388824-kserve-f456bd85f-4xzk7\" (UID: \"426b2504-0e69-40a4-9f4f-2c1fa76230db\") " pod="ai-tenant-e2e-shared-a-388824/rate-test-model-388824-kserve-f456bd85f-4xzk7" Jul 06 02:09:54.035611 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:54.035509 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-388824/rate-test-model-388824-kserve-f456bd85f-4xzk7" Jul 06 02:09:54.173362 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:54.173327 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-388824/rate-test-model-388824-kserve-f456bd85f-4xzk7"] Jul 06 02:09:54.176427 ip-10-0-139-44 kubenswrapper[2580]: W0706 02:09:54.176396 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod426b2504_0e69_40a4_9f4f_2c1fa76230db.slice/crio-7440561748bd9452fecce1a0a56d24e9ec6b76c89fd2febf48cb0a9545032dd5 WatchSource:0}: Error finding container 7440561748bd9452fecce1a0a56d24e9ec6b76c89fd2febf48cb0a9545032dd5: Status 404 returned error can't find the container with id 7440561748bd9452fecce1a0a56d24e9ec6b76c89fd2febf48cb0a9545032dd5 Jul 06 02:09:54.870232 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:54.870190 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-388824/rate-test-model-388824-kserve-f456bd85f-4xzk7" event={"ID":"426b2504-0e69-40a4-9f4f-2c1fa76230db","Type":"ContainerStarted","Data":"bc59e0557bfe107ac928c3f5255932fae5c42f1deab77e373d9626afb167551a"} Jul 06 02:09:54.870232 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:54.870231 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-388824/rate-test-model-388824-kserve-f456bd85f-4xzk7" event={"ID":"426b2504-0e69-40a4-9f4f-2c1fa76230db","Type":"ContainerStarted","Data":"7440561748bd9452fecce1a0a56d24e9ec6b76c89fd2febf48cb0a9545032dd5"} Jul 06 02:09:54.891047 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:54.890990 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-388824/rate-test-model-388824-kserve-f456bd85f-4xzk7" podStartSLOduration=1.890971476 podStartE2EDuration="1.890971476s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 02:09:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 02:09:54.88969152 +0000 UTC m=+2588.860943213" watchObservedRunningTime="2026-07-06 02:09:54.890971476 +0000 UTC m=+2588.862223166" Jul 06 02:09:59.525668 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:59.525606 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-3d280d/sub-test-model-3d280d-kserve-7f5cb9f68f-tbzkh" podUID="d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d" containerName="main" containerID="cri-o://49fc1c00278b8008c8ede617613fab61b8d678cea4047a87d50ead995ce7e0dd" gracePeriod=45 Jul 06 02:09:59.770198 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:59.770171 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3d280d/sub-test-model-3d280d-kserve-7f5cb9f68f-tbzkh" Jul 06 02:09:59.841997 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:59.841905 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d-tmp-dir\") pod \"d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d\" (UID: \"d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d\") " Jul 06 02:09:59.841997 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:59.841948 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d-dshm\") pod \"d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d\" (UID: \"d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d\") " Jul 06 02:09:59.842187 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:59.842005 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d-model-cache\") pod \"d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d\" (UID: \"d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d\") " Jul 06 02:09:59.842187 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:59.842055 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d-tls-certs\") pod \"d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d\" (UID: \"d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d\") " Jul 06 02:09:59.842187 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:59.842090 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rgwps\" (UniqueName: \"kubernetes.io/projected/d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d-kube-api-access-rgwps\") pod \"d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d\" (UID: \"d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d\") " Jul 06 02:09:59.842187 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:59.842125 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d-home\") pod \"d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d\" (UID: \"d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d\") " Jul 06 02:09:59.842358 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:59.842212 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d" (UID: "d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:09:59.842358 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:59.842304 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d-model-cache" (OuterVolumeSpecName: "model-cache") pod "d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d" (UID: "d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:09:59.842493 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:59.842420 2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d-tmp-dir\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 02:09:59.842493 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:59.842431 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d-home" (OuterVolumeSpecName: "home") pod "d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d" (UID: "d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:09:59.842493 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:59.842480 2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d-model-cache\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 02:09:59.844480 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:59.844431 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d-dshm" (OuterVolumeSpecName: "dshm") pod "d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d" (UID: "d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:09:59.844609 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:59.844484 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d-kube-api-access-rgwps" (OuterVolumeSpecName: "kube-api-access-rgwps") pod "d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d" (UID: "d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d"). InnerVolumeSpecName "kube-api-access-rgwps". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 02:09:59.844609 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:59.844505 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d" (UID: "d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 02:09:59.892026 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:59.891996 2580 generic.go:356] "Generic (PLEG): container finished" podID="d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d" containerID="49fc1c00278b8008c8ede617613fab61b8d678cea4047a87d50ead995ce7e0dd" exitCode=0 Jul 06 02:09:59.892213 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:59.892044 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3d280d/sub-test-model-3d280d-kserve-7f5cb9f68f-tbzkh" Jul 06 02:09:59.892213 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:59.892050 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3d280d/sub-test-model-3d280d-kserve-7f5cb9f68f-tbzkh" event={"ID":"d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d","Type":"ContainerDied","Data":"49fc1c00278b8008c8ede617613fab61b8d678cea4047a87d50ead995ce7e0dd"} Jul 06 02:09:59.892213 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:59.892145 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3d280d/sub-test-model-3d280d-kserve-7f5cb9f68f-tbzkh" event={"ID":"d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d","Type":"ContainerDied","Data":"7d5be9784b1df6a344b046f6fbfbceaae8f6142240ad200d2692c8f18726d7c4"} Jul 06 02:09:59.892213 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:59.892162 2580 scope.go:117] "RemoveContainer" containerID="49fc1c00278b8008c8ede617613fab61b8d678cea4047a87d50ead995ce7e0dd" Jul 06 02:09:59.900993 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:59.900975 2580 scope.go:117] "RemoveContainer" containerID="49fc1c00278b8008c8ede617613fab61b8d678cea4047a87d50ead995ce7e0dd" Jul 06 02:09:59.901274 ip-10-0-139-44 kubenswrapper[2580]: E0706 02:09:59.901249 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49fc1c00278b8008c8ede617613fab61b8d678cea4047a87d50ead995ce7e0dd\": container with ID starting with 49fc1c00278b8008c8ede617613fab61b8d678cea4047a87d50ead995ce7e0dd not found: ID does not exist" containerID="49fc1c00278b8008c8ede617613fab61b8d678cea4047a87d50ead995ce7e0dd" Jul 06 02:09:59.901373 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:59.901280 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49fc1c00278b8008c8ede617613fab61b8d678cea4047a87d50ead995ce7e0dd"} err="failed to get container status \"49fc1c00278b8008c8ede617613fab61b8d678cea4047a87d50ead995ce7e0dd\": rpc error: code = NotFound desc = could not find container \"49fc1c00278b8008c8ede617613fab61b8d678cea4047a87d50ead995ce7e0dd\": container with ID starting with 49fc1c00278b8008c8ede617613fab61b8d678cea4047a87d50ead995ce7e0dd not found: ID does not exist" Jul 06 02:09:59.912148 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:59.912121 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-3d280d/sub-test-model-3d280d-kserve-7f5cb9f68f-tbzkh"] Jul 06 02:09:59.914732 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:59.914708 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-3d280d/sub-test-model-3d280d-kserve-7f5cb9f68f-tbzkh"] Jul 06 02:09:59.943721 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:59.943686 2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d-home\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 02:09:59.943721 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:59.943719 2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d-dshm\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 02:09:59.943721 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:59.943730 2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d-tls-certs\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 02:09:59.943913 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:09:59.943742 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rgwps\" (UniqueName: \"kubernetes.io/projected/d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d-kube-api-access-rgwps\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 02:10:00.459772 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:10:00.459736 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d" path="/var/lib/kubelet/pods/d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d/volumes" Jul 06 02:10:04.036233 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:10:04.036194 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-388824/rate-test-model-388824-kserve-f456bd85f-4xzk7" Jul 06 02:10:04.036744 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:10:04.036253 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-388824/rate-test-model-388824-kserve-f456bd85f-4xzk7" Jul 06 02:10:04.048797 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:10:04.048767 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-388824/rate-test-model-388824-kserve-f456bd85f-4xzk7" Jul 06 02:10:04.925796 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:10:04.925762 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-388824/rate-test-model-388824-kserve-f456bd85f-4xzk7" Jul 06 02:10:13.419684 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:10:13.419606 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-3d280d/rate-test-model-3d280d-kserve-6cd875d9f9-mqjrr"] Jul 06 02:10:13.420034 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:10:13.419902 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d" containerName="main" Jul 06 02:10:13.420034 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:10:13.419913 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d" containerName="main" Jul 06 02:10:13.420034 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:10:13.419959 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d32654a1-ac2a-4d2a-af35-bfe8c1a20c9d" containerName="main" Jul 06 02:10:13.424313 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:10:13.424293 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3d280d/rate-test-model-3d280d-kserve-6cd875d9f9-mqjrr" Jul 06 02:10:13.426945 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:10:13.426922 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-3d280d\"/\"openshift-service-ca.crt\"" Jul 06 02:10:13.428099 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:10:13.428076 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-3d280d\"/\"kube-root-ca.crt\"" Jul 06 02:10:13.428304 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:10:13.428274 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-3d280d\"/\"rate-test-model-3d280d-kserve-self-signed-certs\"" Jul 06 02:10:13.428523 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:10:13.428207 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-3d280d\"/\"default-dockercfg-f62w6\"" Jul 06 02:10:13.431289 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:10:13.431263 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-3d280d/rate-test-model-3d280d-kserve-6cd875d9f9-mqjrr"] Jul 06 02:10:13.455714 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:10:13.455678 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:10:13.562466 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:10:13.562397 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jp6x\" (UniqueName: \"kubernetes.io/projected/9556a03e-2a2b-44d6-a841-05a97409f8b1-kube-api-access-2jp6x\") pod \"rate-test-model-3d280d-kserve-6cd875d9f9-mqjrr\" (UID: \"9556a03e-2a2b-44d6-a841-05a97409f8b1\") " pod="ai-tenant-e2e-shared-b-3d280d/rate-test-model-3d280d-kserve-6cd875d9f9-mqjrr" Jul 06 02:10:13.562870 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:10:13.562484 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9556a03e-2a2b-44d6-a841-05a97409f8b1-home\") pod \"rate-test-model-3d280d-kserve-6cd875d9f9-mqjrr\" (UID: \"9556a03e-2a2b-44d6-a841-05a97409f8b1\") " pod="ai-tenant-e2e-shared-b-3d280d/rate-test-model-3d280d-kserve-6cd875d9f9-mqjrr" Jul 06 02:10:13.562870 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:10:13.562514 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9556a03e-2a2b-44d6-a841-05a97409f8b1-model-cache\") pod \"rate-test-model-3d280d-kserve-6cd875d9f9-mqjrr\" (UID: \"9556a03e-2a2b-44d6-a841-05a97409f8b1\") " pod="ai-tenant-e2e-shared-b-3d280d/rate-test-model-3d280d-kserve-6cd875d9f9-mqjrr" Jul 06 02:10:13.562870 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:10:13.562540 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9556a03e-2a2b-44d6-a841-05a97409f8b1-tls-certs\") pod \"rate-test-model-3d280d-kserve-6cd875d9f9-mqjrr\" (UID: \"9556a03e-2a2b-44d6-a841-05a97409f8b1\") " pod="ai-tenant-e2e-shared-b-3d280d/rate-test-model-3d280d-kserve-6cd875d9f9-mqjrr" Jul 06 02:10:13.562870 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:10:13.562593 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9556a03e-2a2b-44d6-a841-05a97409f8b1-dshm\") pod \"rate-test-model-3d280d-kserve-6cd875d9f9-mqjrr\" (UID: \"9556a03e-2a2b-44d6-a841-05a97409f8b1\") " pod="ai-tenant-e2e-shared-b-3d280d/rate-test-model-3d280d-kserve-6cd875d9f9-mqjrr" Jul 06 02:10:13.562870 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:10:13.562693 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9556a03e-2a2b-44d6-a841-05a97409f8b1-tmp-dir\") pod \"rate-test-model-3d280d-kserve-6cd875d9f9-mqjrr\" (UID: \"9556a03e-2a2b-44d6-a841-05a97409f8b1\") " pod="ai-tenant-e2e-shared-b-3d280d/rate-test-model-3d280d-kserve-6cd875d9f9-mqjrr" Jul 06 02:10:13.663562 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:10:13.663412 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jp6x\" (UniqueName: \"kubernetes.io/projected/9556a03e-2a2b-44d6-a841-05a97409f8b1-kube-api-access-2jp6x\") pod \"rate-test-model-3d280d-kserve-6cd875d9f9-mqjrr\" (UID: \"9556a03e-2a2b-44d6-a841-05a97409f8b1\") " pod="ai-tenant-e2e-shared-b-3d280d/rate-test-model-3d280d-kserve-6cd875d9f9-mqjrr" Jul 06 02:10:13.663562 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:10:13.663493 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9556a03e-2a2b-44d6-a841-05a97409f8b1-home\") pod \"rate-test-model-3d280d-kserve-6cd875d9f9-mqjrr\" (UID: \"9556a03e-2a2b-44d6-a841-05a97409f8b1\") " pod="ai-tenant-e2e-shared-b-3d280d/rate-test-model-3d280d-kserve-6cd875d9f9-mqjrr" Jul 06 02:10:13.663562 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:10:13.663523 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9556a03e-2a2b-44d6-a841-05a97409f8b1-model-cache\") pod \"rate-test-model-3d280d-kserve-6cd875d9f9-mqjrr\" (UID: \"9556a03e-2a2b-44d6-a841-05a97409f8b1\") " pod="ai-tenant-e2e-shared-b-3d280d/rate-test-model-3d280d-kserve-6cd875d9f9-mqjrr" Jul 06 02:10:13.663562 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:10:13.663552 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9556a03e-2a2b-44d6-a841-05a97409f8b1-tls-certs\") pod \"rate-test-model-3d280d-kserve-6cd875d9f9-mqjrr\" (UID: \"9556a03e-2a2b-44d6-a841-05a97409f8b1\") " pod="ai-tenant-e2e-shared-b-3d280d/rate-test-model-3d280d-kserve-6cd875d9f9-mqjrr" Jul 06 02:10:13.663893 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:10:13.663574 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9556a03e-2a2b-44d6-a841-05a97409f8b1-dshm\") pod \"rate-test-model-3d280d-kserve-6cd875d9f9-mqjrr\" (UID: \"9556a03e-2a2b-44d6-a841-05a97409f8b1\") " pod="ai-tenant-e2e-shared-b-3d280d/rate-test-model-3d280d-kserve-6cd875d9f9-mqjrr" Jul 06 02:10:13.663893 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:10:13.663668 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9556a03e-2a2b-44d6-a841-05a97409f8b1-tmp-dir\") pod \"rate-test-model-3d280d-kserve-6cd875d9f9-mqjrr\" (UID: \"9556a03e-2a2b-44d6-a841-05a97409f8b1\") " pod="ai-tenant-e2e-shared-b-3d280d/rate-test-model-3d280d-kserve-6cd875d9f9-mqjrr" Jul 06 02:10:13.663999 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:10:13.663954 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9556a03e-2a2b-44d6-a841-05a97409f8b1-home\") pod \"rate-test-model-3d280d-kserve-6cd875d9f9-mqjrr\" (UID: \"9556a03e-2a2b-44d6-a841-05a97409f8b1\") " pod="ai-tenant-e2e-shared-b-3d280d/rate-test-model-3d280d-kserve-6cd875d9f9-mqjrr" Jul 06 02:10:13.663999 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:10:13.663972 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9556a03e-2a2b-44d6-a841-05a97409f8b1-model-cache\") pod \"rate-test-model-3d280d-kserve-6cd875d9f9-mqjrr\" (UID: \"9556a03e-2a2b-44d6-a841-05a97409f8b1\") " pod="ai-tenant-e2e-shared-b-3d280d/rate-test-model-3d280d-kserve-6cd875d9f9-mqjrr" Jul 06 02:10:13.664072 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:10:13.664027 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9556a03e-2a2b-44d6-a841-05a97409f8b1-tmp-dir\") pod \"rate-test-model-3d280d-kserve-6cd875d9f9-mqjrr\" (UID: \"9556a03e-2a2b-44d6-a841-05a97409f8b1\") " pod="ai-tenant-e2e-shared-b-3d280d/rate-test-model-3d280d-kserve-6cd875d9f9-mqjrr" Jul 06 02:10:13.665871 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:10:13.665849 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9556a03e-2a2b-44d6-a841-05a97409f8b1-dshm\") pod \"rate-test-model-3d280d-kserve-6cd875d9f9-mqjrr\" (UID: \"9556a03e-2a2b-44d6-a841-05a97409f8b1\") " pod="ai-tenant-e2e-shared-b-3d280d/rate-test-model-3d280d-kserve-6cd875d9f9-mqjrr" Jul 06 02:10:13.666154 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:10:13.666138 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9556a03e-2a2b-44d6-a841-05a97409f8b1-tls-certs\") pod \"rate-test-model-3d280d-kserve-6cd875d9f9-mqjrr\" (UID: \"9556a03e-2a2b-44d6-a841-05a97409f8b1\") " pod="ai-tenant-e2e-shared-b-3d280d/rate-test-model-3d280d-kserve-6cd875d9f9-mqjrr" Jul 06 02:10:13.671977 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:10:13.671920 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jp6x\" (UniqueName: \"kubernetes.io/projected/9556a03e-2a2b-44d6-a841-05a97409f8b1-kube-api-access-2jp6x\") pod \"rate-test-model-3d280d-kserve-6cd875d9f9-mqjrr\" (UID: \"9556a03e-2a2b-44d6-a841-05a97409f8b1\") " pod="ai-tenant-e2e-shared-b-3d280d/rate-test-model-3d280d-kserve-6cd875d9f9-mqjrr" Jul 06 02:10:13.735043 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:10:13.735004 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3d280d/rate-test-model-3d280d-kserve-6cd875d9f9-mqjrr" Jul 06 02:10:13.877121 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:10:13.877088 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-3d280d/rate-test-model-3d280d-kserve-6cd875d9f9-mqjrr"] Jul 06 02:10:13.880553 ip-10-0-139-44 kubenswrapper[2580]: W0706 02:10:13.880518 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9556a03e_2a2b_44d6_a841_05a97409f8b1.slice/crio-66a093270cf702189cf6ac88d525b106ec295a5e45f4739b4b16308496860dce WatchSource:0}: Error finding container 66a093270cf702189cf6ac88d525b106ec295a5e45f4739b4b16308496860dce: Status 404 returned error can't find the container with id 66a093270cf702189cf6ac88d525b106ec295a5e45f4739b4b16308496860dce Jul 06 02:10:13.953200 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:10:13.953160 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3d280d/rate-test-model-3d280d-kserve-6cd875d9f9-mqjrr" event={"ID":"9556a03e-2a2b-44d6-a841-05a97409f8b1","Type":"ContainerStarted","Data":"7130c43bbcfb9388bdfb7fd04a066a3b43495ffbb5401eda279e84d5620362bf"} Jul 06 02:10:13.953375 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:10:13.953212 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3d280d/rate-test-model-3d280d-kserve-6cd875d9f9-mqjrr" event={"ID":"9556a03e-2a2b-44d6-a841-05a97409f8b1","Type":"ContainerStarted","Data":"66a093270cf702189cf6ac88d525b106ec295a5e45f4739b4b16308496860dce"} Jul 06 02:10:13.972985 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:10:13.972912 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-3d280d/rate-test-model-3d280d-kserve-6cd875d9f9-mqjrr" podStartSLOduration=0.972887933 podStartE2EDuration="972.887933ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 02:10:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 02:10:13.971685932 +0000 UTC m=+2607.942937624" watchObservedRunningTime="2026-07-06 02:10:13.972887933 +0000 UTC m=+2607.944139627" Jul 06 02:10:18.184137 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:10:18.184088 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:10:23.735280 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:10:23.735236 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-3d280d/rate-test-model-3d280d-kserve-6cd875d9f9-mqjrr" Jul 06 02:10:23.735280 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:10:23.735294 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-3d280d/rate-test-model-3d280d-kserve-6cd875d9f9-mqjrr" Jul 06 02:10:23.747836 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:10:23.747803 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-3d280d/rate-test-model-3d280d-kserve-6cd875d9f9-mqjrr" Jul 06 02:10:24.007791 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:10:24.007703 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-3d280d/rate-test-model-3d280d-kserve-6cd875d9f9-mqjrr" Jul 06 02:10:42.223150 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:10:42.223107 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:10:42.322411 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:10:42.322366 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:10:51.686420 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:10:51.686385 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:10:51.786273 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:10:51.786238 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:11:02.505296 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:02.505255 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-388824/rate-test-model-388824-kserve-f456bd85f-4xzk7"] Jul 06 02:11:02.528310 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:02.528280 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:11:03.323195 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:03.323161 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-3d280d/rate-test-model-3d280d-kserve-6cd875d9f9-mqjrr"] Jul 06 02:11:03.339491 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:03.339457 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:11:04.420679 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:04.420647 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:11:04.894271 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:04.894230 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:11:04.991304 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:04.991271 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:11:05.091724 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:05.091688 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:11:05.165777 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:05.165690 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:11:09.961267 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:09.961220 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:11:17.525859 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:17.525799 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-388824/rate-test-model-388824-kserve-f456bd85f-4xzk7" podUID="426b2504-0e69-40a4-9f4f-2c1fa76230db" containerName="main" containerID="cri-o://bc59e0557bfe107ac928c3f5255932fae5c42f1deab77e373d9626afb167551a" gracePeriod=45 Jul 06 02:11:17.776133 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:17.776048 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-388824/rate-test-model-388824-kserve-f456bd85f-4xzk7" Jul 06 02:11:17.824387 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:17.824349 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-skzkz\" (UniqueName: \"kubernetes.io/projected/426b2504-0e69-40a4-9f4f-2c1fa76230db-kube-api-access-skzkz\") pod \"426b2504-0e69-40a4-9f4f-2c1fa76230db\" (UID: \"426b2504-0e69-40a4-9f4f-2c1fa76230db\") " Jul 06 02:11:17.824614 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:17.824398 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/426b2504-0e69-40a4-9f4f-2c1fa76230db-model-cache\") pod \"426b2504-0e69-40a4-9f4f-2c1fa76230db\" (UID: \"426b2504-0e69-40a4-9f4f-2c1fa76230db\") " Jul 06 02:11:17.824614 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:17.824544 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/426b2504-0e69-40a4-9f4f-2c1fa76230db-tls-certs\") pod \"426b2504-0e69-40a4-9f4f-2c1fa76230db\" (UID: \"426b2504-0e69-40a4-9f4f-2c1fa76230db\") " Jul 06 02:11:17.824614 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:17.824573 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/426b2504-0e69-40a4-9f4f-2c1fa76230db-tmp-dir\") pod \"426b2504-0e69-40a4-9f4f-2c1fa76230db\" (UID: \"426b2504-0e69-40a4-9f4f-2c1fa76230db\") " Jul 06 02:11:17.824792 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:17.824615 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/426b2504-0e69-40a4-9f4f-2c1fa76230db-home\") pod \"426b2504-0e69-40a4-9f4f-2c1fa76230db\" (UID: \"426b2504-0e69-40a4-9f4f-2c1fa76230db\") " Jul 06 02:11:17.824792 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:17.824651 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/426b2504-0e69-40a4-9f4f-2c1fa76230db-dshm\") pod \"426b2504-0e69-40a4-9f4f-2c1fa76230db\" (UID: \"426b2504-0e69-40a4-9f4f-2c1fa76230db\") " Jul 06 02:11:17.824792 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:17.824668 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/426b2504-0e69-40a4-9f4f-2c1fa76230db-model-cache" (OuterVolumeSpecName: "model-cache") pod "426b2504-0e69-40a4-9f4f-2c1fa76230db" (UID: "426b2504-0e69-40a4-9f4f-2c1fa76230db"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:11:17.824936 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:17.824810 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/426b2504-0e69-40a4-9f4f-2c1fa76230db-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "426b2504-0e69-40a4-9f4f-2c1fa76230db" (UID: "426b2504-0e69-40a4-9f4f-2c1fa76230db"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:11:17.824936 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:17.824895 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/426b2504-0e69-40a4-9f4f-2c1fa76230db-home" (OuterVolumeSpecName: "home") pod "426b2504-0e69-40a4-9f4f-2c1fa76230db" (UID: "426b2504-0e69-40a4-9f4f-2c1fa76230db"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:11:17.824936 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:17.824907 2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/426b2504-0e69-40a4-9f4f-2c1fa76230db-model-cache\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 02:11:17.824936 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:17.824927 2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/426b2504-0e69-40a4-9f4f-2c1fa76230db-tmp-dir\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 02:11:17.826766 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:17.826733 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/426b2504-0e69-40a4-9f4f-2c1fa76230db-dshm" (OuterVolumeSpecName: "dshm") pod "426b2504-0e69-40a4-9f4f-2c1fa76230db" (UID: "426b2504-0e69-40a4-9f4f-2c1fa76230db"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:11:17.826882 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:17.826803 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/426b2504-0e69-40a4-9f4f-2c1fa76230db-kube-api-access-skzkz" (OuterVolumeSpecName: "kube-api-access-skzkz") pod "426b2504-0e69-40a4-9f4f-2c1fa76230db" (UID: "426b2504-0e69-40a4-9f4f-2c1fa76230db"). InnerVolumeSpecName "kube-api-access-skzkz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 02:11:17.826882 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:17.826855 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/426b2504-0e69-40a4-9f4f-2c1fa76230db-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "426b2504-0e69-40a4-9f4f-2c1fa76230db" (UID: "426b2504-0e69-40a4-9f4f-2c1fa76230db"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 02:11:17.926184 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:17.926141 2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/426b2504-0e69-40a4-9f4f-2c1fa76230db-tls-certs\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 02:11:17.926184 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:17.926174 2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/426b2504-0e69-40a4-9f4f-2c1fa76230db-home\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 02:11:17.926184 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:17.926185 2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/426b2504-0e69-40a4-9f4f-2c1fa76230db-dshm\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 02:11:17.926184 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:17.926195 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-skzkz\" (UniqueName: \"kubernetes.io/projected/426b2504-0e69-40a4-9f4f-2c1fa76230db-kube-api-access-skzkz\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 02:11:18.223914 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:18.223873 2580 generic.go:356] "Generic (PLEG): container finished" podID="426b2504-0e69-40a4-9f4f-2c1fa76230db" containerID="bc59e0557bfe107ac928c3f5255932fae5c42f1deab77e373d9626afb167551a" exitCode=0 Jul 06 02:11:18.224099 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:18.223921 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-388824/rate-test-model-388824-kserve-f456bd85f-4xzk7" Jul 06 02:11:18.224099 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:18.223958 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-388824/rate-test-model-388824-kserve-f456bd85f-4xzk7" event={"ID":"426b2504-0e69-40a4-9f4f-2c1fa76230db","Type":"ContainerDied","Data":"bc59e0557bfe107ac928c3f5255932fae5c42f1deab77e373d9626afb167551a"} Jul 06 02:11:18.224099 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:18.224021 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-388824/rate-test-model-388824-kserve-f456bd85f-4xzk7" event={"ID":"426b2504-0e69-40a4-9f4f-2c1fa76230db","Type":"ContainerDied","Data":"7440561748bd9452fecce1a0a56d24e9ec6b76c89fd2febf48cb0a9545032dd5"} Jul 06 02:11:18.224099 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:18.224041 2580 scope.go:117] "RemoveContainer" containerID="bc59e0557bfe107ac928c3f5255932fae5c42f1deab77e373d9626afb167551a" Jul 06 02:11:18.232787 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:18.232769 2580 scope.go:117] "RemoveContainer" containerID="bc59e0557bfe107ac928c3f5255932fae5c42f1deab77e373d9626afb167551a" Jul 06 02:11:18.233058 ip-10-0-139-44 kubenswrapper[2580]: E0706 02:11:18.233036 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc59e0557bfe107ac928c3f5255932fae5c42f1deab77e373d9626afb167551a\": container with ID starting with bc59e0557bfe107ac928c3f5255932fae5c42f1deab77e373d9626afb167551a not found: ID does not exist" containerID="bc59e0557bfe107ac928c3f5255932fae5c42f1deab77e373d9626afb167551a" Jul 06 02:11:18.233120 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:18.233071 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc59e0557bfe107ac928c3f5255932fae5c42f1deab77e373d9626afb167551a"} err="failed to get container status \"bc59e0557bfe107ac928c3f5255932fae5c42f1deab77e373d9626afb167551a\": rpc error: code = NotFound desc = could not find container \"bc59e0557bfe107ac928c3f5255932fae5c42f1deab77e373d9626afb167551a\": container with ID starting with bc59e0557bfe107ac928c3f5255932fae5c42f1deab77e373d9626afb167551a not found: ID does not exist" Jul 06 02:11:18.243710 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:18.243684 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-388824/rate-test-model-388824-kserve-f456bd85f-4xzk7"] Jul 06 02:11:18.247053 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:18.247028 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-388824/rate-test-model-388824-kserve-f456bd85f-4xzk7"] Jul 06 02:11:18.344408 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:18.344346 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-3d280d/rate-test-model-3d280d-kserve-6cd875d9f9-mqjrr" podUID="9556a03e-2a2b-44d6-a841-05a97409f8b1" containerName="main" containerID="cri-o://7130c43bbcfb9388bdfb7fd04a066a3b43495ffbb5401eda279e84d5620362bf" gracePeriod=45 Jul 06 02:11:18.459399 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:18.459364 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="426b2504-0e69-40a4-9f4f-2c1fa76230db" path="/var/lib/kubelet/pods/426b2504-0e69-40a4-9f4f-2c1fa76230db/volumes" Jul 06 02:11:18.593549 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:18.593519 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3d280d/rate-test-model-3d280d-kserve-6cd875d9f9-mqjrr" Jul 06 02:11:18.631868 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:18.631839 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9556a03e-2a2b-44d6-a841-05a97409f8b1-dshm\") pod \"9556a03e-2a2b-44d6-a841-05a97409f8b1\" (UID: \"9556a03e-2a2b-44d6-a841-05a97409f8b1\") " Jul 06 02:11:18.631868 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:18.631872 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9556a03e-2a2b-44d6-a841-05a97409f8b1-home\") pod \"9556a03e-2a2b-44d6-a841-05a97409f8b1\" (UID: \"9556a03e-2a2b-44d6-a841-05a97409f8b1\") " Jul 06 02:11:18.632077 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:18.631897 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9556a03e-2a2b-44d6-a841-05a97409f8b1-model-cache\") pod \"9556a03e-2a2b-44d6-a841-05a97409f8b1\" (UID: \"9556a03e-2a2b-44d6-a841-05a97409f8b1\") " Jul 06 02:11:18.632077 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:18.631931 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2jp6x\" (UniqueName: \"kubernetes.io/projected/9556a03e-2a2b-44d6-a841-05a97409f8b1-kube-api-access-2jp6x\") pod \"9556a03e-2a2b-44d6-a841-05a97409f8b1\" (UID: \"9556a03e-2a2b-44d6-a841-05a97409f8b1\") " Jul 06 02:11:18.632077 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:18.631948 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9556a03e-2a2b-44d6-a841-05a97409f8b1-tls-certs\") pod \"9556a03e-2a2b-44d6-a841-05a97409f8b1\" (UID: \"9556a03e-2a2b-44d6-a841-05a97409f8b1\") " Jul 06 02:11:18.632077 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:18.632035 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9556a03e-2a2b-44d6-a841-05a97409f8b1-tmp-dir\") pod \"9556a03e-2a2b-44d6-a841-05a97409f8b1\" (UID: \"9556a03e-2a2b-44d6-a841-05a97409f8b1\") " Jul 06 02:11:18.632297 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:18.632114 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9556a03e-2a2b-44d6-a841-05a97409f8b1-home" (OuterVolumeSpecName: "home") pod "9556a03e-2a2b-44d6-a841-05a97409f8b1" (UID: "9556a03e-2a2b-44d6-a841-05a97409f8b1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:11:18.632297 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:18.632216 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9556a03e-2a2b-44d6-a841-05a97409f8b1-model-cache" (OuterVolumeSpecName: "model-cache") pod "9556a03e-2a2b-44d6-a841-05a97409f8b1" (UID: "9556a03e-2a2b-44d6-a841-05a97409f8b1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:11:18.632412 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:18.632309 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9556a03e-2a2b-44d6-a841-05a97409f8b1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9556a03e-2a2b-44d6-a841-05a97409f8b1" (UID: "9556a03e-2a2b-44d6-a841-05a97409f8b1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:11:18.632412 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:18.632366 2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9556a03e-2a2b-44d6-a841-05a97409f8b1-home\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 02:11:18.632412 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:18.632379 2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9556a03e-2a2b-44d6-a841-05a97409f8b1-model-cache\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 02:11:18.634167 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:18.634142 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9556a03e-2a2b-44d6-a841-05a97409f8b1-dshm" (OuterVolumeSpecName: "dshm") pod "9556a03e-2a2b-44d6-a841-05a97409f8b1" (UID: "9556a03e-2a2b-44d6-a841-05a97409f8b1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:11:18.634283 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:18.634163 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9556a03e-2a2b-44d6-a841-05a97409f8b1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9556a03e-2a2b-44d6-a841-05a97409f8b1" (UID: "9556a03e-2a2b-44d6-a841-05a97409f8b1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 02:11:18.634283 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:18.634204 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9556a03e-2a2b-44d6-a841-05a97409f8b1-kube-api-access-2jp6x" (OuterVolumeSpecName: "kube-api-access-2jp6x") pod "9556a03e-2a2b-44d6-a841-05a97409f8b1" (UID: "9556a03e-2a2b-44d6-a841-05a97409f8b1"). InnerVolumeSpecName "kube-api-access-2jp6x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 02:11:18.733321 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:18.733266 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2jp6x\" (UniqueName: \"kubernetes.io/projected/9556a03e-2a2b-44d6-a841-05a97409f8b1-kube-api-access-2jp6x\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 02:11:18.733321 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:18.733315 2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9556a03e-2a2b-44d6-a841-05a97409f8b1-tls-certs\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 02:11:18.733321 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:18.733326 2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9556a03e-2a2b-44d6-a841-05a97409f8b1-tmp-dir\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 02:11:18.733321 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:18.733333 2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9556a03e-2a2b-44d6-a841-05a97409f8b1-dshm\") on node \"ip-10-0-139-44.ec2.internal\" DevicePath \"\"" Jul 06 02:11:19.230616 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:19.230577 2580 generic.go:356] "Generic (PLEG): container finished" podID="9556a03e-2a2b-44d6-a841-05a97409f8b1" containerID="7130c43bbcfb9388bdfb7fd04a066a3b43495ffbb5401eda279e84d5620362bf" exitCode=0 Jul 06 02:11:19.230816 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:19.230639 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3d280d/rate-test-model-3d280d-kserve-6cd875d9f9-mqjrr" Jul 06 02:11:19.230816 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:19.230643 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3d280d/rate-test-model-3d280d-kserve-6cd875d9f9-mqjrr" event={"ID":"9556a03e-2a2b-44d6-a841-05a97409f8b1","Type":"ContainerDied","Data":"7130c43bbcfb9388bdfb7fd04a066a3b43495ffbb5401eda279e84d5620362bf"} Jul 06 02:11:19.230816 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:19.230747 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3d280d/rate-test-model-3d280d-kserve-6cd875d9f9-mqjrr" event={"ID":"9556a03e-2a2b-44d6-a841-05a97409f8b1","Type":"ContainerDied","Data":"66a093270cf702189cf6ac88d525b106ec295a5e45f4739b4b16308496860dce"} Jul 06 02:11:19.230816 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:19.230769 2580 scope.go:117] "RemoveContainer" containerID="7130c43bbcfb9388bdfb7fd04a066a3b43495ffbb5401eda279e84d5620362bf" Jul 06 02:11:19.239397 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:19.239376 2580 scope.go:117] "RemoveContainer" containerID="7130c43bbcfb9388bdfb7fd04a066a3b43495ffbb5401eda279e84d5620362bf" Jul 06 02:11:19.239688 ip-10-0-139-44 kubenswrapper[2580]: E0706 02:11:19.239668 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7130c43bbcfb9388bdfb7fd04a066a3b43495ffbb5401eda279e84d5620362bf\": container with ID starting with 7130c43bbcfb9388bdfb7fd04a066a3b43495ffbb5401eda279e84d5620362bf not found: ID does not exist" containerID="7130c43bbcfb9388bdfb7fd04a066a3b43495ffbb5401eda279e84d5620362bf" Jul 06 02:11:19.239750 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:19.239699 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7130c43bbcfb9388bdfb7fd04a066a3b43495ffbb5401eda279e84d5620362bf"} err="failed to get container status \"7130c43bbcfb9388bdfb7fd04a066a3b43495ffbb5401eda279e84d5620362bf\": rpc error: code = NotFound desc = could not find container \"7130c43bbcfb9388bdfb7fd04a066a3b43495ffbb5401eda279e84d5620362bf\": container with ID starting with 7130c43bbcfb9388bdfb7fd04a066a3b43495ffbb5401eda279e84d5620362bf not found: ID does not exist" Jul 06 02:11:19.250686 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:19.250659 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-3d280d/rate-test-model-3d280d-kserve-6cd875d9f9-mqjrr"] Jul 06 02:11:19.254513 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:19.254489 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-3d280d/rate-test-model-3d280d-kserve-6cd875d9f9-mqjrr"] Jul 06 02:11:20.459698 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:20.459667 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9556a03e-2a2b-44d6-a841-05a97409f8b1" path="/var/lib/kubelet/pods/9556a03e-2a2b-44d6-a841-05a97409f8b1/volumes" Jul 06 02:11:29.862930 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:29.862888 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:11:30.147286 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:30.147243 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:11:30.391260 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:30.391225 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:11:30.545092 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:30.545000 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:11:30.922210 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:30.922179 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:11:31.023115 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:31.023080 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:11:31.124176 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:31.124147 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:11:31.224304 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:31.224206 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:11:36.892700 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:36.892596 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:11:36.954681 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:36.954641 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:11:37.165802 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:37.165718 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:11:37.396131 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:37.396093 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:11:37.841633 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:37.841599 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6d6bbcbc89-bl28q_7beebb74-3118-44cb-8bac-6dbe85d41396/authorino/0.log" Jul 06 02:11:43.160404 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:43.160371 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-cdlg2_22e8a402-6186-4a45-a5c7-20bb3975c6a3/server/0.log" Jul 06 02:11:43.296133 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:43.296088 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-894jd_e9e51d11-b969-4794-ae2e-b34b00bd635b/manager/0.log" Jul 06 02:11:43.413476 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:43.413379 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-656987b587-lfrb4_6bff963e-5246-4800-b50e-84afb8612738/copy-manifests/0.log" Jul 06 02:11:43.431287 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:43.431245 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-656987b587-lfrb4_6bff963e-5246-4800-b50e-84afb8612738/manager/0.log" Jul 06 02:11:45.095272 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:45.095233 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6d6bbcbc89-bl28q_7beebb74-3118-44cb-8bac-6dbe85d41396/authorino/0.log" Jul 06 02:11:45.430859 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:45.430828 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-lmx6s_2ec65f6a-2cde-4d3e-832c-7123ff9dcdc7/kuadrant-console-plugin/0.log" Jul 06 02:11:46.486548 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:46.486514 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7f644fb7f6-brmqh_9a9dd62f-b9cd-4c63-bcae-d2d40082c5de/kube-auth-proxy/0.log" Jul 06 02:11:47.681133 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:47.681075 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-nzfjx_4d52f99a-240b-44c3-9ac5-852ed9b8f175/main/0.log" Jul 06 02:11:47.813356 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:47.813325 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdk2pp_459003f0-b512-48db-9f0f-b7910dcded53/main/0.log" Jul 06 02:11:48.051355 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:48.051246 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-nhrl7_6379a3b1-edbd-40bc-8ad6-0df732308f6a/main/0.log" Jul 06 02:11:51.984541 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:51.984496 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:11:52.092316 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:52.092275 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" limit=1024 Jul 06 02:11:54.835001 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:54.834968 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-xf5mp_e690e9f2-b314-4e44-a211-433d7e8100a1/global-pull-secret-syncer/0.log" Jul 06 02:11:54.893398 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:54.893366 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-cksd7_cb081505-8ac0-48a3-be9e-bf770a830127/konnectivity-agent/0.log" Jul 06 02:11:55.018278 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:11:55.018248 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-44.ec2.internal_fa04038e4e7372411052bdcfb35944fb/haproxy/0.log" Jul 06 02:12:00.376380 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:00.376342 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6d6bbcbc89-bl28q_7beebb74-3118-44cb-8bac-6dbe85d41396/authorino/0.log" Jul 06 02:12:00.458617 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:00.458584 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-lmx6s_2ec65f6a-2cde-4d3e-832c-7123ff9dcdc7/kuadrant-console-plugin/0.log" Jul 06 02:12:02.942522 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:02.942487 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-szz4p_a2b629f3-f3fc-40ef-af43-bfbdc4be7442/node-exporter/0.log" Jul 06 02:12:02.962890 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:02.962859 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-szz4p_a2b629f3-f3fc-40ef-af43-bfbdc4be7442/kube-rbac-proxy/0.log" Jul 06 02:12:02.982751 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:02.982723 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-szz4p_a2b629f3-f3fc-40ef-af43-bfbdc4be7442/init-textfile/0.log" Jul 06 02:12:05.628863 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:05.628822 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-jdvmn/perf-node-gather-daemonset-gff44"] Jul 06 02:12:05.629234 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:05.629170 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9556a03e-2a2b-44d6-a841-05a97409f8b1" containerName="main" Jul 06 02:12:05.629234 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:05.629184 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9556a03e-2a2b-44d6-a841-05a97409f8b1" containerName="main" Jul 06 02:12:05.629234 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:05.629195 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="426b2504-0e69-40a4-9f4f-2c1fa76230db" containerName="main" Jul 06 02:12:05.629234 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:05.629199 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="426b2504-0e69-40a4-9f4f-2c1fa76230db" containerName="main" Jul 06 02:12:05.629368 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:05.629280 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9556a03e-2a2b-44d6-a841-05a97409f8b1" containerName="main" Jul 06 02:12:05.629368 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:05.629289 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="426b2504-0e69-40a4-9f4f-2c1fa76230db" containerName="main" Jul 06 02:12:05.632316 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:05.632299 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jdvmn/perf-node-gather-daemonset-gff44" Jul 06 02:12:05.634804 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:05.634779 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-jdvmn\"/\"default-dockercfg-tg2sc\"" Jul 06 02:12:05.634942 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:05.634780 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jdvmn\"/\"openshift-service-ca.crt\"" Jul 06 02:12:05.635835 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:05.635819 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jdvmn\"/\"kube-root-ca.crt\"" Jul 06 02:12:05.641250 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:05.641227 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jdvmn/perf-node-gather-daemonset-gff44"] Jul 06 02:12:05.646164 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:05.646138 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3752b590-9389-46cf-8228-1898df657c2b-lib-modules\") pod \"perf-node-gather-daemonset-gff44\" (UID: \"3752b590-9389-46cf-8228-1898df657c2b\") " pod="openshift-must-gather-jdvmn/perf-node-gather-daemonset-gff44" Jul 06 02:12:05.646269 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:05.646197 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9lqr\" (UniqueName: \"kubernetes.io/projected/3752b590-9389-46cf-8228-1898df657c2b-kube-api-access-n9lqr\") pod \"perf-node-gather-daemonset-gff44\" (UID: \"3752b590-9389-46cf-8228-1898df657c2b\") " pod="openshift-must-gather-jdvmn/perf-node-gather-daemonset-gff44" Jul 06 02:12:05.646269 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:05.646239 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3752b590-9389-46cf-8228-1898df657c2b-sys\") pod \"perf-node-gather-daemonset-gff44\" (UID: \"3752b590-9389-46cf-8228-1898df657c2b\") " pod="openshift-must-gather-jdvmn/perf-node-gather-daemonset-gff44" Jul 06 02:12:05.646345 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:05.646273 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3752b590-9389-46cf-8228-1898df657c2b-proc\") pod \"perf-node-gather-daemonset-gff44\" (UID: \"3752b590-9389-46cf-8228-1898df657c2b\") " pod="openshift-must-gather-jdvmn/perf-node-gather-daemonset-gff44" Jul 06 02:12:05.646345 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:05.646296 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3752b590-9389-46cf-8228-1898df657c2b-podres\") pod \"perf-node-gather-daemonset-gff44\" (UID: \"3752b590-9389-46cf-8228-1898df657c2b\") " pod="openshift-must-gather-jdvmn/perf-node-gather-daemonset-gff44" Jul 06 02:12:05.747429 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:05.747387 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9lqr\" (UniqueName: \"kubernetes.io/projected/3752b590-9389-46cf-8228-1898df657c2b-kube-api-access-n9lqr\") pod \"perf-node-gather-daemonset-gff44\" (UID: \"3752b590-9389-46cf-8228-1898df657c2b\") " pod="openshift-must-gather-jdvmn/perf-node-gather-daemonset-gff44" Jul 06 02:12:05.747429 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:05.747452 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3752b590-9389-46cf-8228-1898df657c2b-sys\") pod \"perf-node-gather-daemonset-gff44\" (UID: \"3752b590-9389-46cf-8228-1898df657c2b\") " pod="openshift-must-gather-jdvmn/perf-node-gather-daemonset-gff44" Jul 06 02:12:05.747734 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:05.747489 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3752b590-9389-46cf-8228-1898df657c2b-proc\") pod \"perf-node-gather-daemonset-gff44\" (UID: \"3752b590-9389-46cf-8228-1898df657c2b\") " pod="openshift-must-gather-jdvmn/perf-node-gather-daemonset-gff44" Jul 06 02:12:05.747734 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:05.747511 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3752b590-9389-46cf-8228-1898df657c2b-podres\") pod \"perf-node-gather-daemonset-gff44\" (UID: \"3752b590-9389-46cf-8228-1898df657c2b\") " pod="openshift-must-gather-jdvmn/perf-node-gather-daemonset-gff44" Jul 06 02:12:05.747734 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:05.747545 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3752b590-9389-46cf-8228-1898df657c2b-lib-modules\") pod \"perf-node-gather-daemonset-gff44\" (UID: \"3752b590-9389-46cf-8228-1898df657c2b\") " pod="openshift-must-gather-jdvmn/perf-node-gather-daemonset-gff44" Jul 06 02:12:05.747734 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:05.747601 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3752b590-9389-46cf-8228-1898df657c2b-sys\") pod \"perf-node-gather-daemonset-gff44\" (UID: \"3752b590-9389-46cf-8228-1898df657c2b\") " pod="openshift-must-gather-jdvmn/perf-node-gather-daemonset-gff44" Jul 06 02:12:05.747734 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:05.747642 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3752b590-9389-46cf-8228-1898df657c2b-proc\") pod \"perf-node-gather-daemonset-gff44\" (UID: \"3752b590-9389-46cf-8228-1898df657c2b\") " pod="openshift-must-gather-jdvmn/perf-node-gather-daemonset-gff44" Jul 06 02:12:05.747734 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:05.747691 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3752b590-9389-46cf-8228-1898df657c2b-podres\") pod \"perf-node-gather-daemonset-gff44\" (UID: \"3752b590-9389-46cf-8228-1898df657c2b\") " pod="openshift-must-gather-jdvmn/perf-node-gather-daemonset-gff44" Jul 06 02:12:05.747734 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:05.747699 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3752b590-9389-46cf-8228-1898df657c2b-lib-modules\") pod \"perf-node-gather-daemonset-gff44\" (UID: \"3752b590-9389-46cf-8228-1898df657c2b\") " pod="openshift-must-gather-jdvmn/perf-node-gather-daemonset-gff44" Jul 06 02:12:05.756263 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:05.756239 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9lqr\" (UniqueName: \"kubernetes.io/projected/3752b590-9389-46cf-8228-1898df657c2b-kube-api-access-n9lqr\") pod \"perf-node-gather-daemonset-gff44\" (UID: \"3752b590-9389-46cf-8228-1898df657c2b\") " pod="openshift-must-gather-jdvmn/perf-node-gather-daemonset-gff44" Jul 06 02:12:05.942167 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:05.942132 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jdvmn/perf-node-gather-daemonset-gff44" Jul 06 02:12:05.949056 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:05.949026 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-68kkp_108029ee-5264-4c60-91b5-a390aea7fe33/download-server/0.log" Jul 06 02:12:06.074458 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:06.074404 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jdvmn/perf-node-gather-daemonset-gff44"] Jul 06 02:12:06.077229 ip-10-0-139-44 kubenswrapper[2580]: W0706 02:12:06.077204 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod3752b590_9389_46cf_8228_1898df657c2b.slice/crio-67a5e81c66de26d795afa556589656f2569434ef407650fb5ec48725ca58ba9f WatchSource:0}: Error finding container 67a5e81c66de26d795afa556589656f2569434ef407650fb5ec48725ca58ba9f: Status 404 returned error can't find the container with id 67a5e81c66de26d795afa556589656f2569434ef407650fb5ec48725ca58ba9f Jul 06 02:12:06.438894 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:06.438855 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jdvmn/perf-node-gather-daemonset-gff44" event={"ID":"3752b590-9389-46cf-8228-1898df657c2b","Type":"ContainerStarted","Data":"1e1e7e585a532c20a0a3bbd4ea1d1f3ea8feb809aef518820e9a33fefd52efda"} Jul 06 02:12:06.439085 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:06.438908 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jdvmn/perf-node-gather-daemonset-gff44" event={"ID":"3752b590-9389-46cf-8228-1898df657c2b","Type":"ContainerStarted","Data":"67a5e81c66de26d795afa556589656f2569434ef407650fb5ec48725ca58ba9f"} Jul 06 02:12:06.439085 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:06.438947 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-jdvmn/perf-node-gather-daemonset-gff44" Jul 06 02:12:06.455195 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:06.455139 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-jdvmn/perf-node-gather-daemonset-gff44" podStartSLOduration=1.455120516 podStartE2EDuration="1.455120516s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 02:12:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 02:12:06.453532169 +0000 UTC m=+2720.424783861" watchObservedRunningTime="2026-07-06 02:12:06.455120516 +0000 UTC m=+2720.426372208" Jul 06 02:12:07.679103 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:07.679073 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dxjmj_d5f07e4d-9391-4b17-8cd2-298dde338112/dns/0.log" Jul 06 02:12:07.697970 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:07.697942 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dxjmj_d5f07e4d-9391-4b17-8cd2-298dde338112/kube-rbac-proxy/0.log" Jul 06 02:12:07.744412 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:07.744378 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-82mwf_00a6f863-8ae0-4749-8f27-3bdf91cd8775/dns-node-resolver/0.log" Jul 06 02:12:08.323909 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:08.323860 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-67887675c-sjf6n_bcdcad6b-89cf-4818-9365-9b521ed419f3/registry/0.log" Jul 06 02:12:08.370614 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:08.370585 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-b46kx_4164d767-f4d0-459d-b0c1-cc8f8ccbaf1a/node-ca/0.log" Jul 06 02:12:09.436113 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:09.436072 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7f644fb7f6-brmqh_9a9dd62f-b9cd-4c63-bcae-d2d40082c5de/kube-auth-proxy/0.log" Jul 06 02:12:10.353138 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:10.353107 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-v7w8q_d90b9b62-2b7e-439a-9d24-4893aa5543d9/serve-healthcheck-canary/0.log" Jul 06 02:12:10.828919 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:10.828888 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fcdfg_227d7764-6d83-4e82-953d-e33128c2c10a/kube-rbac-proxy/0.log" Jul 06 02:12:10.848065 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:10.848033 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fcdfg_227d7764-6d83-4e82-953d-e33128c2c10a/exporter/0.log" Jul 06 02:12:10.868815 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:10.868774 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fcdfg_227d7764-6d83-4e82-953d-e33128c2c10a/extractor/0.log" Jul 06 02:12:12.458902 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:12.458874 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-jdvmn/perf-node-gather-daemonset-gff44" Jul 06 02:12:13.520788 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:13.520765 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-cdlg2_22e8a402-6186-4a45-a5c7-20bb3975c6a3/server/0.log" Jul 06 02:12:13.624197 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:13.624161 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-894jd_e9e51d11-b969-4794-ae2e-b34b00bd635b/manager/0.log" Jul 06 02:12:13.740218 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:13.740180 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-656987b587-lfrb4_6bff963e-5246-4800-b50e-84afb8612738/manager/0.log" Jul 06 02:12:13.756705 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:13.756674 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-656987b587-lfrb4_6bff963e-5246-4800-b50e-84afb8612738/copy-manifests/0.log" Jul 06 02:12:15.392405 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:15.392373 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-577bbf7cf9-xqvfn_cfbbbe93-c39e-4799-8aa6-bb2ca44d790d/manager/0.log" Jul 06 02:12:16.315219 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:16.315185 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7f644fb7f6-brmqh_9a9dd62f-b9cd-4c63-bcae-d2d40082c5de/kube-auth-proxy/0.log" Jul 06 02:12:17.829712 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:17.829673 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7f644fb7f6-brmqh_9a9dd62f-b9cd-4c63-bcae-d2d40082c5de/kube-auth-proxy/0.log" Jul 06 02:12:22.063889 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:22.063860 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bdpq8_86824090-3c80-459c-97b6-4e34bcac19c3/kube-multus-additional-cni-plugins/0.log" Jul 06 02:12:22.083074 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:22.083046 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bdpq8_86824090-3c80-459c-97b6-4e34bcac19c3/egress-router-binary-copy/0.log" Jul 06 02:12:22.101606 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:22.101578 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bdpq8_86824090-3c80-459c-97b6-4e34bcac19c3/cni-plugins/0.log" Jul 06 02:12:22.120675 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:22.120649 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bdpq8_86824090-3c80-459c-97b6-4e34bcac19c3/bond-cni-plugin/0.log" Jul 06 02:12:22.140202 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:22.140174 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bdpq8_86824090-3c80-459c-97b6-4e34bcac19c3/routeoverride-cni/0.log" Jul 06 02:12:22.161811 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:22.161787 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bdpq8_86824090-3c80-459c-97b6-4e34bcac19c3/whereabouts-cni-bincopy/0.log" Jul 06 02:12:22.180982 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:22.180958 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bdpq8_86824090-3c80-459c-97b6-4e34bcac19c3/whereabouts-cni/0.log" Jul 06 02:12:22.396955 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:22.396929 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-xszvt_ee8fc4f9-9bc1-4b56-a435-f158b76784fb/kube-multus/0.log" Jul 06 02:12:22.500623 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:22.500597 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-qnp65_15c8ffd6-1e00-4975-b807-63e00ad36b44/network-metrics-daemon/0.log" Jul 06 02:12:22.517459 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:22.517410 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-qnp65_15c8ffd6-1e00-4975-b807-63e00ad36b44/kube-rbac-proxy/0.log" Jul 06 02:12:23.321142 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:23.321107 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2sd2m_04c8a1d0-bd84-4b0c-934b-efe85acfa81d/ovn-controller/0.log" Jul 06 02:12:23.343492 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:23.343461 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2sd2m_04c8a1d0-bd84-4b0c-934b-efe85acfa81d/ovn-acl-logging/0.log" Jul 06 02:12:23.374563 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:23.374532 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2sd2m_04c8a1d0-bd84-4b0c-934b-efe85acfa81d/kube-rbac-proxy-node/0.log" Jul 06 02:12:23.404391 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:23.404354 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2sd2m_04c8a1d0-bd84-4b0c-934b-efe85acfa81d/kube-rbac-proxy-ovn-metrics/0.log" Jul 06 02:12:23.424004 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:23.423970 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2sd2m_04c8a1d0-bd84-4b0c-934b-efe85acfa81d/northd/0.log" Jul 06 02:12:23.443079 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:23.443047 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2sd2m_04c8a1d0-bd84-4b0c-934b-efe85acfa81d/nbdb/0.log" Jul 06 02:12:23.462058 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:23.462030 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2sd2m_04c8a1d0-bd84-4b0c-934b-efe85acfa81d/sbdb/0.log" Jul 06 02:12:23.540329 ip-10-0-139-44 kubenswrapper[2580]: I0706 02:12:23.540293 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2sd2m_04c8a1d0-bd84-4b0c-934b-efe85acfa81d/ovnkube-controller/0.log"